Method and device for detecting fan-shaped area in ultrasonic image and medium

Abstract

The embodiment of the invention provides a method and a device for detecting a fan-shaped region in an ultrasonic image and a medium. The detection method comprises the following steps: preprocessing an ultrasonic image for training to obtain an original mask and a reverse mask of the ultrasonic image for training; respectively inputting the original mask and the reverse mask into an initial neural network model to obtain a first prediction mask corresponding to the original mask and a second prediction mask corresponding to the reverse mask; based on the difference between the original mask and the first prediction mask and the difference between the reverse mask and the second prediction mask, optimizing the initial neural network model to obtain a fan-shaped region detection model; and inputting a to-be-detected ultrasonic image into the fan-shaped region detection model, and preliminarily segmenting a fan-shaped region in the to-be-detected ultrasonic image. According to the method and the device provided by the embodiment of the invention, the detection accuracy of the fan-shaped region can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of ultrasonic inspection technology, and in particular to a detection method, a detection device, and a medium for a sector-shaped area in an ultrasonic image. Background Art

[0002] Ultrasound images have noise and artifacts. For example, speckle noise may be misjudged as a feature in a sector, or blur the boundary between the sector and the background. This makes it difficult for traditional image processing methods to extract and identify sector regions. For example, due to problems such as disconnected regions and inaccurate positioning of the maximum connected domain, traditional methods may not be able to effectively distinguish and extract sector regions, that is, the accuracy of sector region detection is low. Summary of the invention

[0003] In view of this, an embodiment of the present application provides a method, a detection device and a medium for detecting a fan-shaped area in an ultrasound image, which can improve the detection accuracy of the fan-shaped area.

[0004] In a first aspect, a method for detecting fan-shaped areas in an ultrasonic image is provided, comprising: preprocessing a training ultrasonic image to obtain an original mask and an inverted mask of the training ultrasonic image; inputting the original mask and the inverted mask of the training ultrasonic image into an initial neural network model, respectively, to obtain a first prediction mask corresponding to the original mask of the training ultrasonic image and a second prediction mask corresponding to the inverted mask; optimizing the initial neural network model based on the difference between the original mask of the training ultrasonic image and the first prediction mask and the difference between the inverted mask and the second prediction mask to obtain a fan-shaped area detection model; inputting an ultrasonic image to be detected into the fan-shaped area detection model to preliminarily segment the fan-shaped area in the ultrasonic image to be detected.

[0005] In this embodiment, the use of a powerful neural network model can effectively solve the problems of noise, artifacts and maximum connected domains in ultrasound images, and thus quickly and accurately identify the fan-shaped areas in ultrasound images. In addition, the fan-shaped area detection model obtained by training and optimizing the reverse attention mechanism can more accurately segment the boundary part of the fan-shaped area in the ultrasound image to be detected, emphasize the importance of the fan-shaped area boundary, and effectively solve the problem of fan-shaped boundary burrs, thereby improving the detection accuracy of the fan-shaped area in the ultrasound image.

[0006] In a possible implementation, the detection method further includes: using a traditional image processing algorithm to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation, and the traditional image processing algorithm includes a convex hull algorithm and / or a completion algorithm.

[0007] In this embodiment, in combination with the traditional image processing method, the completion algorithm and / or the convex hull algorithm are used on the accurate maximum connected domain to further quickly and accurately identify the fan-shaped area in the ultrasound image.

[0008] In one possible implementation, a traditional image processing algorithm is used to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation, including: using a convex hull algorithm to determine the boundary line of the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation; using a completion algorithm to complete the area within the boundary line; and again using the convex hull algorithm to process the fan-shaped area in the ultrasonic image to be detected obtained after the completion process.

[0009] In this embodiment, the convex hull algorithm is first used to find the boundary line of the fan-shaped area in the ultrasonic image to be detected based on the segmentation result of the fan-shaped area detection model, and then the completion algorithm is used to complete the area within the boundary line on the left and right, and the convex hull algorithm is used again to obtain the final fan-shaped area, which has a better detection result.

[0010] In one possible implementation, the ultrasonic image to be detected includes a first ultrasonic image to be detected and a second ultrasonic image to be detected, and a traditional image processing algorithm is used to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation, including: while the second ultrasonic image to be detected is input into the fan-shaped area detection model and the fan-shaped area in the second ultrasonic image to be detected is preliminarily segmented, the traditional image processing algorithm is used to process the fan-shaped area in the first ultrasonic image to be detected obtained by preliminary segmentation.

[0011] In this embodiment, different ultrasound images to be detected in the ultrasound video stream are processed in parallel using a pipeline, which can fully utilize the computing unit to improve the detection efficiency.

[0012] In a second aspect, a detection device for a fan-shaped area in an ultrasonic image is provided, comprising: a training unit, configured to preprocess a training ultrasonic image to obtain an original mask and an inverted mask of the training ultrasonic image, input the original mask and the inverted mask of the training ultrasonic image into an initial neural network model, respectively, to obtain a first prediction mask corresponding to the original mask of the training ultrasonic image and a second prediction mask corresponding to the inverted mask, and optimize the initial neural network model based on the difference between the original mask of the training ultrasonic image and the first prediction mask and the difference between the inverted mask and the second prediction mask, to obtain a fan-shaped area detection model;

[0013] The detection unit is used to input the ultrasonic image to be detected into the fan-shaped area detection model to preliminarily segment the fan-shaped area in the ultrasonic image to be detected.

[0014] In a possible implementation, the detection device further includes: a processing unit, which uses a traditional image processing algorithm to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation, and the traditional image processing algorithm includes a convex hull algorithm and / or a completion algorithm.

[0015] In one possible implementation, the processing unit is specifically used to: use a convex hull algorithm to determine the boundary line of the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation; use a completion algorithm to complete the area within the boundary line; and use the convex hull algorithm again to process the fan-shaped area in the ultrasonic image to be detected obtained after the completion process.

[0016] In a possible implementation, the ultrasonic image to be detected includes a first ultrasonic image to be detected and a second ultrasonic image to be detected, and the processing unit is specifically used to: when the detection unit inputs the second ultrasonic image to be detected into the fan-shaped area detection model, and preliminarily segments the fan-shaped area in the second ultrasonic image to be detected, use the traditional image processing algorithm to process the fan-shaped area in the first ultrasonic image to be detected obtained by the preliminarily segmented segmentation.

[0017] In a third aspect, a device for detecting a fan-shaped area in an ultrasound image is provided, comprising: at least one processor, and at least one memory communicatively connected to the at least one processor, wherein the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the detection method described in the first aspect and any possible implementation method of the first aspect.

[0018] Optionally, the at least one processor includes a central processing unit (CPU) and a graphics processing unit (GPU), wherein the GPU is used to implement the function of a deep neural network model, for example, corresponding to the training unit and the detection unit in the detection device provided in the second aspect and any possible implementation thereof. The CPU is used to implement the function of traditional image processing, for example, corresponding to the processing unit in the detection device provided in the second aspect and any possible implementation thereof.

[0019] In a fourth aspect, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute the detection method as described in the first aspect and any possible implementation method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the drawings without paying creative work.

[0021] Figure 1 A schematic block diagram of a method for detecting a fan-shaped area in an ultrasound image according to an embodiment of the present application is shown.

[0022] Figure 2 A schematic diagram of obtaining the original mask and the inverted mask according to an embodiment of the present application is shown.

[0023] Figure 3 A supervision schematic diagram using an original mask and an inverted mask according to an embodiment of the present application is shown.

[0024] Figure 4 A schematic diagram of a sector-shaped area preliminarily segmented by a sector-shaped area detection model according to an embodiment of the present application is shown.

[0025] Figure 5 A schematic diagram of a sector-shaped area obtained by a traditional image processing method in an embodiment of the present application is shown.

[0026] Figure 6 A schematic block diagram showing the parallel processing of different ultrasound images to be detected according to an embodiment of the present application is shown.

[0027] Figure 7 A schematic block diagram of a device for detecting a fan-shaped area in an ultrasound image according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] Due to its low cost, high safety and portability, ultrasound technology is widely used in prenatal examinations and has become an effective tool for preventing birth defects. However, current ultrasound examinations are still at the traditional stage, including moving the ultrasound probe, manually selecting standard sections, manually outlining key anatomical structures, and manual diagnosis. This is highly dependent on the doctor's experience and ability, and the homogeneity of the examination results is low. It takes up to 30 to 40 minutes and the workload of doctors is heavy.

[0030] In recent years, with the development of artificial intelligence technology, especially deep learning technology, more and more researchers are committed to studying the automation and intelligence of the entire process of ultrasound examination. This includes automatic recognition of standard ultrasound sections, automatic segmentation of key anatomical structures, automatic measurement of growth parameters, and auxiliary diagnosis of diseases. The realization of this intelligent ultrasound technology is expected to shorten the examination time to 10 to 15 minutes and reduce the workload of doctors. Among these tasks, the automatic recognition of standard sections is particularly critical and basic, which provides the necessary basic conditions for the implementation of subsequent work. This task includes judging whether the image magnification ratio is appropriate, whether key anatomical structures exist, and whether the image gain is appropriate.

[0031] The magnification ratio of ultrasound images is usually determined by the size of the region of interest (ROI) of the ultrasound section and the size of the fan-shaped area in the ultrasound image. Currently, many studies have shown that deep neural networks can better detect the ROI of the ultrasound section, but there are relatively few studies on how to quickly and accurately detect the fan-shaped area in the ultrasound image.

[0032] Ultrasound images themselves have characteristics such as noise and artifacts, which makes traditional image processing methods face great challenges in extracting and identifying sector-shaped areas. For example, due to problems such as regional disconnection and inaccurate positioning of the maximum connected domain, traditional methods may not be able to effectively distinguish and extract sector-shaped areas. Secondly, affected by the burrs on the edges of the sector-shaped areas, traditional image processing methods may cause the identified sector-shaped areas to exceed the actual area. In addition, other features contained in ultrasound images, such as menu bar information, ultrasound machine logos, etc., also bring great difficulties to traditional image processing methods.

[0033] Therefore, an embodiment of the present application provides a method for detecting a fan-shaped area in an ultrasound image, which can improve the detection accuracy of the fan-shaped area.

[0034] Figure 1 is a schematic block diagram of a method for detecting a fan-shaped area in an ultrasound image provided by an embodiment of the present application. Figure 1 As shown, the detection method 100 includes part or all of the following contents.

[0035] S110, preprocessing the training ultrasound image to obtain an original mask and a reverse mask of the training ultrasound image.

[0036] S120, inputting the original mask and the inverted mask of the training ultrasound image into the initial neural network model respectively, so as to obtain a first prediction mask corresponding to the original mask of the training ultrasound image and a second prediction mask corresponding to the inverted mask;

[0037] S130, optimizing the initial neural network model based on the difference between the original mask of the training ultrasound image and the first prediction mask and the difference between the inverted mask and the second prediction mask to obtain a fan-shaped area detection model;

[0038] S140, inputting the ultrasonic image to be detected into the fan-shaped region detection model, and preliminarily segmenting the fan-shaped region in the ultrasonic image to be detected.

[0039] Optionally, the detection method 100 may be executed in a processor, for example, in a CPU in a computer device. For another example, a portion is executed in a GPU in a computer device, and another portion is executed in a CPU in a computer device.

[0040] In an embodiment of the present application, the detection method 100 includes at least two processes, a training process of a fan-shaped area detection model and a process of segmenting the fan-shaped area in the ultrasound image to be detected using the fan-shaped area detection model.

[0041] In the training process of the sector area detection model, it is first necessary to collect a sufficient number of ultrasound images as training data. For example, cooperation can be established with medical institutions such as hospitals and clinics to obtain a large number of ultrasound images.

[0042] After obtaining a large number of training ultrasound images, these training ultrasound images can be preprocessed. For example, if these ultrasound images are in color, they can be converted into grayscale images first. Further, the grayscale image can be thresholded to obtain a preliminary result of the original mask. For example, a simple fixed threshold method or a self-threshold method can be used. Optionally, in order to further optimize the original mask, morphological operations can also be performed. For example, an erosion operation is used to remove small noise points or an expansion operation is used to fill small holes. In some other embodiments, the automatically generated mask may have some inaccuracies, and the initial result of the original mask can also be manually corrected by a manual annotation tool to obtain the final original mask.

[0043] Once the original mask is obtained, it is relatively simple to obtain the inverted mask. For example, if the original mask is a binary mask (containing only 0 and 1), then obtaining the inverted mask is the most direct. Specifically, if the value of the original mask at a certain pixel position is 1, it means that the area corresponding to the pixel is selected in the original mask (possibly a fan-shaped area), then the value of the inverted mask at this position is 0, indicating that the area corresponding to the pixel is regarded as a non-selected area in the inverted mask (possibly a background area). For another example, when the value of the original mask is not a simple binary, but in an interval (for example, a probability mask indicating that each pixel belongs to a certain area generated by a sigmoid function, etc.), the inverted mask can also be implemented by simple subtraction. Specifically, if the value of the original mask at a certain pixel position is 0.7, indicating that the pixel has a high probability of belonging to a specific area (such as a fan-shaped area), then the value of the inverted mask at this position is 0.3, indicating that the pixel has a relatively low probability of belonging to this specific area, but is more likely to belong to other areas (such as background areas).

[0044] Figure 2 FIG. 1 shows a schematic diagram of the process from an original ultrasound image, to annotating the original ultrasound image, to obtaining its original mask and inverting the mask. Figure 2 It can be seen that the original mask focuses on the content within the fan-shaped area, while the inverted mask focuses on the content outside the fan-shaped area.

[0045] After the processor obtains the ultrasound image to be processed, it first selects a suitable deep neural network model. For example, convolutional neural network, recurrent neural network, long short-term memory network, etc. Take LeNet as an example, it contains convolutional layer, pooling layer and fully connected layer. First, determine the input layer size of the network. For ultrasound images, it needs to be set according to the actual size of the image (such as width, height and number of channels). For example, the ultrasound image is a grayscale image with 1 channel and a size of 200×200 pixels, then the shape of the input layer can be set to (1,200,200). Again, define the convolution layer, select the appropriate convolution kernel size (such as 3×3 or 5×5), the number of convolution kernels (determines the number of output channels) and the step size. For example, set the convolution kernel size of the first convolution layer to 3×3, the number to 16, and the step size to 1. In this way, when the ultrasound image is input to this convolution layer, the convolution kernel will slide on the image for convolution operation to extract local features. The pooling layer can choose maximum pooling or average pooling to reduce the amount of data while retaining important features. For example, adding a 2×2 max pooling layer after the convolution layer will reduce the size of the feature map after convolution by half. In addition, the convolution kernels of the convolution layer and the weights of the fully connected layer need to be initialized. Common initialization methods include random initialization (such as normal distribution or uniform distribution initialization) and pre-training initialization. If random initialization is used, the normal distribution can be used to initialize the weights of the convolution kernels.

[0046] After the initial neural network model is obtained, the original mask and the inverted mask of the training ultrasound image can be respectively input into the initial neural network model for supervised training. It should be noted that the inputting of the original mask into the initial neural network model for supervised training and the inputting of the inverted mask into the initial neural network model for supervised training can be performed sequentially or simultaneously, and the embodiments of the present application are not limited thereto.

[0047] When using the original mask and the inverted mask, they can be used as different training samples to input corresponding labels. For example, for the sample corresponding to the original mask, its label is the original mask itself; for the sample corresponding to the inverted mask, its label is the inverted mask. This allows the model to learn the characteristics of the fan-shaped area and the non-fan-shaped area.

[0048] It should be noted that the input of the initial neural network model can be the original mask and the inversion mask, that is, the preprocessing of the training ultrasound image and the initial neural network model are different functional modules. In other embodiments, the input of the initial neural network model can also be the training ultrasound image. The initial neural network model can first preprocess the training ultrasound image to obtain the original mask and the inversion mask, and then the initial neural network model obtains the first prediction mask corresponding to the original mask and the second prediction mask corresponding to the inversion mask based on the original mask and the inversion mask. For example, the initial neural network model may include an encoder and a decoder, the encoder is used to preprocess the training ultrasound image, and the decoder is used to map the original mask and the inversion mask obtained after the preprocessing back to the original image size for segmentation prediction. That is, after the training ultrasound image undergoes the convolution, pooling and upsampling operations of each layer of the initial neural network model, the first prediction mask and the second prediction mask can be obtained. Among them, the first prediction mask can be understood as the segmentation result of the fan-shaped area of ​​the training ultrasound image, and the second prediction mask can be understood as the segmentation result of the non-fan-shaped area of ​​the training ultrasound image.

[0049] After obtaining the first prediction mask and the second prediction mask, the initial neural network model can be further optimized to obtain the final sector area detection model.

[0050] Specifically, a loss function can be defined to quantify the difference between the first prediction mask and the original mask and the difference between the second prediction mask and the inverted mask. For example, the difference between the original mask and the first prediction mask at each pixel and the difference between the inverted mask and the second prediction mask at each pixel can be measured based on the pixel cross entropy loss function, so that the initial neural network model can minimize this difference during the training process. Next, the parameters of the model can also be updated according to the difference measured by the loss function (the difference between the original mask and the first prediction mask, and the difference between the inverted mask and the second prediction mask), so that the model can further gradually reduce these differences during the subsequent training process. During the training process, in addition to the value of the loss function, other indicators can also be monitored to evaluate the performance of the model, such as accuracy, recall, precision, and F1-score. By monitoring these indicators, the performance changes of the model during the training process can be understood, especially the segmentation effect of the model on the fan-shaped area and the non-fan-shaped area. The training strategy can also be adjusted according to the changes in the monitoring indicators. For example, if the value of the loss function no longer decreases or the monitoring indicator no longer increases, it may be that overfitting or slow model convergence occurs. For overfitting, regularization methods such as L1 and L2 regularization, Dropout, etc. can be used. L1 regularization will make some parameters of the model become 0, thereby reducing the complexity of the model; L2 regularization will reduce the sum of squares of the model parameters, which plays a role in smoothing the model. Dropout is to randomly discard a part of neurons during the training process to prevent excessive dependence between neurons. If the model converges slowly, you can try to adjust the learning rate. Increasing the learning rate may speed up the convergence, but it may also cause the model to fail to converge to the optimal solution; reducing the learning rate can make the model converge more stably, but the convergence speed will be slower. After supervised training, the final fan-shaped area detection model can be obtained.

[0051] Figure 3 A schematic block diagram of supervised training of an initial neural network model using original masks and inverted masks is shown.

[0052] After the fan-shaped region detection model is trained, the fan-shaped region detection model is used to perform preliminary segmentation on the ultrasound image to be detected. Specifically, first, ensure that the format of the ultrasound image to be detected is consistent with the image format used when the fan-shaped region detection model is trained. This may include matching in terms of image size, number of channels, etc. For example, if an ultrasound image with a size of 256×256 pixels and a single channel (grayscale image) is used for model training, the image to be detected needs to be converted to the same size and channel format. Secondly, the image can be normalized so that the pixel values ​​of the image are in a suitable range. A common normalization method is to normalize the pixel values ​​to the [0,1] interval or to have zero mean and unit variance. For ultrasound images, assuming that the pixel value range is [0,255], it can be normalized to [0,1] by a simple linear transformation. Then, the preprocessed ultrasound image to be detected is input into the fan-shaped region detection model. The model performs a forward propagation operation on the input image to obtain the prediction result. In the fan-shaped area detection model, the prediction result is usually a segmentation mask of the same size as the input image (or the same size after upsampling and other operations). Each pixel value in this segmentation mask represents the probability that the pixel belongs to the fan-shaped area (if it is in probability form) or directly a binary segmentation result (0 represents a non-fan-shaped area, and 1 represents a fan-shaped area). For example, in a fan-shaped area detection model based on a convolutional neural network, after a series of convolution, pooling, and upsampling operations, a tensor of shape [1, H, W] (assuming the batch size is 1, H is the image height, and W is the image width) is output as the prediction result.

[0053] In some embodiments, if the prediction result is a segmentation mask in the form of probability, threshold processing is required to obtain the final binary segmentation result, and then the preliminary segmentation result of the fan-shaped area is obtained. In other embodiments, if the model outputs the binary segmentation result, it can be directly used as the preliminary segmented fan-shaped area.

[0054] In this embodiment, the use of a powerful neural network model can effectively solve the problems of noise, artifacts and maximum connected domains in ultrasound images, and thus quickly and accurately identify the fan-shaped areas in ultrasound images. In addition, the fan-shaped area detection model obtained by training and optimizing the reverse attention mechanism can more accurately segment the boundary part of the fan-shaped area in the ultrasound image to be detected, emphasize the importance of the fan-shaped area boundary, and effectively solve the problem of fan-shaped boundary burrs, thereby improving the detection accuracy of the fan-shaped area in the ultrasound image.

[0055] Optionally, in an embodiment of the present application, the detection method further includes: using a traditional image processing algorithm to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation, and the traditional image processing algorithm includes a convex hull algorithm and / or a completion algorithm.

[0056] Since the ultrasound images are initially segmented using a deep neural network model, Figure 4 As shown, some information in the fan-shaped area is missing. Therefore, the symmetry and other characteristics of the fan-shaped area can be further utilized, and the fan-shaped area in the ultrasound image obtained by preliminary segmentation can be processed accordingly using traditional image processing methods, thereby improving the detection accuracy of the fan-shaped area in the ultrasound image.

[0057] In some embodiments, the use of a traditional image processing algorithm to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation may include: using a convex hull algorithm to determine the boundary line of the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation; using a completion algorithm to complete the area within the boundary line; and again using the convex hull algorithm to process the fan-shaped area in the ultrasonic image to be detected obtained after the completion process.

[0058] The convex hull refers to the smallest convex polygon that contains a given set of plane points. For the boundary point set in the fan-shaped area obtained by the initial segmentation, the convex hull algorithm can find a convex polygon to approximate the boundary of this fan-shaped area. In computational geometry, commonly used convex hull algorithms are Graham scanning and Jarvis stepping. The basic idea of ​​the Graham scanning method is to first find a starting point (usually the leftmost or bottommost point), and then sort the remaining points according to the polar angles and scan these points to construct the convex hull. The Jarvis stepping method (also known as the gift wrapping algorithm) starts from an outermost point and constructs a convex hull by continuously looking for the next outermost point.

[0059] The completion algorithm is mainly used to fill the holes or missing parts in the fan-shaped area obtained by the initial segmentation. A common completion algorithm is based on the expansion and erosion operations in morphological operations. The expansion operation can expand the boundary of the object (here is the fan-shaped area) outward and fill some small holes; the erosion operation shrinks the boundary inward. By reasonably combining the expansion and erosion operations, the fan-shaped area can be completed. In addition, there are completion methods based on algorithms such as region growing. The region growing algorithm starts from a seed point (which can be a known point in the fan-shaped area) and gradually merges adjacent pixels into the area according to certain similarity criteria (such as the similarity of pixel grayscale values) until the stopping condition is met (such as there are no adjacent pixels that meet the conditions).

[0060] For example, the convex hull algorithm can be used to find the boundary points first, and then two boundary lines that can contain all the boundary points can be found through the boundary points, and the slope K of the two boundary lines can be verified to be almost 0. On this basis, the center line of the sector area can be found, and the area within the boundary line can be completed on the left and right through the center line of the sector area, and the convex hull algorithm can be used again to obtain the final sector area, such as Figure 5 shown.

[0061] In this embodiment, the convex hull algorithm is first used to find the boundary line of the fan-shaped area in the ultrasonic image to be detected based on the segmentation result of the fan-shaped area detection model, and then the completion algorithm is used to complete the area within the boundary line on the left and right, and the convex hull algorithm is used again to obtain the final fan-shaped area, which has a better detection result.

[0062] Optionally, the ultrasonic image to be detected includes a first ultrasonic image to be detected and a second ultrasonic image to be detected, and the traditional image processing algorithm is used to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation, including: while inputting the second ultrasonic image to be detected into the fan-shaped area detection model and preliminarily segmenting the fan-shaped area in the second ultrasonic image to be detected, the traditional image processing algorithm is used to process the fan-shaped area in the first ultrasonic image to be detected obtained by preliminary segmentation.

[0063] Usually, an ultrasound video stream includes multiple ultrasound images. In the embodiment of the present application, the preliminary segmentation of the fan-shaped area using the fan-shaped area detection model and the processing of the fan-shaped area using the traditional image processing method are respectively executed in different functional modules or different processors. At this time, the pipeline parallel working mode can be adopted. While the traditional image processing is performed on the previous ultrasound image to be detected, the fan-shaped area of ​​the next ultrasound image to be detected is preliminarily segmented using the fan-shaped area detection model, thereby improving the detection efficiency of the fan-shaped area in the ultrasound image. The pipeline parallel mode is as follows: Figure 6 As shown. Among them, while the first ultrasonic image to be detected is processed by the traditional image processing algorithm, the second ultrasonic image to be detected is preliminarily segmented by using the fan-shaped area detection model; while the second ultrasonic image to be detected is completed by using the traditional image processing algorithm, the third ultrasonic image to be detected is preliminarily segmented by using the fan-shaped area detection model.

[0064] Optionally, the fan-shaped area detection model can be a deep neural network model, for example, a convolutional neural network model.

[0065] It should be noted that the reverse attention mechanism can be added before any network layer in the deep neural network model, or the reverse attention mechanism can be added between any two network layers in the multiple network layers in the deep neural network model. The embodiment of the present application does not limit the timing of adding the reverse attention mechanism.

[0066] The present application also provides a device for detecting a sector-shaped area in an ultrasound image. Figure 7 As shown, the detection device 200 includes part or all of the following contents.

[0067] The training unit 210 is used to preprocess the training ultrasound image to obtain an original mask and an inverted mask of the training ultrasound image, input the original mask and the inverted mask of the training ultrasound image into an initial neural network model respectively to obtain a first prediction mask corresponding to the original mask of the training ultrasound image and a second prediction mask corresponding to the inverted mask, and optimize the initial neural network model based on the difference between the original mask of the training ultrasound image and the first prediction mask and the difference between the inverted mask and the second prediction mask to obtain a sector area detection model;

[0068] The detection unit 220 is used to input the ultrasonic image to be detected into the fan-shaped area detection model to preliminarily segment the fan-shaped area in the ultrasonic image to be detected.

[0069] Optionally, the detection device further includes: a processing unit, which uses a traditional image processing algorithm to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation, and the traditional image processing algorithm includes a convex hull algorithm and / or a completion algorithm.

[0070] Optionally, the processing unit is specifically used to: use a convex hull algorithm to determine the boundary line of the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation; use a completion algorithm to complete the area within the boundary line; and use the convex hull algorithm again to process the fan-shaped area in the ultrasonic image to be detected obtained after the completion process.

[0071] Optionally, the ultrasonic image to be detected includes a first ultrasonic image to be detected and a second ultrasonic image to be detected, and the processing unit is specifically used to: when the detection unit inputs the second ultrasonic image to be detected into the fan-shaped area detection model and preliminarily segments the fan-shaped area in the second ultrasonic image to be detected, use the traditional image processing algorithm to process the fan-shaped area in the first ultrasonic image to be detected obtained by the preliminary segmentation.

[0072] Based on the same idea, the embodiment of the present application also provides another detection device for fan-shaped areas in ultrasound images. The detection device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the program instructions are executed by the processor, the detection device executes the above-mentioned various method embodiments.

[0073] It should be noted that the detailed contents of the device side embodiment can be found in the method side embodiment, and for the sake of brevity, they will not be repeated here.

[0074] Based on the same idea, an embodiment of the present application also provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned various method embodiments.

[0075] Although the present application has been described with reference to preferred embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, the various technical features mentioned in the various embodiments may be combined in any manner as long as there are no structural conflicts. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for detecting a sector area in an ultrasonic image, characterized in that: include: Preprocessing the training ultrasound image to obtain an original mask and a reverse mask of the training ultrasound image; Inputting the original mask and the inverted mask into an initial neural network model respectively to obtain a first prediction mask corresponding to the original mask and a second prediction mask corresponding to the inverted mask; Based on the difference between the original mask and the first predicted mask and the difference between the inverted mask and the second predicted mask, optimizing the initial neural network model to obtain a sector area detection model; The ultrasonic image to be detected is input into the fan-shaped region detection model to preliminarily segment the fan-shaped region in the ultrasonic image to be detected.

2. The detection method according to claim 1, characterized in that: The detection method further comprises: The fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation is processed using a traditional image processing algorithm, wherein the traditional image processing algorithm includes a convex hull algorithm and / or a completion algorithm.

3. The detection method according to claim 2, characterized in that: The method of using a traditional image processing algorithm to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation includes: Using the convex hull algorithm, determining the boundary line of the sector-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation; Using the completion algorithm, the area within the boundary line is completed; The convex hull algorithm is used again to process the fan-shaped area in the ultrasonic image to be detected obtained after the completion process.

4. The detection method according to claim 2, characterized in that: The ultrasonic image to be detected includes a first ultrasonic image to be detected and a second ultrasonic image to be detected, and the conventional image processing algorithm is used to process the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation, including: When the second ultrasonic image to be detected is input into the fan-shaped area detection model and the fan-shaped area in the second ultrasonic image to be detected is preliminarily segmented, the traditional image processing algorithm is used to process the fan-shaped area in the first ultrasonic image to be detected obtained by the preliminarily segmented area.

5. A device for detecting a sector-shaped area in an ultrasonic image, characterized in that: include: A training unit is used to preprocess the training ultrasound image to obtain an original mask and an inverted mask of the training ultrasound image, input the original mask and the inverted mask into an initial neural network model respectively to obtain a first prediction mask corresponding to the original mask and a second prediction mask corresponding to the inverted mask, and optimize the initial neural network model based on a difference between the original mask and the first prediction mask and a difference between the inverted mask and the second prediction mask to obtain a sector area detection model; The detection unit is used to input the ultrasonic image to be detected into the fan-shaped area detection model to preliminarily segment the fan-shaped area in the ultrasonic image to be detected.

6. The detection device according to claim 5, characterized in that: The detection device also includes: The processing unit processes the fan-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation using a traditional image processing algorithm, wherein the traditional image processing algorithm includes a convex hull algorithm and / or a completion algorithm.

7. The detection device according to claim 6, characterized in that: The processing unit is specifically used for: Using the convex hull algorithm, determining the boundary line of the sector-shaped area in the ultrasonic image to be detected obtained by preliminary segmentation; Using the completion algorithm, the area within the boundary line is completed; The convex hull algorithm is used again to process the fan-shaped area in the ultrasonic image to be detected obtained after the completion process.

8. The detection device according to claim 6, characterized in that: The ultrasonic image to be detected includes a first ultrasonic image to be detected and a second ultrasonic image to be detected, and the processing unit is specifically used for: When the detection unit inputs the second ultrasonic image to be detected into the fan-shaped area detection model and preliminarily segments the fan-shaped area in the second ultrasonic image to be detected, the traditional image processing algorithm is used to process the fan-shaped area in the first ultrasonic image to be detected obtained by the preliminarily segmented area.

9. A device for detecting a sector-shaped area in an ultrasonic image, characterized in that: include: At least one processor, and at least one memory communicatively connected to the at least one processor, wherein the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the detection method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the detection method according to any one of claims 1 to 4.

Images