Breast ultrasound image segmentation method and device

Abstract

The present invention provides a method and apparatus for segmenting breast ultrasound images. The method comprises: obtaining an ultrasound image of a subject's breast region, the ultrasound image containing a breast lesion; identifying the type of BI-RADS feature of the breast lesion based on the ultrasound image, the BI-RADS feature including at least one of shape feature, direction feature, edge feature, internal echo feature, rear echo feature, calcification feature, and blood flow feature; determining a target segmentation model corresponding to the type of BI-RADS feature from a plurality of segmentation models corresponding to each type of preset BI-RADS feature; and performing breast lesion segmentation on the ultrasound image using the target segmentation model to segment the breast lesion region from the ultrasound image. The characteristics of the breast lesion are fully considered during segmentation, and the accuracy of breast lesion region segmentation is improved by using the target segmentation model corresponding to the type of BI-RADS feature to segment the ultrasound image.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of medical ultrasound technology, and particularly to a breast ultrasound image segmentation method and device. Background Art

[0002] Breast ultrasound images can clearly demonstrate the location, morphology, internal structure, and adjacent tissue changes of all layers of breast soft tissue and lesions therein. They are economical, convenient, non-invasive, painless, radioactive, and highly reproducible, making them an important method for breast examination. With the continuous development of computer science and technology, computer-aided diagnosis (CAD) systems are increasingly being used for intelligent diagnosis using breast ultrasound images. The accuracy of breast lesion segmentation significantly impacts the accuracy of intelligent diagnostic results. Therefore, accurately segmenting breast lesion regions from breast ultrasound images is of great significance.

[0003] like Figure 1 As shown, the morphology of breast lesions is as follows Figure 1 The shapes shown in (a) and (b) are regular, the boundaries are smooth and clear, and there are also Figure 1 The irregular shapes and unclear boundaries shown in (c), (d) and (f) are also Figure 1 Figures (e) and (f) show how the posterior echo attenuation creates acoustic shadowing, making it difficult to discern the lesion's boundaries. Breast lesions are complex, varied, and diverse in appearance. This, combined with interference from artifacts and acoustic shadowing, makes accurately segmenting breast lesions from breast ultrasound images a challenging task. Existing methods for segmenting breast lesions from breast ultrasound images using fixed models or algorithms suffer from low accuracy, which compromises the accuracy of intelligent diagnostics in CAD systems. Summary of the Invention

[0004] The embodiments of the present invention provide a method and device for segmenting breast ultrasound images, which are used to solve the problem of low accuracy in segmenting breast lesion areas in existing methods.

[0005] In a first aspect, an embodiment of the present invention provides a breast ultrasound image segmentation method, comprising:

[0006] Acquiring an ultrasound image of the subject's breast area, wherein the ultrasound image includes a breast lesion;

[0007] Identifying the type of Breast Imaging Reporting and Data System (BI-RADS) features of breast lesions based on ultrasound images, where the BI-RADS features include at least one of shape features, orientation features, edge features, internal echo features, posterior echo features, calcification features, and blood flow features;

[0008] Determining a target segmentation model corresponding to the type of BI-RADS feature from a plurality of segmentation models corresponding to each type of preset BI-RADS feature;

[0009] The target segmentation model is used to segment breast lesions in ultrasound images and segment the breast lesion area from the ultrasound image.

[0010] In a second aspect, an embodiment of the present invention provides a breast ultrasound image segmentation method, comprising:

[0011] Acquiring an ultrasound image of the subject's breast area, wherein the ultrasound image includes a breast lesion;

[0012] Identify BI-RADS classification of breast lesions based on ultrasound images;

[0013] determining a target segmentation model corresponding to the BI-RADS grade from a preset first segmentation model, a second segmentation model, a third segmentation model, a fourth segmentation model, a fifth segmentation model, a sixth segmentation model, and a seventh segmentation model, wherein the first segmentation model, the second segmentation model, the third segmentation model, the fourth segmentation model, the fifth segmentation model, the sixth segmentation model, and the seventh segmentation model are used to segment ultrasound images with BI-RADS grades of 2, 3, 4a, 4b, 4c, 5, and 6, respectively;

[0014] The target segmentation model is used to segment breast lesions in ultrasound images and segment the breast lesion area from the ultrasound image.

[0015] In a third aspect, an embodiment of the present invention provides a breast ultrasound image segmentation method, comprising:

[0016] Acquiring an ultrasound image of the subject's breast area, wherein the ultrasound image includes a breast lesion;

[0017] determining, based on the ultrasound image, a classification probability of each type of BI-RADS feature of the breast lesion, where the BI-RADS feature includes at least one of a shape feature, a direction feature, an edge feature, an internal echo feature, a rear echo feature, a calcification feature, and a blood flow feature;

[0018] Multiple segmentation models corresponding to different types of BI-RADS features are used to segment breast lesions in ultrasound images, and multiple breast lesion segmentation results corresponding to different types of BI-RADS features are obtained.

[0019] According to the classification probability, multiple breast lesion segmentation results corresponding to each type of BI-RADS features are fused to obtain the breast lesion area in the ultrasound image.

[0020] In a fourth aspect, an embodiment of the present invention provides a breast ultrasound image segmentation method, comprising:

[0021] Acquiring an ultrasound image of the subject's breast area, wherein the ultrasound image includes a breast lesion;

[0022] The probability of a breast lesion belonging to each BI-RADS grade based on ultrasound images;

[0023] The first segmentation model, the second segmentation model, the third segmentation model, the fourth segmentation model, the fifth segmentation model, the sixth segmentation model, and the seventh segmentation model are used to segment the ultrasound image for breast lesions, respectively, to obtain multiple breast lesion segmentation results corresponding to grade 2, grade 3, grade 4a, grade 4b, grade 4c, grade 5, and grade 6, respectively. The first segmentation model, the second segmentation model, the third segmentation model, the fourth segmentation model, the fifth segmentation model, the sixth segmentation model, and the seventh segmentation model are used to segment the ultrasound image with BI-RADS grade 2, grade 3, grade 4a, grade 4b, grade 4c, grade 5, and grade 6, respectively;

[0024] The multiple breast lesion segmentation results are fused according to the probability that the breast lesion belongs to each BI-RADS grade to obtain the breast lesion area in the ultrasound image.

[0025] In a fifth aspect, an embodiment of the present invention provides an ultrasonic imaging device, comprising:

[0026] Ultrasound probe;

[0027] The transmitting circuit is used to output the corresponding transmitting sequence to the ultrasonic probe according to the set mode, so as to control the ultrasonic probe to transmit the corresponding ultrasonic wave;

[0028] A receiving circuit, used for receiving the ultrasonic echo signal output by the ultrasonic probe and outputting ultrasonic echo data;

[0029] A display for outputting visual information;

[0030] A processor is configured to execute the breast ultrasound image segmentation method described in any one of the above embodiments.

[0031] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the breast ultrasound image segmentation method as described in any of the above embodiments.

[0032] The breast ultrasound image segmentation method and device provided by the embodiments of the present invention obtain an ultrasound image of the subject's breast region, identify the type of BI-RADS feature of the breast lesion based on the ultrasound image, then determine a target segmentation model corresponding to the type of BI-RADS feature from multiple segmentation models corresponding to each type of preset BI-RADS feature, and finally use the target segmentation model to segment the breast lesion in the ultrasound image. This enables targeted segmentation of breast lesions with different manifestations using different segmentation models. Because the characteristics of the breast lesions are fully considered during segmentation, the accuracy of breast lesion region segmentation can be improved, thereby helping to improve the accuracy of intelligent diagnosis in CAD systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a breast lesion region provided by one embodiment of the present invention;

[0034] Figure 2 A structural block diagram of an ultrasonic imaging device provided in one embodiment of the present invention;

[0035] Figure 3 A flowchart of a breast ultrasound image segmentation method provided by one embodiment of the present invention;

[0036] Figure 4 A flowchart of a breast ultrasound image segmentation method provided by another embodiment of the present invention;

[0037] Figure 5 A flowchart of a breast ultrasound image segmentation method provided by another embodiment of the present invention;

[0038] Figure 6 This is a flowchart of a breast ultrasound image segmentation method provided by yet another embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present application to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted in different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification. This is to avoid the core portion of the present application being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They will fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0040] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0041] The serial numbers assigned to components herein, such as "first," "second," etc., are used solely to distinguish the objects being described and do not convey any sequential or technical meaning. References to "connection" and "coupling" herein, unless otherwise specified, include both direct and indirect connections (couplings).

[0042] like Figure 2 As shown, the ultrasound imaging device provided by the present invention may include: an ultrasound probe 20, a transmitting / receiving circuit 30 (i.e., a transmitting circuit 310 and a receiving circuit 320), a beamforming module 40, an IQ demodulation module 50, a memory 60, a processor 70, and a human-computer interaction device. The processor 70 may include a control module 710 and an image processing module 720.

[0043] The ultrasound probe 20 includes a transducer (not shown) composed of multiple array elements arranged in an array. The array elements can be arranged in a row to form a linear array, or in a two-dimensional matrix to form a planar array. The array elements can also form a convex array. The array elements are used to transmit ultrasonic beams based on excitation electrical signals or to convert received ultrasonic beams into electrical signals. Therefore, each array element can be used to convert electrical pulse signals into and from ultrasonic beams, thereby transmitting ultrasonic waves to a target area of ​​human tissue (e.g., the breast area containing a breast lesion in this embodiment) and receiving echoes of ultrasonic waves reflected from the tissue. During ultrasonic testing, the transmitting circuit 310 and the receiving circuit 320 can control which array elements are used to transmit and which are used to receive ultrasonic beams, or control the time slots used to transmit and receive ultrasonic beams. The array elements involved in ultrasonic transmission can be excited by electrical signals simultaneously, thereby transmitting ultrasonic waves simultaneously; or they can be excited by multiple electrical signals with a certain time interval, thereby continuously transmitting ultrasonic waves with a certain time interval.

[0044] In this embodiment, the user moves the ultrasound probe 20 to select a suitable position and angle to transmit ultrasound to the breast area 10 and receive the echo of the ultrasound returned by the breast area 10, obtains and outputs the electrical signal of the echo, and the electrical signal of the echo is a channel analog electrical signal formed by the receiving array element as a channel, which carries amplitude information, frequency information and time information.

[0045] The transmitting circuit 310 is configured to generate a transmit sequence under the control of the control module 710 of the processor 70. The transmit sequence is used to control some or all of the multiple array elements to transmit ultrasound waves toward biological tissue. Transmit sequence parameters include the array element positions, the number of array elements, and ultrasound beam transmission parameters (e.g., amplitude, frequency, number of transmissions, transmission interval, transmission angle, waveform, focal position, etc.). In some cases, the transmitting circuit 310 is further configured to phase-delay the transmitted beam so that different transmitting array elements transmit ultrasound waves at different times, allowing each transmitted ultrasound beam to be focused on a predetermined region of interest. Transmit sequence parameters may vary for different operating modes, such as B-image mode, C-image mode, and D-image mode (Doppler mode). After the echo signals are received by the receiving circuit 320 and processed by subsequent modules and corresponding algorithms, a B image reflecting tissue anatomical structure, a C image reflecting tissue anatomical structure and blood flow information, and a D image reflecting Doppler spectrum images can be generated.

[0046] The receiving circuit 320 is used to receive and process the electrical signals of ultrasonic echoes from the ultrasonic probe 20. The receiving circuit 320 may include one or more amplifiers, analog-to-digital converters (ADCs), and other components. The amplifiers are used to amplify the received electrical signals of ultrasonic echoes after appropriate gain compensation, and the ADCs are used to sample the analog echo signals at predetermined intervals, converting them into digitized signals. The digitized echo signals still retain amplitude, frequency, and phase information. The data output by the receiving circuit 320 can be sent to the beamforming module 40 for processing or to the memory 60 for storage.

[0047] The beamforming module 40 is signal-connected to the receiving circuit 320 and is used to perform beamforming processing, such as delay and weighted summation, on the signal output by the receiving circuit 320. Because the distances between the ultrasound receiving points in the measured tissue and the receiving array elements vary, the channel data of the same receiving point output by different receiving array elements have different delays. This requires delay processing, phase alignment, and weighted summation of the different channel data from the same receiving point to obtain beamformed ultrasound image data. The ultrasound image data output by the beamforming module 40 is also called radio frequency data (RF data). The beamforming module 40 outputs the RF data to the IQ demodulation module 50. In some embodiments, the beamforming module 40 may also output the RF data to the memory 60 for caching or storage, or directly output the RF data to the image processing module 720 of the processor 70 for image processing.

[0048] The beamforming module 40 may perform the aforementioned functions in hardware, firmware, or software. For example, the beamforming module 40 may include a central controller circuit (CPU), one or more microprocessor chips, or any other electronic components capable of processing input data according to specific logic instructions. When the beamforming module 40 is implemented in software, it may execute instructions stored on a tangible and non-transitory computer-readable medium (e.g., the memory 60) to perform beamforming calculations using any appropriate beamforming method.

[0049] The IQ demodulation module 50 removes the signal carrier through IQ demodulation, extracts the tissue structure information contained in the signal, and performs filtering to remove noise. The resulting signal is called a baseband signal (IQ data pair). The IQ demodulation module 50 outputs the IQ data pair to the image processing module 720 of the processor 70 for image processing. In some embodiments, the IQ demodulation module 50 also outputs the IQ data pair to the memory 60 for caching or storage, so that the image processing module 720 can read the data from the memory 60 for subsequent image processing.

[0050] The processor 70 is configured to be a central control circuit (CPU), one or more microprocessors, a graphics controller circuit (GPU) or any other electronic component that can process input data according to specific logical instructions. It can control peripheral electronic components according to input instructions or predetermined instructions, or read and / or save data from the memory 60. It can also process the input data by executing the program in the memory 60, for example, performing one or more processing operations on the collected ultrasound data according to one or more working modes. The processing operations include but are not limited to adjusting or limiting the form of ultrasound waves emitted by the ultrasound probe 20, generating various image frames for subsequent display on the display 80 of the human-computer interaction device, or adjusting or limiting the content and form displayed on the display 80, or adjusting one or more image display settings displayed on the display 80 (such as ultrasound images, interface components, and positioning areas of interest).

[0051] Image processing module 720 processes the data output by beamforming module 40 or IQ demodulation module 50 to generate a grayscale image showing signal strength variations within the scanning range. This grayscale image reflects the internal anatomical structure of the tissue, referred to as a B-image. Image processing module 720 can output the B-image to display on display 80 of the human-computer interaction device.

[0052] The human-computer interaction device is used for human-computer interaction, that is, receiving user input and outputting visual information; it can receive user input using a keyboard, operation buttons, mouse, trackball, etc., or a touch screen integrated with a display; it outputs visual information using a display 80.

[0053] The memory 60 can be a tangible and non-transitory computer-readable medium, such as a flash memory card, a solid-state memory, a hard disk, etc., for storing data or programs. For example, the memory 60 can be used to store the acquired ultrasound data or image frames generated by the processor 70 that are not immediately displayed, or the memory 60 can store a graphical user interface, one or more default image display settings, and programming instructions for the processor, the beamforming module, or the IQ decoding module.

[0054] It should be noted that Figure 2 The structure shown is for illustration only and may also include Figure 2 More or fewer components than shown, or with Figure 2 Different configurations shown. Figure 2 Each component shown in the figure may be implemented using hardware and / or software. Figure 2 The ultrasonic imaging device shown can be used to execute the breast ultrasonic image segmentation method provided by any embodiment of the present invention.

[0055] Please refer to Figure 3, a breast ultrasound image segmentation method provided by an embodiment of the present invention may include:

[0056] S301: Acquire an ultrasonic image of the breast area of ​​the subject, where the ultrasonic image contains breast lesions.

[0057] In an optional embodiment, an ultrasonic image of the breast area of ​​the subject can be obtained by transmitting ultrasonic waves to the breast area of ​​the subject through the ultrasonic probe of the ultrasonic imaging device, and receiving ultrasonic echoes returned from the breast area to obtain ultrasonic echo data, and generating an ultrasonic image of the breast area of ​​the subject in real time based on the ultrasonic echo data. Specifically, the doctor can apply a coupling agent to the skin surface of the subject's breast that is fully exposed, and then hold the ultrasonic probe close to the patient's breast skin for scanning. In another optional embodiment, a pre-stored ultrasonic image of the subject's breast area can also be obtained from a storage device. The ultrasonic image obtained in this embodiment contains breast lesions.

[0058] S302. Identify the type of BI-RADS features of the breast lesion according to the ultrasound image. The BI-RADS features include at least one of shape features, direction features, edge features, internal echo features, rear echo features, calcification features, and blood flow features.

[0059] In this embodiment, the identification of BI-RADS feature types of breast lesions based on ultrasound images can be performed using either traditional image processing methods or deep learning methods. For example, the acquired ultrasound images can be input into a pre-trained BI-RADS feature recognition model to determine the BI-RADS feature types of breast lesions. The BI-RADS feature recognition model can be trained using sample ultrasound images labeled with BI-RADS feature types.

[0060] In this embodiment, the breast lesions in the ultrasound image can also be detected and located first, that is, the region of interest (ROI) of the breast lesions is determined in the ultrasound image, and then the type of BI-RADS feature of the breast lesions is identified based on the determined region of interest of the breast lesions. Among them, the detection and positioning of breast lesions in the ultrasound image can be based on deep learning, machine learning, traditional image processing and other algorithms. When detecting and locating breast lesions based on deep learning, it is necessary to first train the deep learning ROI detection model based on the collected sample ultrasound images and the annotation results of the region of interest of the breast lesions by senior physicians (the region of interest of the breast lesions annotated by the physician can be, for example, the minimum bounding box of the breast lesion or the boundary of the breast lesion). The ROI detection model can use but is not limited to FasterRCNN, SSD, YOLO, CenterNet, CornerNet, etc. During the network training phase, the error between the detection result and the annotation result of the breast lesion region of interest is calculated during the iteration process, and the weights in the network are continuously updated with the purpose of minimizing the error. This process is repeated continuously, so that the detection result gradually approaches the true value of the breast lesion ROI, and a trained ROI detection model is obtained. This model can realize the automatic positioning of breast lesions for new input ultrasound images. When traditional image processing combined with machine learning is used to detect and locate breast lesions, the following steps are usually included: (1) Find the target region based on traditional image processing methods, such as using the Select Search algorithm; (2) Transform the target region to a fixed size and use image processing methods to extract feature vectors such as gradient and texture of the image, such as Sift operator, HoG operator, GLCM gray-level co-occurrence matrix, etc.; (3) Train the feature vectors of the target region through traditional machine learning algorithms to obtain a classification model for the target frame; (4) Obtain the target bounding box, i.e., the ROI of the breast lesion, through regression methods. The type of BI-RADS feature for identifying breast lesions based on the region of interest of a determined breast lesion can be based on a traditional machine learning classification algorithm or a deep learning classification algorithm. The traditional machine learning classification algorithm may include, but is not limited to, a decision tree algorithm, a Bayesian algorithm, a k-nearest neighbor (kNN) algorithm, and a support vector machine (SVM) algorithm. The deep learning classification algorithm may form an artificial neural network by stacking convolutional layers, pooling layers, and fully connected layers to identify the type of BI-RADS feature on the input ultrasound image. The deep learning classification algorithm may include, but is not limited to, a VGG, ResNet, Inception, and other networks.

[0061] In this embodiment, only one BI-RADS feature type of a breast lesion can be identified, for example, only the type of the breast lesion shape feature, or only the type of the breast lesion edge feature, or only the type of the breast lesion rear echo feature, etc. In this embodiment, multiple BI-RADS feature types of a breast lesion can also be identified, for example, the types of the breast lesion edge feature and shape feature, or the types of the breast lesion shape feature and rear echo feature, or the types of the breast lesion shape feature, edge feature, and rear echo feature, etc. Various combinations of BI-RADS features are possible, and will not be detailed here. According to the relevant regulations of BI-RADS, the types of shape features include elliptical, circular and irregular; the types of direction features include parallel and uneven; the types of edge features include smooth, blurred, angular, microlobed and burred; the types of internal echo features include anechoic, hypoechoic, isoechoic, hyperechoic, cystic and solid mixed echo and uneven echo; the types of posterior echo features include enhancement, no change, attenuation and mixed echo; the types of calcification features include no calcification, calcification within the mass, calcification outside the mass and intraductal calcification; the types of blood flow features include no blood flow, marginal blood flow and internal blood flow.

[0062] S303: Determine a target segmentation model corresponding to the type of BI-RADS feature from a plurality of segmentation models corresponding to preset BI-RADS feature types.

[0063] In this embodiment, a corresponding segmentation model can be pre-built for each type of BI-RADS feature. For example, for shape features, corresponding segmentation models can be built for elliptical, circular and irregular shapes respectively; for edge features, corresponding segmentation models can be built for smoothing, blurring, angulation, differential lobes and burrs respectively. The segmentation model can be built based on traditional image processing algorithms, traditional machine learning algorithms, deep learning algorithms, etc. Among them, when building a segmentation model based on deep learning, a supervised learning strategy can be adopted. By stacking modules such as convolutional layers, pooling layers, upsampling layers, deconvolution layers and fully connected layers, a segmentation model is established. Then, the real segmentation annotation information is made into a mask image (mask) that is consistent with the size of the input image as supervision information, so that the segmentation model learns and outputs the area to be segmented. The segmentation model can adopt but is not limited to Mask R-CNN, U-Net, Solo, etc.

[0064] After the type of BI-RADS features of the breast lesion is identified, a target segmentation model corresponding to the type can be determined from multiple preset segmentation models based on the type so as to perform targeted segmentation.

[0065] S304: Use the target segmentation model to perform breast lesion segmentation on the ultrasound image, and segment the breast lesion area from the ultrasound image.

[0066] After the target segmentation model is determined, the ultrasound image can be input into the target segmentation model to segment the breast lesion, so as to determine the breast lesion area in the ultrasound image.

[0067] Prior art prior to this application usually pursues generalized, universal models, using only one segmentation model to handle all segmentation situations. Figure 1 Taking the ultrasound image shown in as an example, the same segmentation model will be used to Figure 1 The ultrasound images shown in (a) to (f) are used to segment breast lesions. Figure 1 The different manifestations of breast lesions in each ultrasound image are segmented using the matching segmentation model. Figure 1 Taking the ultrasound image shown in (c) as an example, the rear echo feature type is identified. Assuming that the type of the rear echo feature is enhanced, a segmentation model corresponding to the enhanced type will be selected from multiple preset segmentation models to identify the rear echo feature. Figure 1 The ultrasound image shown in (c) is segmented; Figure 1 Taking the ultrasound image shown in (f) as an example, the rear echo feature type is identified. Assuming that the type of the rear echo feature identified is attenuation, a segmentation model corresponding to the attenuation type will be selected from multiple preset segmentation models to identify the rear echo feature. Figure 1 The ultrasound image shown in (f) is segmented.

[0068] The breast ultrasound image segmentation method provided in this embodiment obtains an ultrasound image of the subject's breast region, identifies the type of BI-RADS feature of the breast lesion based on the ultrasound image, then determines a target segmentation model corresponding to the type of BI-RADS feature from multiple pre-set segmentation models corresponding to each BI-RADS feature type. Finally, the target segmentation model is used to segment the breast lesion in the ultrasound image, thereby enabling targeted segmentation of breast lesions with different appearances using different segmentation models. Because the characteristics of the breast lesions are fully considered during segmentation, the accuracy of breast lesion segmentation can be improved, thereby helping to improve the accuracy of intelligent diagnosis in CAD systems.

[0069] BI-RADS features include shape, orientation, edge, internal echo, posterior echo, calcification, and blood flow. Breast lesion segmentation can be performed based on any one or more of these BI-RADS features. Since edge, shape, and posterior echo have a significant impact on boundary conditions, the following describes how to segment breast lesions based on edge, shape, and posterior echo types, respectively.

[0070] In an optional embodiment, breast lesions can be segmented according to the type of edge features. Specifically, identifying the type of BI-RADS features of breast lesions according to ultrasound images includes: identifying the type of edge features of breast lesions according to ultrasound images; determining the target segmentation model corresponding to the type of BI-RADS features from multiple segmentation models corresponding to each type of preset BI-RADS features includes: determining the target segmentation model corresponding to the type of edge features from a preset smooth segmentation model, a fuzzy segmentation model, an angular segmentation model, a differential leaf segmentation model, and a burr segmentation model, the smooth segmentation model, the fuzzy segmentation model, the angular segmentation model, the differential leaf segmentation model, and the burr segmentation model are used to segment ultrasound images with smooth, fuzzy, angular, differential leaf, and burr edge features, respectively. The smooth segmentation model in this embodiment is trained based on sample ultrasound images with marked breast lesion areas and smooth edge features. Therefore, using the smooth segmentation model to segment ultrasound images with smooth edge features will have higher segmentation accuracy. The fuzzy segmentation model in this embodiment is trained based on sample ultrasound images with labeled breast lesion areas and fuzzy edge features. Therefore, the fuzzy segmentation model will have a higher segmentation accuracy when used to segment ultrasound images with fuzzy edge features. The angular segmentation model in this embodiment is trained based on sample ultrasound images with labeled breast lesion areas and angular edge features. Therefore, the angular segmentation model will have a higher segmentation accuracy when used to segment ultrasound images with angled edge features. The differential leaf segmentation model in this embodiment is trained based on sample ultrasound images with labeled breast lesion areas and differential leaf edge features. Therefore, the differential leaf segmentation model will have a higher segmentation accuracy when used to segment ultrasound images with differential leaf edge features. The burr segmentation model in this embodiment is trained based on sample ultrasound images with labeled breast lesion areas and burr edge features. Therefore, the burr segmentation model will have a higher segmentation accuracy when used to segment ultrasound images with burr edge features. Assuming that Figure 1 The type of edge feature in (a) is smoothing, so the smoothing segmentation model can be used as the target segmentation model. Figure 1The ultrasound image shown in (a) is used to segment the breast lesion; assuming that Figure 1 The type of edge feature in (d) is fuzzy, so the fuzzy segmentation model can be used as the target segmentation model. Figure 1 Breast lesions are segmented using the ultrasound image shown in (d).

[0071] In an optional embodiment, breast lesion segmentation can be performed based on the type of shape feature. Specifically, identifying the type of BI-RADS feature of a breast lesion based on an ultrasound image includes: identifying the type of shape feature of the breast lesion based on the ultrasound image; determining a target segmentation model corresponding to the type of BI-RADS feature from multiple segmentation models corresponding to each type of BI-RADS feature includes: determining a target segmentation model corresponding to the type of shape feature from a preset elliptical segmentation model, a circular segmentation model, and an irregular segmentation model, wherein the elliptical segmentation model, the circular segmentation model, and the irregular segmentation model are respectively used to segment ultrasound images with elliptical, circular, and irregular shape features. The elliptical segmentation model in this embodiment is trained based on sample ultrasound images with labeled breast lesion regions and elliptical shape features. Therefore, using the elliptical segmentation model for segmenting ultrasound images with elliptical shape features will have higher segmentation accuracy. The circular segmentation model in this embodiment is trained based on sample ultrasound images with labeled breast lesion regions and circular shape features. Therefore, using the circular segmentation model for segmenting ultrasound images with circular shape features will have higher segmentation accuracy. The irregular segmentation model in this embodiment is trained based on sample ultrasound images with annotated breast lesion areas and irregular shape features. Therefore, using the irregular segmentation model to segment ultrasound images with irregular shape features will have higher segmentation accuracy. Figure 1 In (a), the shape feature type is circular, so the circular segmentation model can be used as the target segmentation model. Figure 1 The ultrasound image shown in (a) is used to segment the breast lesion; assuming that Figure 1 The type of shape feature of (f) is irregular, so the irregular segmentation model can be used as the target segmentation model. Figure 1 Breast lesions are segmented using the ultrasound image shown in (f).

[0072] In an optional embodiment, breast lesions can be segmented according to the type of rear echo feature. Specifically, identifying the type of BI-RADS feature of a breast lesion according to an ultrasound image includes: identifying the type of rear echo feature of a breast lesion according to an ultrasound image; determining a target segmentation model corresponding to the type of BI-RADS feature from a plurality of segmentation models corresponding to each type of preset BI-RADS feature includes: determining a target segmentation model corresponding to the type of rear echo feature from a preset enhanced segmentation model, an unchanged segmentation model, an attenuated segmentation model, and a mixed echo segmentation model, wherein the enhanced segmentation model, the unchanged segmentation model, the attenuated segmentation model, and the mixed echo segmentation model are used to segment ultrasound images whose rear echo feature types are enhanced, unchanged, attenuated, and mixed echo, respectively. The enhanced segmentation model in this embodiment is trained based on sample ultrasound images with annotated breast lesion areas and whose rear echo feature types are enhanced. Therefore, using the enhanced segmentation model to segment ultrasound images whose rear echo feature types are enhanced will have higher segmentation accuracy. The unchanged segmentation model in this embodiment is obtained by training based on sample ultrasound images with labeled breast lesion areas and whose rear echo features are of unchanged type. Therefore, using the unchanged segmentation model to segment ultrasound images with unchanged type of rear echo features will have higher segmentation accuracy. The attenuation segmentation model in this embodiment is obtained by training based on sample ultrasound images with labeled breast lesion areas and whose rear echo features are of attenuated type. Therefore, using the attenuation segmentation model to segment ultrasound images with attenuated type of rear echo features will have higher segmentation accuracy. The mixed echo segmentation model in this embodiment is obtained by training based on sample ultrasound images with labeled breast lesion areas and whose rear echo features are of mixed echo type. Therefore, using the mixed echo segmentation model to segment ultrasound images with mixed echo type of rear echo features will have higher segmentation accuracy. Assuming that Figure 1 The type of the rear echo feature in (a) is unchanged, so the unchanged segmentation model can be used as the target segmentation model. Figure 1 The ultrasound image shown in (a) is used to segment the breast lesion; assuming that Figure 1 The type of the rear echo feature in (c) is enhancement, so the enhanced segmentation model can be used as the target segmentation model. Figure 1 Breast lesions are segmented using the ultrasound image shown in (c).

[0073] The above describes how to segment breast lesions based on the type of edge features, shape features, or rear echo features. It should be noted that similar methods can be used to segment breast lesions based on the type of directional features, internal echo features, calcification features, or blood flow features, and these methods will not be described in detail here.

[0074] It is understood that breast lesion segmentation can be performed based on the types of multiple BI-RADS features. Figure 1 Taking (a) in the figure as an example, the type of its edge feature is smooth, the type of its shape feature is circular, and the type of its rear echo feature is unchanged, then the target segmentation model includes a smooth segmentation model, a circular segmentation model, and an unchanged segmentation model. When the target segmentation model includes multiple segmentation models, the target segmentation model is used to segment the ultrasound image for breast lesions, and segmenting the breast lesion area from the ultrasound image includes: using each segmentation model in the target segmentation model to segment the ultrasound image for breast lesions respectively; fusing the segmentation results obtained by the multiple segmentation models included in the target segmentation model to obtain the breast lesion area in the ultrasound image. That is to say, the smooth segmentation model, the circular segmentation model, and the unchanged segmentation model can be used respectively to segment the ultrasound image for breast lesions. Figure 1 The ultrasound image shown in (a) is segmented for breast lesions. The segmentation results are then fused according to preset weights to determine the breast lesion region in the ultrasound image. The preset weights can be determined based on the influence of each BI-RADS feature on the boundary conditions. For example, the weight of the edge feature is 0.5, the weight of the shape feature is 0.3, and the weight of the back echo feature is 0.2.

[0075] It should be noted that, in addition to the fact that the target segmentation model may include multiple segmentation models when performing breast lesion segmentation based on multiple BI-RADS feature types, the target segmentation model may also include multiple segmentation models when performing breast lesion segmentation based on edge features. This is because the edge feature types of an ultrasound image may correspond to multiple. Figure 1 Taking the ultrasound image shown in (f) as an example, assuming that the types of edge features identified are differential leaf, angle and burr, the target segmentation model includes differential leaf segmentation model, angle segmentation model and burr segmentation model. Each segmentation model in the target segmentation model is used to segment the edge of the image. Figure 1 The ultrasound image shown in (f) is used to segment the breast lesions, and then the multiple segmentation results are fused to determine Figure 1 The breast lesion area in (f) can be further combined with the edge feature and the rear echo feature to perform breast lesion segmentation. Figure 1 Taking the ultrasound image shown in (f) as an example, assuming that the types of identified edge features are differential lobes, angles, and burrs, and the type of rear echo features is attenuation, the target segmentation model includes differential lobes segmentation model, angle segmentation model, burr segmentation model, and attenuation segmentation model.

[0076] Considering that when performing ultrasound imaging of the breast area, there may be interference from human factors or the external environment, resulting in low quality of the acquired ultrasound image, which in turn leads to inaccurate types of BI-RADS features of breast lesions identified based on the ultrasound image, thereby reducing the accuracy of breast lesion segmentation. Therefore, based on any of the above embodiments, in order to further eliminate interference and improve the accuracy of breast lesion segmentation, the method provided in this embodiment may further include, before determining the target segmentation model corresponding to the type of BI-RADS features, receiving a user input operation, the input operation is used to confirm, modify or supplement the type of BI-RADS features of the identified breast lesions; and determining the type of BI-RADS features of the breast lesions based on the input operation. The user can input confirmation, modification or supplement information of the type of BI-RADS features of the breast lesions through an external input device such as a mouse, keyboard, touch screen, etc. When the user agrees with the identified type, confirmation is performed; when the user believes that the identified type is incorrect, modification is performed; when the user believes that the identified type is missing, the user can supplement it. That is to say, the type of BI-RADS features of the breast lesion determined according to the user's input operation will be more accurate, so that a more matching segmentation model can be selected, which helps to improve the segmentation accuracy.

[0077] To facilitate user viewing of breast lesion regions in ultrasound images, the method provided in this embodiment, based on any of the above embodiments, may further include: displaying the breast lesion regions segmented from the ultrasound image on a display interface. For example, the boundaries of the breast lesion regions may be displayed in the ultrasound image; the breast lesion regions may be highlighted; or only the segmented breast lesion regions may be displayed.

[0078] Please refer to Figure 4 Another embodiment of the present invention provides a breast ultrasound image segmentation method that may include:

[0079] S401: Acquire an ultrasonic image of the breast area of ​​the subject, where the ultrasonic image contains breast lesions.

[0080] Please refer to S301 for the specific implementation, which will not be described again here.

[0081] S402. Identify the BI-RADS grade of the breast lesion based on the ultrasound image.

[0082] The BI-RADS grading of breast lesions includes level 0, level 1, level 2, level 3, level 4a, level 4b, level 4c, level 5 and level 6. It should be noted that BI-RADS level 0 indicates that the assessment is incomplete and further imaging examination is required; BI-RADS level 1 indicates that the assessment is negative and no lesion is found. The lesion segmentation of this application does not involve level 0 and level 1. Therefore, the BI-RADS grading of breast lesions identified in this application includes one of level 2, level 3, level 4a, level 4b, level 4c, level 5 and level 6. In this embodiment, both traditional image processing methods and deep learning methods can be used to identify the BI-RADS grading of breast lesions based on ultrasound images. Taking the deep learning method as an example, a BI-RADS grading recognition model can be trained based on sample ultrasound images labeled with BI-RADS grading, and then the acquired ultrasound image can be input into the pre-trained BI-RADS grading recognition model to determine the BI-RADS grading of the breast lesion.

[0083] In this embodiment, the breast lesion in the ultrasound image may be detected and located first, that is, a region of interest (ROI) of the breast lesion may be determined in the ultrasound image, and then the BI-RADS grade of the breast lesion may be identified based on the determined ROI. The specific implementation method may be referred to in S302 and will not be described in detail here.

[0084] S403. Determine a target segmentation model corresponding to the BI-RADS grade from the preset first segmentation model, second segmentation model, third segmentation model, fourth segmentation model, fifth segmentation model, sixth segmentation model, and seventh segmentation model. The first segmentation model, the second segmentation model, the third segmentation model, the fourth segmentation model, the fifth segmentation model, the sixth segmentation model, and the seventh segmentation model are used to segment ultrasound images with BI-RADS grades of 2, 3, 4a, 4b, 4c, 5, and 6, respectively.

[0085] In this embodiment, a corresponding segmentation model can be pre-constructed for each BI-RADS classification. For example, a first segmentation model, a second segmentation model, a third segmentation model, a fourth segmentation model, a fifth segmentation model, a sixth segmentation model, and a seventh segmentation model can be constructed for segmenting ultrasound images with BI-RADS classifications of 2, 3, 4a, 4b, 4c, 5, and 6, respectively. The first segmentation model can be trained based on sample ultrasound images with a BI-RADS classification of 2 and labeled with breast lesions. Therefore, using the first segmentation model to segment ultrasound images with a BI-RADS classification of 2 will have higher segmentation accuracy. The second segmentation model can be trained based on sample ultrasound images with a BI-RADS classification of 3 and labeled with breast lesions. Therefore, using the second segmentation model to segment ultrasound images with a BI-RADS classification of 3 will have higher segmentation accuracy. The third segmentation model can be trained based on sample ultrasound images with a BI-RADS classification of 4a and labeled with breast lesions. Therefore, using the third segmentation model to segment ultrasound images with a BI-RADS classification of 4a will have higher segmentation accuracy. The same goes for other cases, so I won’t go into details here.

[0086] After identifying the BI-RADS grade of the breast lesion, the target segmentation model corresponding to the BI-RADS grade can be determined from the seven preset segmentation models based on the BI-RADS grade so as to perform targeted segmentation.

[0087] S404: Use the target segmentation model to perform breast lesion segmentation on the ultrasound image, and segment the breast lesion area from the ultrasound image.

[0088] After the target segmentation model is determined, the ultrasound image can be input into the target segmentation model to segment the breast lesion, so as to determine the breast lesion area in the ultrasound image.

[0089] The breast ultrasound image segmentation method provided in this embodiment obtains an ultrasound image of the subject's breast region, identifies the BI-RADS grade of the breast lesion based on the ultrasound image, then determines a target segmentation model corresponding to the BI-RADS grade from the first through seventh preset segmentation models. Finally, the target segmentation model is used to segment the breast lesion in the ultrasound image. This enables targeted segmentation of breast lesions of different BI-RADS grades using different segmentation models. Because the BI-RADS grade of the breast lesion is fully considered during segmentation, the accuracy of breast lesion segmentation can be improved, thereby contributing to the improvement of the accuracy of intelligent diagnosis in CAD systems.

[0090] Considering that when performing ultrasound imaging of the breast area, there may be interference from human factors or the external environment, resulting in low quality of the acquired ultrasound image, which in turn leads to inaccurate BI-RADS grading of breast lesions identified based on the ultrasound image, thereby reducing the accuracy of breast lesion segmentation. Therefore, based on the above embodiment, in order to further eliminate interference and improve the accuracy of breast lesion segmentation, the method provided in this embodiment may further include: receiving user input operations before determining the target segmentation model corresponding to the BI-RADS grade; the input operations are used to confirm, modify or supplement the BI-RADS grade of the identified breast lesion; and determining the BI-RADS grade of the breast lesion based on the input operations. The user can input confirmation, modification or supplement information of the BI-RADS grade of the breast lesion through an external input device such as a mouse, keyboard, touch screen, etc. When the user agrees with the identified BI-RADS grade, confirmation is performed; when the user believes that the identified BI-RADS grade is incorrect, modification is performed; when the identified BI-RADS grade is missing, the user can supplement it. That is to say, the BI-RADS grade of the breast lesion determined according to the user's input operation will be more accurate, so that a more matching segmentation model can be selected, which helps to improve the segmentation accuracy.

[0091] To facilitate users in viewing the breast lesion region in the ultrasound image, based on the above embodiment, the method provided in this embodiment may further include: displaying the breast lesion region segmented from the ultrasound image on a display interface.

[0092] Please refer to Figure 5 Another embodiment of the present invention provides a breast ultrasound image segmentation method that may include:

[0093] S501: Acquire an ultrasonic image of the breast area of ​​the subject, where the ultrasonic image contains breast lesions.

[0094] Please refer to S301 for the specific implementation, which will not be described again here.

[0095] S502. Determine the classification probability of each type of BI-RADS feature of the breast lesion based on the ultrasound image. The BI-RADS feature includes at least one of a shape feature, a direction feature, an edge feature, an internal echo feature, a rear echo feature, a calcification feature, and a blood flow feature.

[0096] In this embodiment, the acquired ultrasound image can be input into a pre-trained BI-RADS feature classification model to determine the classification probability of each type of BI-RADS feature of the breast lesion. The BI-RADS feature classification model takes the ultrasound image as input and outputs the classification probability of each type of BI-RADS feature of the breast lesion in the ultrasound image. It can be trained based on sample ultrasound images that are labeled with the classification probability of each type of BI-RADS feature. In this embodiment, only the classification probability of each type of BI-RADS feature of the breast lesion can be determined. For example, only the classification probability of the breast lesion belonging to the circular, elliptical and irregular shapes in the shape feature can be determined. Figure 1 As an example, assume that according to Figure 1 The ultrasound image shown in (a) determines that the probabilities of the breast lesion belonging to the circular, elliptical, and irregular shapes, respectively, are 0.6, 0.3, and 0.1. In this embodiment, the classification probabilities of a breast lesion belonging to each of multiple BI-RADS features can also be determined. For example, the probabilities of a breast lesion belonging to the circular, elliptical, and irregular shapes, as well as the probabilities of a breast lesion belonging to the blood flow feature, as no blood flow, marginal blood flow, and internal blood flow, can be determined. Any combination of BI-RADS features is possible and will not be detailed here.

[0097] In this embodiment, the breast lesion in the ultrasound image may be detected and located first. Specifically, a region of interest (ROI) of the breast lesion may be determined in the ultrasound image. Then, based on the determined ROI of the breast lesion, the classification probabilities of various types of BI-RADS features of the breast lesion may be determined. The specific implementation of detecting and locating the breast lesion may be referred to in S302 and will not be further described here.

[0098] S503 , using multiple segmentation models corresponding to different types of BI-RADS features to segment breast lesions on the ultrasound image, and obtaining multiple breast lesion segmentation results corresponding to different types of BI-RADS features.

[0099] In this embodiment, a corresponding segmentation model can be pre-built for each type of BI-RADS feature. For example, for shape features, corresponding segmentation models can be built for elliptical, circular, and irregular shapes; for edge features, corresponding segmentation models can be built for smooth, blurred, angular, microlobed, and burr features. The specific method for building the segmentation model can be referred to S303 and will not be repeated here.

[0100] For the acquired ultrasound images, multiple segmentation models corresponding to different types of BI-RADS features are used to segment breast lesions and obtain multiple segmentation results.

[0101] In this embodiment, there is no restriction on the execution order of S502 and S503.

[0102] S504 , fusing multiple breast lesion segmentation results corresponding to various types of BI-RADS features according to the classification probability to obtain a breast lesion region in the ultrasound image.

[0103] The resulting multiple segmentation results are fused based on classification probabilities, synthesizing the segmentation results from multiple segmentation models to produce a final result. For example, classification probabilities can be used as weights to weight the multiple segmentation results. Alternatively, only the segmentation results corresponding to classification probabilities greater than a preset probability threshold are fused to eliminate interference and further improve segmentation accuracy.

[0104] The breast ultrasound image segmentation method provided in this embodiment obtains an ultrasound image of the subject's breast region and determines the classification probability of each type of BI-RADS feature of the breast lesion based on the ultrasound image. Multiple segmentation models corresponding to each type of BI-RADS feature are then used to segment the ultrasound image, obtaining multiple breast lesion segmentation results corresponding to each type of BI-RADS feature. Finally, the multiple breast lesion segmentation results corresponding to each type of BI-RADS feature are fused based on the classification probability, achieving targeted fusion of breast lesions with different manifestations. Because the characteristics of the breast lesion are fully considered when determining the breast lesion region, the accuracy of breast lesion region segmentation can be improved, thereby helping to improve the accuracy of intelligent diagnosis in the CAD system.

[0105] BI-RADS features include shape, orientation, edge, internal echo, posterior echo, calcification, and blood flow. The classification probabilities of any one or more BI-RADS features can be used to determine the area of ​​a breast lesion. Because edge, shape, and posterior echo features significantly influence boundary conditions, the following describes how to determine the area of ​​a breast lesion based on the classification probabilities of each edge, shape, and posterior echo features.

[0106] In an optional embodiment, the segmentation results of multiple segmentation models corresponding to each type of edge feature can be fused according to the classification probability of each type of edge feature to determine the final breast lesion area. Specifically, determining the classification probability of each type of BI-RADS feature of the breast lesion according to the ultrasound image includes: determining the classification probability of the edge feature types of the breast lesion according to the ultrasound image as smooth, fuzzy, angular, differential lobes and burrs; using multiple segmentation models corresponding to each type of BI-RADS feature to segment the ultrasound image for breast lesions, and obtaining multiple breast lesion segmentation results corresponding to each type of BI-RADS feature includes: using a smooth segmentation model, a fuzzy segmentation model, an angular segmentation model, a differential lobes segmentation model and a burr segmentation model to segment the ultrasound image for breast lesions, and obtaining multiple breast lesion segmentation results corresponding to smooth, fuzzy, angular, differential lobes and burrs. The smooth segmentation model, fuzzy segmentation model, angular segmentation model, differential lobes segmentation model and burr segmentation model in this embodiment can refer to the embodiment described above and will not be repeated here. Assuming that it is determined Figure 1 In (a), the probabilities of the edge feature types being smooth, fuzzy, angular, differential leaf, and burr are 0.68, 0.2, 0.01, 0.1, and 0.01, respectively. 0.68, 0.2, 0.01, 0.1, and 0.01 can be used as weights to weight the segmentation results of the smooth segmentation model, fuzzy segmentation model, angular segmentation model, differential leaf segmentation model, and burr segmentation model.

[0107] In an optional embodiment, the segmentation results of multiple segmentation models corresponding to each type of shape feature can be fused according to the classification probability of each type of shape feature to determine the final breast lesion area. Specifically, determining the classification probability of each type of BI-RADS feature of the breast lesion based on the ultrasound image includes: determining the classification probability that the shape feature types of the breast lesion are elliptical, circular and irregular according to the ultrasound image; using multiple segmentation models corresponding to each type of BI-RADS feature to segment the ultrasound image for breast lesions, and obtaining multiple breast lesion segmentation results corresponding to each type of BI-RADS feature includes: using an elliptical segmentation model, a circular segmentation model and an irregular segmentation model to segment the ultrasound image for breast lesions, and obtaining multiple breast lesion segmentation results corresponding to elliptical, circular and irregular shapes. The elliptical segmentation model, the circular segmentation model and the irregular segmentation model in this embodiment can refer to the embodiment described above and will not be repeated here. Assuming that it is determined Figure 1 In (a), the probabilities that the shape features are elliptical, circular, and irregular are 0.1, 0.85, and 0.05, respectively. 0.1, 0.85, and 0.05 can then be used as weights to weight the segmentation results of the elliptical segmentation model, the circular segmentation model, and the irregular segmentation model.

[0108] In an optional embodiment, the segmentation results of multiple segmentation models corresponding to each type of rear echo feature can be fused according to the classification probability of each type of rear echo feature to determine the final breast lesion area. Specifically, determining the classification probability of each type of BI-RADS feature of the breast lesion according to the ultrasound image includes: determining the classification probability of the types of rear echo features of the breast lesion as enhancement, no change, attenuation and mixed echo according to the ultrasound image; using multiple segmentation models corresponding to each type of BI-RADS feature to segment the ultrasound image for breast lesions, and obtaining multiple breast lesion segmentation results corresponding to each type of BI-RADS feature includes: using the enhancement segmentation model, the no change segmentation model, the attenuation segmentation model and the mixed echo segmentation model to segment the ultrasound image for breast lesions, and obtaining multiple breast lesion segmentation results corresponding to enhancement, no change, attenuation and mixed echo. The enhancement segmentation model, the no change segmentation model, the attenuation segmentation model and the mixed echo segmentation model in this embodiment can refer to the embodiment described above and will not be repeated here. Assuming that it is determined Figure 1 In (a), the probabilities of the types of rear echo features being enhancement, no change, attenuation, and mixed echo are 0.1, 0.8, 0.05, and 0.05, respectively. Then, 0.1, 0.8, 0.05, and 0.05 can be used as weights to weight the segmentation results of the enhancement segmentation model, the no change segmentation model, the attenuation segmentation model, and the mixed echo segmentation model.

[0109] The above describes how to determine the breast lesion area based on the classification probabilities of various types of edge features, the classification probabilities of various types of shape features, or the classification probabilities of various types of posterior echo features. It should be noted that similar methods can be used to determine the breast lesion area based on the classification probabilities of various types of directional features, the classification probabilities of various types of internal echo features, the classification probabilities of various types of calcification features, or the classification probabilities of various types of blood flow features, and these methods will not be described in detail here.

[0110] To facilitate users in viewing the breast lesion region in the ultrasound image, based on the above embodiment, the method provided in this embodiment may further include: displaying the breast lesion region segmented from the ultrasound image on a display interface.

[0111] Please refer to Figure 6 Another embodiment of the present invention provides a breast ultrasound image segmentation method that may include:

[0112] S601: Acquire an ultrasonic image of the breast area of ​​the subject, where the ultrasonic image contains breast lesions.

[0113] Please refer to S301 for the specific implementation, which will not be described again here.

[0114] S602: Determine the probability of the breast lesion belonging to each BI-RADS grade based on the ultrasound image.

[0115] The BI-RADS classification of breast lesions includes level 0, level 1, level 2, level 3, level 4a, level 4b, level 4c, level 5 and level 6. It should be noted that BI-RADS level 0 indicates that the assessment is incomplete and further imaging examination is required; BI-RADS level 1 indicates that the assessment is negative and no lesion is found. The lesion segmentation of this application does not involve level 0 and level 1. Therefore, the BI-RADS classification of breast lesions identified in this application includes one of level 2, level 3, level 4a, level 4b, level 4c, level 5 and level 6. In this embodiment, both traditional image processing methods and deep learning methods can be used to determine the probability of breast lesions belonging to each BI-RADS classification based on ultrasound images. Taking the deep learning method as an example, a BI-RADS classification classification model can be trained based on sample ultrasound images labeled with BI-RADS classifications, and then the acquired ultrasound images can be input into the pre-trained BI-RADS classification model to determine the probability of breast lesions belonging to each BI-RADS classification.

[0116] In this embodiment, the breast lesion in the ultrasound image may be detected and located first. Specifically, a region of interest (ROI) of the breast lesion may be determined in the ultrasound image. Then, based on the determined ROI of the breast lesion, the probability of the breast lesion belonging to each BI-RADS classification may be determined. The specific implementation of detecting and locating the breast lesion may be referred to in S302 and will not be further described here.

[0117] S603. Use the first segmentation model, the second segmentation model, the third segmentation model, the fourth segmentation model, the fifth segmentation model, the sixth segmentation model and the seventh segmentation model to perform breast lesion segmentation on the ultrasound image, respectively, and obtain multiple breast lesion segmentation results corresponding to level 2, level 3, level 4a, level 4b, level 4c, level 5 and level 6, respectively. The first segmentation model, the second segmentation model, the third segmentation model, the fourth segmentation model, the fifth segmentation model, the sixth segmentation model and the seventh segmentation model are used to segment ultrasound images with BI-RADS grades of 2, level 3, level 4a, level 4b, level 4c, level 5 and level 6, respectively.

[0118] The first to seventh segmentation models in this embodiment can refer to S403 and will not be described in detail here. The first to seventh segmentation models are used to segment the ultrasound image into breast lesions, respectively, to obtain seven segmentation results.

[0119] In this embodiment, there is no restriction on the execution order of S602 and S603.

[0120] S604: Fusing the multiple breast lesion segmentation results obtained based on the probability that the breast lesion belongs to each BI-RADS grade, to obtain a breast lesion region in the ultrasound image.

[0121] The seven segmentation results are fused based on the classification probabilities, combining the segmentation results of the seven segmentation models to produce a final result. For example, the classification probabilities can be used as weights to weight the seven segmentation results. Alternatively, only the segmentation results corresponding to classification probabilities greater than a preset probability threshold are fused to eliminate interference factors and further improve segmentation accuracy.

[0122] The breast ultrasound image segmentation method provided in this embodiment obtains an ultrasound image of the subject's breast region and determines the probability of a breast lesion belonging to each BI-RADS grade based on the ultrasound image. The method then uses the first through seventh segmentation models to segment the ultrasound image, respectively, to obtain seven breast lesion segmentation results. Finally, the seven breast lesion segmentation results are fused based on the probabilities of the breast lesions belonging to each BI-RADS grade, thereby achieving targeted fusion of breast lesions with different manifestations. Because the BI-RADS grade of the breast lesion is fully considered when determining the breast lesion region, the accuracy of breast lesion region segmentation can be improved, thereby contributing to the improvement of the accuracy of intelligent diagnosis in the CAD system.

[0123] To facilitate users in viewing the breast lesion region in the ultrasound image, based on the above embodiment, the method provided in this embodiment may further include: displaying the breast lesion region segmented from the ultrasound image on a display interface.

[0124] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, the various operational steps and components used to perform the operational steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0125] Additionally, as will be appreciated by those skilled in the art, the principles of this disclosure may be embodied in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing device to form a machine, such that the instructions executed on the computer or other programmable data processing device can generate a device that implements a specified function. These computer program instructions may also be stored in a computer-readable memory, which can instruct the computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory can form an article of manufacture that includes an implementation device that implements the specified function. The computer program instructions may also be loaded onto a computer or other programmable data processing device, causing the computer or other programmable device to execute a series of operational steps to produce a computer-implemented process, such that the instructions executed on the computer or other programmable device provide the steps for implementing the specified function.

[0126] Although the principles of this invention have been shown in various embodiments, many modifications of structure, arrangement, proportion, elements, materials and components that are particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments are intended to be included within the scope of this invention.

[0127] The foregoing detailed description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Therefore, the present disclosure will be considered in an illustrative rather than a restrictive sense, and all such modifications will be included within its scope. Similarly, the advantages, other advantages and solutions to the problems of the various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more specific, should not be interpreted as critical, required or necessary. The term "comprising" and any other variants used in this article are all non-exclusive inclusions, so that a process, method, article or device that includes a list of elements includes not only these elements, but also other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0128] The above examples are used to illustrate the present invention, which are only used to help understand the present invention and are not intended to limit the present invention. Those skilled in the art can make several simple deductions, modifications or substitutions based on the concept of the present invention.

Claims

1. A breast ultrasound image segmentation method, characterized in that: include: Acquiring an ultrasonic image of a breast region of a subject, wherein the ultrasonic image includes a breast lesion; Determining, based on the ultrasound image, classification probabilities of various types of BI-RADS features of the breast lesion, where the BI-RADS features include at least one of shape features, direction features, edge features, internal echo features, rear echo features, calcification features, and blood flow features; specifically, determining, based on the ultrasound image, classification probabilities of the rear echo features of the breast lesion being enhancement, no change, attenuation, and mixed echoes; Using multiple segmentation models corresponding to each type of BI-RADS feature to segment the ultrasound image for breast lesions, respectively, to obtain multiple breast lesion segmentation results corresponding to each type of BI-RADS feature, including: using an enhancement segmentation model, an unchanged segmentation model, an attenuation segmentation model, and a mixed echo segmentation model to segment the ultrasound image for breast lesions, respectively, to obtain multiple breast lesion segmentation results corresponding to enhancement, unchanged, attenuated, and mixed echoes, respectively; The classification probability is used as a weight to weight multiple breast lesion segmentation results corresponding to each type of the BI-RADS feature to obtain a breast lesion area in the ultrasound image.

2. The method according to claim 1, wherein Determining the classification probabilities of the BI-RADS features of the breast lesions according to the ultrasound image includes: determining the classification probabilities of the edge features of the breast lesions being smooth, blurred, angular, microlobed, and burred, respectively, according to the ultrasound image; The method of using multiple segmentation models corresponding to each type of BI-RADS feature to segment breast lesions on the ultrasound image respectively to obtain multiple breast lesion segmentation results corresponding to each type of BI-RADS feature includes: using a smooth segmentation model, a fuzzy segmentation model, an angular segmentation model, a differential leaf segmentation model and a burr segmentation model to segment breast lesions on the ultrasound image respectively to obtain multiple breast lesion segmentation results corresponding to smooth, fuzzy, angular, differential leaf and burr respectively.

3. The method according to claim 1, wherein Determining the classification probabilities of the BI-RADS features of the breast lesions according to the ultrasound image includes: determining the classification probabilities of the shape features of the breast lesions being elliptical, circular, and irregular, respectively, according to the ultrasound image; The method of using multiple segmentation models corresponding to each type of BI-RADS feature to segment breast lesions on the ultrasound image to obtain multiple breast lesion segmentation results corresponding to each type of BI-RADS feature includes: using an elliptical segmentation model, a circular segmentation model and an irregular segmentation model to segment breast lesions on the ultrasound image to obtain multiple breast lesion segmentation results corresponding to elliptical, circular and irregular shapes, respectively.

4. The method according to any one of claims 1 to 3, wherein The method further includes: displaying the breast lesion region segmented from the ultrasound image on a display interface.

5. An ultrasonic imaging device, characterized in that: include: Ultrasound probe; a transmitting circuit, configured to output a corresponding transmitting sequence to the ultrasonic probe according to a set mode, so as to control the ultrasonic probe to transmit corresponding ultrasonic waves; a receiving circuit, configured to receive the ultrasonic echo signal output by the ultrasonic probe and output ultrasonic echo data; A display for outputting visual information; A processor, configured to execute the breast ultrasound image segmentation method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the breast ultrasound image segmentation method according to any one of claims 1 to 4.

Images