Ultrasound imaging method and apparatus for breast

Abstract

Embodiments of the present application provide an ultrasound imaging method and device for breast, the method comprising: acquiring an ultrasound image of a breast region of a subject; analyzing the ultrasound image based on a pre-trained intelligent breast lesion analysis model to obtain a first feature value set corresponding to a BI-RADS feature set value of a breast lesion in the breast region of the subject; detecting a user's operation of modifying or confirming the first feature value set to obtain a second feature value set; and determining a BI-RADS classification of the breast lesion according to the first feature value set, the second feature value set, and the ultrasound image. The method of the embodiments of the present application not only utilizes the ultrasound image of the breast region, but also combines the feedback information of the user when determining the BI-RADS classification of the breast lesion, thereby improving the accuracy of the BI-RADS classification.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to an ultrasound imaging method and device for breast tissue. Background Technology

[0002] Breast cancer is a malignant tumor that occurs in the glandular epithelial tissue of the breast. According to cancer statistics, breast cancer ranks first in the incidence of malignant tumors in women, making early screening for breast cancer particularly important. Breast ultrasound can clearly show the location, shape, internal structure, and changes in adjacent tissues of each layer of breast soft tissue and lesions. It has many advantages, including being economical, convenient, non-invasive, painless, non-radioactive, and highly repeatable, and has become one of the important methods for early breast cancer screening.

[0003] Breast lesions present with complex symptoms. Currently, the most widely used and relatively authoritative diagnostic standard in clinical practice is the Breast Imaging Reporting and Data System (BI-RADS) proposed by the American College of Radiology (ACR). BI-RADS uses standardized, professional terminology to diagnose and classify lesions; however, its diagnostic rules are complex and numerous, making them difficult for junior or primary care physicians to memorize, thus affecting the efficiency of clinical diagnosis. With the rapid development of artificial intelligence technology, especially deep learning, computer-aided diagnosis is used for intelligent analysis of breast ultrasound images, providing clinicians with automated and efficient auxiliary diagnostic tools with significant clinical value. While existing AI-based breast ultrasound image analysis methods and systems help improve the efficiency of clinicians' diagnoses, their accuracy still needs improvement because they typically only use image information as input data for analysis. Summary of the Invention

[0004] This invention provides a method and device for ultrasound imaging of the breast, which addresses the problem of low accuracy in existing methods.

[0005] In a first aspect, embodiments of the present invention provide an ultrasound imaging method for breast tissue, comprising:

[0006] Obtain ultrasound images of the breast region of the subject;

[0007] The ultrasound image is analyzed based on a pre-trained intelligent analysis model for breast lesions to obtain the first feature value set corresponding to the BI-RADS feature set values ​​of the breast lesions in the breast region of the subject. The intelligent analysis model for breast lesions is trained based on sample ultrasound images labeled with BI-RADS feature set values.

[0008] Detect user actions that modify or confirm the first feature value set to obtain the second feature value set;

[0009] The BI-RADS classification of the breast lesion is determined based on the first set of eigenvalues, the second set of eigenvalues, and the ultrasound image.

[0010] In one embodiment, the BI-RADS feature set includes shape type, orientation type, edge type, echo type, posterior echo type, calcification type, and blood supply type.

[0011] In one embodiment, determining the BI-RADS classification of the breast lesion based on the first feature set, the second feature set, and the ultrasound image includes:

[0012] Image information feature vectors are obtained based on the ultrasound images. Where, x i w is the feature vector corresponding to the i-th BI-RADS feature in the BI-RADS feature set extracted from the ultrasound image. i For x i The initial weights, where n is the number of BI-RADS features in the BI-RADS feature set;

[0013] If the i-th BI-RADS feature has the same value in both the first feature set and the second feature set, then w′ i =w i +Δ Adjust x i The weights; if the i-th BI-RADS feature has different values ​​in the first feature set and the second feature set, then by w′ i =w i -Δ Adjust x i The weights; where Δ is the weight adjustment amount, w′ i For the adjusted x i The weights;

[0014] Based on the feature vectors corresponding to each BI-RADS feature and their adjusted weights, the image information feature vectors used to determine the BI-RADS classification of the breast lesion are determined.

[0015] The image information feature vector X′ used to determine the BI-RADS classification of the breast lesion is input into a pre-trained BI-RADS classification model to obtain the BI-RADS classification of the breast lesion. The BI-RADS classification model is trained based on the sample feature vector labeled with the BI-RADS classification.

[0016] In one embodiment, the method further includes:

[0017] Obtain the attribute feature vector based on the second feature value set. Among them, y i Let r be the value of the i-th BI-RADS feature in the second feature set. i For y i The initial weights;

[0018] If the i-th BI-RADS feature has the same value in both the first feature set and the second feature set, then through r′ i =r i +Δ Adjust y i The weights; if the i-th BI-RADS feature has different values ​​in the first feature set and the second feature set, then through r′ i =r i -Δ Adjust y i The weights; where r′ i After adjustment y i The weights;

[0019] Based on the values ​​of each BI-RADS feature in the second feature set and their corresponding adjusted weights, an attribute information feature vector for determining the BI-RADS grading of the breast lesion is determined.

[0020] The image information feature vector X′ used to determine the BI-RADS classification of the breast lesion and the attribute information feature vector Y′ used to determine the BI-RADS classification of the breast lesion are fused together.

[0021] The fused feature vector is input into the pre-trained BI-RADS grading model to obtain the BI-RADS grading of the breast lesion.

[0022] In one embodiment, the ultrasound images of the subject's breast region are obtained as multiple frames of ultrasound images. The pre-trained intelligent analysis model for breast lesions analyzes the ultrasound images to obtain a first set of feature values ​​corresponding to the BI-RADS feature set values ​​of the breast lesions in the subject's breast region, including:

[0023] The pre-trained intelligent analysis model for breast lesions is used to analyze any one of the multiple ultrasound images to obtain the set of values ​​for the BI-RADS feature set corresponding to the arbitrary ultrasound image.

[0024] The first feature value set is obtained from the value set of the BI-RADS feature set corresponding to the multi-frame ultrasound images according to a preset strategy.

[0025] In one embodiment, determining the BI-RADS classification of the breast lesion based on the first feature set, the second feature set, and the ultrasound image includes:

[0026] When the value of any BI-RADS feature in the value set of the BI-RADS feature set corresponding to any ultrasound image in the multi-frame ultrasound images is the same as the value of any BI-RADS feature in the second feature set, the weight of the ultrasound image in determining the value of any BI-RADS feature in the BI-RADS classification is increased; when the value of any BI-RADS feature in the value set of the BI-RADS feature set corresponding to any ultrasound image is different from the value of any BI-RADS feature in the second feature set, the weight of the ultrasound image in determining the value of any BI-RADS feature in the BI-RADS classification is decreased.

[0027] In one embodiment, before detecting the user's modification or confirmation operation on the first feature value set, the method further includes:

[0028] The ultrasound image and the first set of feature values ​​are displayed in a comparative manner on the display interface.

[0029] In a second aspect, embodiments of the present invention provide an ultrasound imaging device, comprising:

[0030] An ultrasound probe is used to emit ultrasound waves to the target tissue of the subject and receive the echo of the ultrasound waves returned by the target tissue. Based on the received ultrasound echo, an ultrasound echo signal is output, which carries tissue structure information of the target tissue.

[0031] The transmitting circuit is used to output the corresponding transmitting sequence to the ultrasonic probe according to the set mode, so as to control the ultrasonic probe to emit the corresponding ultrasonic waves;

[0032] The receiving circuit is used to receive the ultrasonic echo signal output by the ultrasonic probe and output ultrasonic echo data.

[0033] A display is used to output visual information;

[0034] A processor for performing an ultrasound imaging method of the breast as provided in any of the above embodiments.

[0035] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the ultrasound imaging method for breasts as provided in any of the above embodiments.

[0036] The ultrasound imaging method and device for breasts provided in this invention acquires ultrasound images of a subject's breast region. First, based on a pre-trained intelligent analysis model for breast lesions, the ultrasound images are analyzed to obtain a first set of feature values ​​corresponding to the BI-RADS feature set values ​​of the breast lesions in the subject's breast region. Then, user modifications or confirmations to the first set of feature values ​​are detected to obtain a second set of feature values. Finally, the BI-RADS grade of the breast lesion is determined based on the first set of feature values, the second set of feature values, and the ultrasound image. Since the second set of feature values ​​reflects not only the ultrasound image information of the breast lesion but also the user's judgment of the breast lesion based on clinical experience, determining the BI-RADS grade of the breast lesion based on the second set of feature values ​​helps improve the accuracy of BI-RADS grading. Attached Figure Description

[0037] Figure 1 This is a structural block diagram of an ultrasound imaging device provided in an embodiment of the present invention;

[0038] Figure 2 This invention provides an ultrasound imaging method for breast tissue according to an embodiment of the present invention.

[0039] Figure 3 An ultrasound imaging method for breast tissue provided in another embodiment of the present invention;

[0040] Figures 4A-4C This is a schematic diagram of a display interface provided in an embodiment of the present invention;

[0041] Figure 5 This invention provides an ultrasound imaging method for breast tissue, which is another embodiment of the present invention.

[0042] Figure 6 A schematic diagram of a display interface provided in another embodiment of the present invention;

[0043] Figure 7 This invention provides an ultrasound imaging method for breast tissue according to another embodiment of the invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0045] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0046] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).

[0047] like Figure 1 As shown, the ultrasound imaging device provided by the present invention may include: an ultrasound probe 20, a transmitting / receiving circuit 30 (i.e., a transmitting circuit 310 and a receiving circuit 320), a beamforming module 40, an IQ demodulation module 50, a memory 60, a processor 70, and a human-computer interaction device. The processor 70 may include a control module 710 and an image processing module 720.

[0048] The ultrasound probe 20 includes a transducer (not shown) composed of multiple array elements arranged in an array. These elements can be arranged in a row to form a linear array, or in a two-dimensional matrix to form a planar array, or they can form a convex array. Each element is used to emit an ultrasonic beam according to an excitation electrical signal, or to convert a received ultrasonic beam into an electrical signal. Therefore, each element can be used to achieve the mutual conversion between electrical pulse signals and ultrasonic beams, thereby emitting ultrasonic waves to a target area of ​​human tissue (e.g., the breast area in this embodiment), or to receive echoes of ultrasonic waves reflected back from the tissue. During ultrasound detection, the transmitting circuit 310 and the receiving circuit 320 can control which elements are used to emit ultrasonic beams and which are used to receive ultrasonic beams, or control the elements to be used in time-slotted manner to emit ultrasonic beams or receive echoes of ultrasonic beams. Elements participating in ultrasonic wave emission can be simultaneously excited by electrical signals to emit ultrasonic waves simultaneously; or elements participating in ultrasonic wave emission can be excited by several electrical signals with a certain time interval to continuously emit ultrasonic waves with a certain time interval.

[0049] In this embodiment, the user selects a suitable position and angle by moving the ultrasound probe 20 to emit ultrasound waves to the breast region 10 and receives the echo of the ultrasound waves returned from the breast region 10. The user obtains and outputs the electrical signal of the echo. The electrical signal of the echo is a channel analog electrical signal formed by the receiving array element as the channel, which carries amplitude information, frequency information and time information.

[0050] The transmitting circuit 310 generates a transmission sequence under the control of the control module 710 of the processor 70. This transmission sequence controls some or all of the multiple array elements to transmit ultrasound waves to the biological tissue. The transmission sequence parameters include the position and number of array elements, and the ultrasound beam transmission parameters (e.g., amplitude, frequency, number of transmissions, transmission interval, transmission angle, waveform, focusing position, etc.). In some cases, the transmitting circuit 310 also performs phase delay on the transmitted beam, allowing different transmitting array elements to transmit ultrasound waves at different times, so that each transmitted ultrasound beam can be focused in a predetermined region of interest. Different operating modes, such as B-image mode, C-image mode, and D-image mode (Doppler mode), may have different transmission sequence parameters. After the echo signal is received by the receiving circuit 320 and processed by subsequent modules and corresponding algorithms, a B-image reflecting the tissue anatomy, a C-image reflecting the tissue anatomy and blood flow information, and a D-image reflecting the Doppler spectrum can be generated.

[0051] The receiving circuit 320 receives the electrical signal of the ultrasonic echo from the ultrasonic probe 20 and processes it. The receiving circuit 320 may include one or more amplifiers, an analog-to-digital converter (ADC), etc. The amplifier amplifies the received ultrasonic echo signal after appropriate gain compensation, and the ADC samples the analog echo signal at predetermined time intervals, converting it into a digitized signal. The digitized echo signal still retains amplitude, frequency, and phase information. The data output from the receiving circuit 320 can be sent to the beamforming module 40 for processing, or to the memory 60 for storage.

[0052] The beamforming module 40 is signal-connected to the receiving circuit 320 and is used to perform beamforming processing such as delay and weighted summation on the signal output by the receiving circuit 320. Because the distance from the ultrasonic receiving point in the tested tissue to the receiving array element varies, the channel data of the same receiving point output by different receiving array elements has delay differences, requiring delay processing to align the phases and perform weighted summation on the different channel data of the same receiving point to obtain the beamformed ultrasonic image data. The ultrasonic image data output by the beamforming module 40 is also called radio frequency (RF) data. The beamforming module 40 outputs the RF data to the IQ demodulation module 50. In some embodiments, the beamforming module 40 can also output the RF data to the memory 60 for caching or storage, or directly output the RF data to the image processing module 720 of the processor 70 for image processing.

[0053] The beamforming module 40 can perform the above functions in hardware, firmware or software. For example, the beamforming module 40 may include a central controller circuit (CPU), one or more microprocessor chips or any other electronic components capable of processing input data according to specific logic instructions. When the beamforming module 40 is implemented in software, it can execute instructions stored on a tangible and non-transitory computer-readable medium (e.g., memory 60) to perform beamforming calculations using any appropriate beamforming method.

[0054] The IQ demodulation module 50 removes the signal carrier through IQ demodulation, extracts the tissue structure information contained in the signal, and filters to remove noise. The signal obtained at this time is called the baseband signal (IQ data pair). The IQ demodulation module 50 outputs the IQ data pair to the image processing module 720 of the processor 70 for image processing. In some embodiments, the IQ demodulation module 50 also outputs the IQ data pair to the memory 60 for buffering or storage, so that the image processing module 720 can read the data from the memory 60 for subsequent image processing.

[0055] The processor 70 is configured to process input data according to specific logical instructions, including a central controller circuit (CPU), one or more microprocessors, a graphics controller circuit (GPU), or any other electronic components. It can control peripheral electronic components according to input instructions or predetermined instructions, or perform data reading and / or saving to the memory 60. It can also process input data by executing programs in the memory 60, such as performing one or more processing operations on acquired ultrasound data according to one or more operating modes. These processing operations include, but are not limited to, adjusting or limiting the form of ultrasound waves emitted by the ultrasound probe 20, generating various image frames for display on the display 80 of a subsequent human-computer interaction device, adjusting or limiting the content and form displayed on the display 80, or adjusting one or more image display settings on the display 80 (e.g., ultrasound images, interface components, locating regions of interest). The processor 70 provided in this embodiment can be used to execute the ultrasound imaging method for breasts provided in any embodiment of the present invention.

[0056] The image processing module 720 processes the data output from the beamforming module 40 or the IQ demodulation module 50 to generate a grayscale image showing the changes in signal strength within the scanning range. This grayscale image reflects the internal anatomical structure of the tissue and is called a B-image. The image processing module 720 can output the B-image to the display 80 of the human-computer interaction device for display.

[0057] Human-computer interaction devices are used for human-computer interaction, that is, to receive user input and output visual information; the user input can be received through a keyboard, operation buttons, mouse, trackball, etc., or a touch screen integrated with the display; the visual information output is displayed on a monitor.

[0058] The memory 60 may be a tangible and non-transitory computer-readable medium, such as a flash memory card, solid-state memory, hard disk, etc., for storing data or programs. For example, the memory 60 may be used to store acquired ultrasound data or image frames generated by the processor 70 that are not immediately displayed, or the memory 60 may store a graphical user interface, one or more default image display settings, or programming instructions for the processor, beamforming module, or IQ decoding module.

[0059] Please refer to Figure 2 ,based on Figure 1 The ultrasound imaging device shown provides a method for ultrasound imaging of the breast.

[0060] like Figure 2 As shown, the ultrasound imaging method for breasts provided in this embodiment may include:

[0061] S101. Obtain ultrasound images of the subject's breast area.

[0062] S102. The ultrasound image is analyzed based on the pre-trained intelligent analysis model of breast lesions to obtain the first feature value set corresponding to the BI-RADS feature set value of the breast lesion in the breast region of the subject. The intelligent analysis model of breast lesions is trained based on the sample ultrasound images labeled with the BI-RADS feature set value.

[0063] S103. Detect the user's modification or confirmation operation on the first feature value set to obtain the second feature value set.

[0064] S104. Determine the BI-RADS classification of the breast lesion based on the first set of eigenvalues, the second set of eigenvalues, and the ultrasound image.

[0065] In one optional implementation, determining the BI-RADS classification of the breast lesion based on the first set of feature values, the second set of feature values, and the ultrasound image may include:

[0066] Image information feature vectors are obtained based on the ultrasound images. Where, x i w is the feature vector corresponding to the i-th BI-RADS feature in the BI-RADS feature set extracted from the ultrasound image. i For x i The initial weights, where n is the number of BI-RADS features in the BI-RADS feature set;

[0067] If the i-th BI-RADS feature has the same value in both the first feature set and the second feature set, then w′ i =w i +Δ Adjust x i The weights; if the i-th BI-RADS feature has different values ​​in the first feature set and the second feature set, then by w′ i =w i -Δ Adjust x i The weights; where Δ is the weight adjustment amount, w′ i For the adjusted x i The weights;

[0068] Based on the feature vectors corresponding to each BI-RADS feature and their adjusted weights, the image information feature vectors used to determine the BI-RADS classification of the breast lesion are determined.

[0069] The image information feature vector X′ used to determine the BI-RADS classification of the breast lesion is input into a pre-trained BI-RADS classification model to obtain the BI-RADS classification of the breast lesion. The BI-RADS classification model is trained based on the sample feature vector labeled with the BI-RADS classification.

[0070] To further improve the accuracy of BI-RADS classification, attribute feature vectors can be added when determining the BI-RADS classification. These can include:

[0071] Obtain the attribute feature vector based on the second feature value set. Among them, y i Let r be the value of the i-th BI-RADS feature in the second feature set. i For y i The initial weights;

[0072] If the i-th BI-RADS feature has the same value in both the first feature set and the second feature set, then through r′ i =r i +Δ Adjust y i The weights; if the i-th BI-RADS feature has different values ​​in the first feature set and the second feature set, then through r′ i =r i -Δ Adjust y i The weights; where r′ i After adjustment y i The weights;

[0073] Based on the values ​​of each BI-RADS feature in the second feature set and their corresponding adjusted weights, an attribute information feature vector for determining the BI-RADS grading of the breast lesion is determined.

[0074] The image information feature vector X′ used to determine the BI-RADS classification of the breast lesion and the attribute information feature vector Y′ used to determine the BI-RADS classification of the breast lesion are fused together.

[0075] The fused feature vector is input into the pre-trained BI-RADS grading model to obtain the BI-RADS grading of the breast lesion.

[0076] The ultrasound imaging method for breast tissue provided in this embodiment acquires ultrasound images of the breast region of a subject. First, it analyzes the ultrasound images based on a pre-trained intelligent analysis model for breast lesions to obtain a first set of feature values ​​corresponding to the BI-RADS feature set values ​​of the breast lesions in the subject's breast region. Then, it detects user modifications or confirmations to the first set of feature values ​​to obtain a second set of feature values. Finally, it determines the BI-RADS grade of the breast lesion based on the first set of feature values, the second set of feature values, and the ultrasound image. Since the second set of feature values ​​not only reflects the ultrasound image information of the breast lesion but also reflects the user's judgment of the breast lesion based on clinical experience, determining the BI-RADS grade of the breast lesion based on the second set of feature values ​​helps improve the accuracy of BI-RADS grading.

[0077] Please refer to Figure 3 ,based on Figure 1 The ultrasound imaging device shown provides a method for ultrasound imaging of the breast.

[0078] like Figure 3 As shown, the ultrasound imaging method for breasts provided in this embodiment may include:

[0079] S201. Obtain ultrasound images of the subject's breast area.

[0080] In this embodiment, ultrasound images of the breast region can be acquired in real time or read from ultrasound images pre-stored in a storage medium. For example, it can be obtained through... Figure 1 The ultrasound probe 20 of the ultrasound imaging device emits ultrasound waves to the breast region of the subject in real time. The receiving circuit 320 processes the ultrasound echo electrical signals received from the ultrasound probe 20, and then processes them through the beamforming module 40, the IQ demodulation module 50, and the image processing module 720 to acquire ultrasound images of the subject's breast region in real time. For example, pre-acquired ultrasound images of the subject's breast region can also be retrieved from the memory 60.

[0081] In this embodiment, either a single ultrasound image of the subject's breast region or multiple ultrasound images of the subject's breast region can be acquired. The number of ultrasound images is not limited in this embodiment.

[0082] S202. Based on the ultrasound images, determine the set of values ​​corresponding to the BI-RADS feature set of the breast lesions in the patient's breast region to obtain the first feature value set.

[0083] In this embodiment, after acquiring ultrasound images of the patient's breast region, the region of interest (ROI) of the breast lesion can be detected in the acquired ultrasound images, the boundary of the breast lesion can be segmented, and then the BI-RADS features of the breast lesion can be analyzed to determine the set of values ​​corresponding to the BI-RADS feature set of the breast lesion in the patient's breast region.

[0084] Breast lesion ROI detection can be performed using deep learning, machine learning, or traditional image processing methods. This embodiment does not limit the specific implementation method of breast lesion ROI detection. Deep learning-based breast lesion ROI detection requires first training a deep learning ROI detection network based on collected breast region ultrasound image data and annotations of breast lesion ROIs in ultrasound images by senior physicians. ROIs can be annotated using coordinate information, such as rectangular boxes. Deep learning ROI detection networks can use, but are not limited to, RCNN, Faster RCNN, SSD, and YOLO. During network training, the error between the detected and annotated breast lesion ROIs is calculated during iterations, and the weights in the network are continuously updated with the goal of minimizing the error. This process is repeated until the detection result gradually approaches the true value of the breast lesion ROI, resulting in a trained breast lesion ROI detection model. This model can achieve automated detection and extraction of breast lesion ROIs from input ultrasound image data. Traditional image processing methods or machine learning for breast lesion ROI detection typically involve the following steps: (a) finding candidate regions using image processing methods, such as the Select Search algorithm; (b) transforming the candidate regions to a fixed size and extracting feature vectors such as gradients and textures using image processing methods, such as Sift operators, HoG operators, and GLCM gray-level co-occurrence matrix; (c) training the feature vectors of the candidate regions using traditional machine learning algorithms to obtain a classification model for the candidate regions; and (d) obtaining the bounding box of the target, i.e., the breast lesion ROI, using regression methods. Another machine learning-based method for breast lesion ROI extraction involves training a machine learning model based on collected ultrasound images and breast lesion ROI annotations. For example, using SVM, K-means, or C-means machine learning models to perform binary classification on the grayscale or texture values ​​of pixels to determine whether each pixel belongs to the ROI region, thereby achieving breast lesion ROI extraction.

[0085] Methods for segmenting the boundaries of breast lesions include, but are not limited to: (1) Extracting the boundaries of the detected breast lesion ROI or ultrasound full image based on a deep learning segmentation network. For example, Unet, FCN, and networks improved based on them can be used for deep learning segmentation. When performing deep learning segmentation, the input is the ultrasound image and the corresponding labeled area of ​​the ultrasound image. The labeled area can be a binary image of the breast lesion, or the location information of the breast lesion can be written into a labeled file such as xml or json. Calculate the error between the segmentation result output by the deep learning segmentation network and the labeled result, and iterate to minimize the error until the segmentation result approaches the true value. (2) Using a multi-task deep learning network for simultaneous detection and segmentation to extract the boundaries. Commonly used networks include mask-Rcnn, PolarMask, SOLO, etc. The first step of such networks is usually to locate the approximate location of the ROI, and then to perform fine segmentation of the target area. (3) Using traditional image processing algorithms, such as region-based segmentation algorithms, including region growing, watershed algorithm, Otsu thresholding, etc.; and gradient-based segmentation algorithms, such as Sobel, Canny operator, etc. (4) A machine learning-based method for segmenting breast lesions is adopted. Based on the collected ultrasound images and annotation results, a machine learning segmentation model is trained. Machine learning models such as SVM, Kmeans, and Cmeans can be used to perform binary classification on the gray value or texture value of the image pixels to determine whether each pixel or the texture feature vector representing the current pixel belongs to the breast lesion ROI, thereby achieving the segmentation of the breast lesion ROI boundary.

[0086] Methods for analyzing BI-RADS features of breast lesions include, but are not limited to: analyzing each BI-RADS feature based solely on deep learning, analyzing each BI-RADS feature based solely on traditional image features combined with machine learning, and analyzing BI-RADS features by combining the above two approaches. Specifically, it may include: (1) predicting multiple BI-RADS features based on a multi-task deep learning network. In one optional implementation, the extracted ROI region of the breast lesion can be used as input, and multiple branches of the multi-task deep learning network can be used directly to predict each BI-RADS feature. For example, shape type, orientation type, echo type, calcification type, and edge type can be regarded as 5 prediction tasks, and a large multi-task deep learning network contains 5 branches to handle 5 different prediction tasks respectively. The backbone network used in each convolutional block includes, but is not limited to, typical deep learning convolutional classification networks, such as AlexNet, ResNet, VGG, etc. When training the multi-task deep learning network, the classification subnetworks of each BI-RADS feature can be trained separately, or the entire network can be trained simultaneously. Specifically, by calculating the error between the prediction results and the calibration results of each branch, the calibration results can be understood as the true results of the shape type, orientation type, echo type, calcification type and edge type of breast lesions. Then, through continuous iteration, the prediction results gradually approach the calibration results, and finally a multi-task deep learning network model that can perform multiple BI-RADS feature predictions is obtained. (2) Construct a deep learning network for each BI-RADS feature, and construct multiple deep learning networks in parallel to analyze multiple BI-RADS features. The deep learning network can adopt deep learning convolutional classification network, including but not limited to AlexNet, ResNet, VGG, etc. (3) Use feature extraction algorithms to extract features for each BI-RADS feature, set appropriate thresholds according to the extracted features, or concatenate the features and use machine learning algorithms for analysis. In an optional implementation, features of breast lesions can be extracted, including but not limited to histogram, gray-level co-occurrence matrix features, etc., and input into machine learning models such as SVM, Kmean, KNN to predict the echo type of breast lesions to obtain the analysis results of the echo type of breast lesions. (4) Treat each BI-RADS feature as a prediction or classification task. For different BI-RADS features, design an algorithm or model suitable for the feature based on different schemes. The scheme can be based on deep learning or on traditional image processing methods combined with machine learning.

[0087] In one optional implementation, the BI-RADS feature set may include shape type, orientation type, edge type, echo type, posterior echo type, calcification type, and blood supply type. For example, if the BI-RADS features of a breast lesion in the patient's breast region determined based on ultrasound images are: irregular shape, parallel orientation, angular edge, hypoechoic echo, no change in posterior echo, intramass calcification, and internal blood supply, then the first feature set could be: irregular shape, parallel, angular, hypoechoic, no change, intramass calcification, and internal blood supply. It should be noted that the BI-RADS feature set in this embodiment may also include more or fewer BI-RADS features than described above. For example, the BI-RADS feature set may only include shape type, orientation type, and edge type.

[0088] In this embodiment, the BI-RADS classification of breast lesions in the breast region of the subject can also be determined based on ultrasound images.

[0089] S203. Display the first set of feature values ​​on the display interface.

[0090] In this embodiment, after obtaining the first feature value set, to facilitate user viewing and modification or confirmation of the first feature value set, it can be displayed on the display interface. For example, BI-RADS feature names can be associated with their corresponding values.

[0091] In one optional implementation, to further facilitate user modification or confirmation of the first feature value set, the ultrasound images of the patient's breast region and the first feature value set can be displayed on the display interface in a comparative manner. For example, the ultrasound images and the first feature value set can be displayed in different areas of the display interface, allowing the user to view the ultrasound images while verifying the values ​​of each BI-RADS feature in the first feature value set. When multiple ultrasound images of the patient's breast region are obtained, the multiple ultrasound images can be automatically scrolled at a preset frequency, or only the ROIs of breast lesions in the multiple ultrasound images can be displayed.

[0092] In this embodiment, the BI-RADS classification of breast lesions in the subject's breast region, determined based on ultrasound images, can also be displayed on the display interface.

[0093] Please refer to Figure 4A , Figure 4A This is a schematic diagram illustrating the display of a first set of feature values ​​on a display interface, as provided in one embodiment.

[0094] S204. Detect the user's modification or confirmation operation on the first feature value set to obtain the second feature value set.

[0095] After viewing the first set of feature values, if a user has doubts about the value of a certain BI-RADS feature, they can modify the value using an input device such as a mouse or keyboard. If they agree with the value of a certain BI-RADS feature, they can confirm it using the same input device. In practice, drop-down menus, radio buttons, or similar methods can be used to display the value range of each BI-RADS feature for user modification or confirmation.

[0096] Understandably, the second set of eigenvalues ​​can reflect not only the ultrasound image information of breast lesions, but also the user's judgment of breast lesions based on clinical experience.

[0097] S205. Display the second set of feature values ​​on the display interface.

[0098] To facilitate users in viewing the values ​​of each BI-RADS feature after modification or confirmation, a second feature value set can be displayed on the display interface. For example, the second feature value set can be displayed in real time when the user makes modifications or confirms.

[0099] For example, if the user thinks Figure 4A The angle of the edge type was not accurate enough, so it was modified to "differential lobes, burrs", and... Figure 4A If the values ​​of other BI-RADS features have been confirmed, the display interface will show the following: Figure 4B Display the second set of eigenvalues. Figure 4B This is a schematic diagram illustrating the display of a second feature value set on a display interface, as provided in one embodiment. Users can save the second feature value set using the "Save" button and initiate BI-RADS grading analysis for breast lesions using the "Analyze" button.

[0100] S206. Determine the BI-RADS classification of breast lesions based on the second set of eigenvalues.

[0101] The second feature set, obtained by modifying or confirming the values ​​of each BI-RADS feature in the first feature set based on ultrasound images, can not only reflect the information of breast lesions in ultrasound images, but also the user's judgment information on breast lesions. Therefore, determining the BI-RADS classification of breast lesions based on the second feature set will help improve the accuracy of BI-RADS classification.

[0102] In one alternative implementation, the second feature set can be used as input to output the BI-RADS classification of the breast lesion. For example, a machine learning approach can be used to pre-train a BI-RADS classification model based on a BI-RADS feature set labeled with the BI-RADS classification.

[0103] In another alternative implementation, to fully utilize the information from the ultrasound images, the ultrasound images of the breast region and the second set of feature values ​​can be used simultaneously as input to output the BI-RADS classification of the breast lesion. For example, machine learning methods can be employed to pre-train a BI-RADS classification model based on ultrasound images labeled with BI-RADS classifications and a set of BI-RADS feature values.

[0104] S207. Display the BI-RADS classification of breast lesions on the display interface.

[0105] To facilitate users' viewing of the BI-RADS classification of breast lesions and enable them to perform corresponding diagnostic and treatment operations based on the BI-RADS classification, this embodiment displays the BI-RADS classification of the breast lesions on the display interface after it has been determined. Figure 4B Taking the set of second eigenvalues ​​shown as an example, Figure 4C The BI-RADS classification of the breast lesion determined based on this second set of eigenvalues ​​is displayed. It is understood that, for ease of viewing, both the second set of eigenvalues ​​and the BI-RADS classification of the breast lesion can be displayed simultaneously on the interface.

[0106] The ultrasound imaging method for breast tissue provided in this embodiment acquires ultrasound images of the patient's breast region. First, based on the ultrasound images, it determines the value set corresponding to the BI-RADS feature set of the breast lesion in the patient's breast region, obtaining a first feature value set, which is then displayed on the display interface. Next, it detects user modifications or confirmations to the first feature value set, obtaining a second feature value set, which is also displayed on the display interface. Finally, it determines the BI-RADS grade of the breast lesion based on the second feature value set, and displays the BI-RADS grade of the breast lesion on the display interface. Since the second feature value set not only reflects the ultrasound image information of the breast lesion but also reflects the user's judgment of the breast lesion based on clinical experience, determining the BI-RADS grade of the breast lesion based on the second feature value set helps improve the accuracy of BI-RADS grading.

[0107] Based on the above embodiments, the following will provide a more detailed explanation of how to determine the BI-RADS grading of breast lesions based on the second feature set. The method of determining the BI-RADS grading based on the second feature set, combined with the user's clinical feedback information, can be achieved by weighting and optimizing one or more of the ultrasound images used in the BI-RADS grading algorithm, the feature vectors corresponding to the ultrasound images, and the BI-RADS feature values. The weighting and optimization strategy may include: (1) strengthening the role of ultrasound images, feature vectors corresponding to ultrasound images, and / or corresponding BI-RADS feature values ​​for BI-RADS features that have not been modified (i.e. confirmed) by the doctor in the BI-RADS grading process. The significance of this is: For the BI-RADS features that the doctor did not modify, it indicates that the BI-RADS feature values ​​obtained by the intelligent algorithm analysis are consistent with the doctor's evaluation. The corresponding ultrasound image, the feature vector corresponding to the ultrasound image and the BI-RADS feature values ​​are relatively reliable and reasonable for BI-RADS grading analysis. Therefore, the role of the corresponding ultrasound image, the feature vector corresponding to the ultrasound image and / or the BI-RADS feature values ​​in BI-RADS grading should be strengthened. (2) The role of the ultrasound image, the feature vector corresponding to the ultrasound image and / or the BI-RADS feature values ​​corresponding to the BI-RADS features modified by the doctor in the BI-RADS grading process is weakened. The significance is that: for the BI-RADS features modified by the doctor, it indicates that the BI-RADS feature values ​​obtained by the intelligent algorithm analysis are inconsistent with the doctor's evaluation. The corresponding ultrasound image, the feature vector corresponding to the ultrasound image and the BI-RADS feature values ​​are relatively unreliable for BI-RADS grading analysis. For example, the ultrasound image may be blurry or the features are not obvious, which leads to inaccurate BI-RADS feature analysis results.

[0108] In one optional implementation, determining the BI-RADS grading of breast lesions based on the second feature set may specifically include: for any BI-RADS feature in the BI-RADS feature set, extracting the feature vector corresponding to any BI-RADS feature from the ultrasound image; when any BI-RADS feature has the same value in the first and second feature sets, increasing the weight of the feature vector corresponding to any BI-RADS feature in determining the BI-RADS grading of breast lesions; when any BI-RADS feature has different values ​​in the first and second feature sets, decreasing the weight of the feature vector corresponding to any BI-RADS feature in determining the BI-RADS grading of breast lesions.

[0109] When the BI-RADS feature takes the same value in the first and second eigenvalue sets, it indicates that the user has not modified the value of the BI-RADS feature. This suggests that the assessment result based on ultrasound images using the intelligent algorithm is consistent with the user's assessment result based on clinical experience, and the role of the eigenvector corresponding to this BI-RADS feature in determining the BI-RADS grade should be strengthened. When the BI-RADS feature takes different values ​​in the first and second eigenvalue sets, it indicates that the user has modified the value of the BI-RADS feature. This suggests that the assessment result based on ultrasound images using the intelligent algorithm is inconsistent with the user's assessment result based on clinical experience, and the role of the eigenvector corresponding to this BI-RADS feature in determining the BI-RADS grade should be weakened.

[0110] Taking the BI-RADS feature set, which includes shape type, orientation type, edge type, echo type, posterior echo type, calcification type, and blood supply type, as an example, X represents the image information feature vector obtained based on ultrasound images for the BI-RADS grading task. For example, X is the feature vector input to the BI-RADS grading model.

[0111] X=w0x0+w1x1+w2x2+w3x3+w4x4+w5x5+w6x6;

[0112] Where x0 is the feature vector corresponding to the shape type extracted from the ultrasound image, and w0 is the weight of the feature vector. Similarly, x1, x2, x3, x4, x5, and x6 are the feature vectors corresponding to the direction type, edge type, echo type, posterior echo type, calcification type, and blood supply type extracted from the ultrasound image, respectively, and w1 to w6 are the weights of each feature vector.

[0113] After obtaining the first set of feature values, the user modifies the values ​​of shape type, edge type, and calcification type through the input device and confirms the values ​​of other BI-RADS features. Then, the weights can be adjusted for weighted optimization as follows: w′0=w0-Δ; w′2=w2-Δ; w′5=w5-Δ; w′1=w1+Δ; w′3=w3+Δ; w′4=w4+Δ; w′6=w6+Δ; X′=w′0x0+w′1x1+w′2x2+w′3x3+w′4x4+w′5x5+w′6x6; where Δ is the weight adjustment amount, which can be preset, such as setting Δ to 0.1.

[0114] Then, the BI-RADS classification is determined based on the weighted optimized image information feature vector X′, so as to strengthen the role of the feature vectors corresponding to the user-confirmed BI-RADS features in the BI-RADS classification process, and weaken the role of the feature vectors corresponding to the user-modified BI-RADS features in the BI-RADS classification process, thereby improving the accuracy of BI-RADS classification.

[0115] In another optional implementation, determining the BI-RADS grade of breast lesions based on the second feature set may specifically include: for any BI-RADS feature in the BI-RADS feature set, when the value of any BI-RADS feature is the same in the first feature set and the second feature set, increasing the weight of the value of any BI-RADS feature in the second feature set in determining the BI-RADS grade of breast lesions; when the value of any BI-RADS feature is different in the first feature set and the second feature set, decreasing the weight of any BI-RADS feature in the second feature set in determining the BI-RADS grade of breast lesions.

[0116] When the BI-RADS feature takes the same value in the first and second feature sets, it indicates that the user has not modified the value of the BI-RADS feature. This shows that the assessment result based on ultrasound images using the intelligent algorithm is consistent with the user's assessment result based on clinical experience, which can strengthen the role of the BI-RADS feature value in determining the BI-RADS grade. When the BI-RADS feature takes different values ​​in the first and second feature sets, it indicates that the user has modified the value of the BI-RADS feature. This shows that the assessment result based on ultrasound images using the intelligent algorithm is inconsistent with the user's assessment result based on clinical experience, which can weaken the role of the BI-RADS feature value in determining the BI-RADS grade.

[0117] Taking the BI-RADS feature set, which includes shape type, orientation type, edge type, echo type, posterior echo type, calcification type, and blood supply type, as an example, Y represents the BI-RADS attribute feature vector obtained from ultrasound images for the BI-RADS grading task.

[0118] Y=r0y0+r1y1+r2y2+r3y3+r4y4+r5y5+r6y6;

[0119] Where y0 is the shape type value in the second feature set. Similarly, y1, y2, y3, y4, y5, and y6 are the orientation type, edge type, echo type, posterior echo type, calcification type, and blood supply type values ​​in the second feature set, respectively. r0 to r6 are the initial weights of each BI-RADS feature value. Assuming the user modifies the values ​​of shape type, edge type, and calcification type via input device, and confirms the values ​​of other BI-RADS features, then weighted optimization can be performed by adjusting the weights as follows: r′0=r0-Δ; r′2=r2-Δ; r′5=r5-Δ; r′1=r1+Δ; r′3=r3+Δ; r′4=r4+Δ; r′6=r6+Δ; Y′=r′0y0+r′1y1+r′2y2+r′3y3+r′4y4+r′5y5+r′6y6; where Δ is the weight adjustment amount, which can be preset, such as setting Δ to 0.1.

[0120] Then, the BI-RADS classification is determined based on the weighted optimized BI-RADS attribute feature vector Y′, so as to strengthen the role of user-confirmed BI-RADS feature values ​​in the BI-RADS classification process and weaken the role of user-modified BI-RADS feature values ​​in the BI-RADS classification process, thereby improving the accuracy of BI-RADS classification.

[0121] The above embodiments improve the accuracy of BI-RADS classification by weighting and optimizing the image information feature vector X and the BI-RADS attribute feature vector Y. Alternatively, weighted optimization can be performed simultaneously on both the image information feature vector X and the BI-RADS attribute feature vector Y, meaning the BI-RADS classification can be determined using the weighted optimized image information feature vector X′ and the weighted optimized BI-RADS attribute feature vector Y′. For example, X′+Y′ can be used as the feature vector for determining the BI-RADS classification, where "+" indicates the fusion of the two feature vectors, such as feature vector concatenation. This embodiment does not limit the fusion method.

[0122] When the acquired ultrasound images of the patient's breast region are multiple frames, each frame can be analyzed separately to obtain the set of values ​​for the BI-RADS feature set corresponding to each frame. Determining the BI-RADS grading of breast lesions based on the second feature set specifically includes: when the value of any BI-RADS feature in the set of values ​​for any frame of the multi-frame ultrasound images is the same as the value of any BI-RADS feature in the second feature set, increasing the weight of any frame of the ultrasound images in determining the value of any BI-RADS feature in the BI-RADS grading; when the value of any BI-RADS feature in the set of values ​​for any frame of the ultrasound images is different from the value of any BI-RADS feature in the second feature set, decreasing the weight of any frame of the ultrasound images in determining the value of any BI-RADS feature in the BI-RADS grading.

[0123] The above embodiments detail how to determine BI-RADS grading by combining user feedback on BI-RADS feature values ​​with ultrasound images of the breast region. This involves improving the accuracy of BI-RADS grading by detecting user confirmation or modification of BI-RADS feature values. The following will explain how to correct the values ​​of the BI-RADS information set by combining user feedback on the BI-RADS information set with ultrasound images of the breast region to improve the accuracy of BI-RADS information. The BI-RADS information set includes the BI-RADS feature set and the BI-RADS grading; users can confirm or modify both BI-RADS feature values ​​and BI-RADS grading. Please refer to [link / reference]. Figure 5 , Figure 5 This is another embodiment of the present invention providing an ultrasound imaging method for the breast. For example... Figure 5 As shown, the ultrasound imaging method for breasts provided in this embodiment may include:

[0124] S401. Obtain ultrasound images of the subject's breast region.

[0125] The specific implementation method for obtaining ultrasound images of the subject's breast region in this embodiment can refer to step S201 in the above embodiment, and will not be repeated here.

[0126] S402. Determine the value set corresponding to the BI-RADS information set of the breast lesion in the breast region of the subject based on the ultrasound image, so as to obtain the first value set, wherein the BI-RADS information set includes the BI-RADS feature set and the BI-RADS classification.

[0127] In one optional implementation, after acquiring ultrasound images of the patient's breast region, regions of interest (ROIs) of breast lesions can be detected in the acquired ultrasound images. The boundaries of the breast lesions are segmented, and then the BI-RADS information of the breast lesions is analyzed to determine the value set corresponding to the BI-RADS information set of the breast lesions in the patient's breast region. The specific implementation methods for detecting breast lesion ROIs and segmenting breast lesion boundaries can refer to step S202 in the above embodiments, and will not be repeated here. The analysis of the BI-RADS information of breast lesions can also refer to the method for analyzing the BI-RADS features of breast lesions in step S202, simply by adding BI-RADS grading to the output.

[0128] The following sections will elaborate on how to determine the BI-RADS information set corresponding to the breast lesions in the patient's breast region based on ultrasound images, using both traditional image processing methods and deep learning methods. When using traditional image processing methods, the first step is to extract feature vectors corresponding to the breast lesions from the ultrasound images. These feature vectors include one or more of the following: histogram, gray-level co-occurrence matrix features, scale-invariant feature transform (SIFT) features, and histogram of oriented gradient (HOG) features. Then, the BI-RADS information set corresponding to the breast lesions in the patient's breast region is determined based on these feature vectors. When using deep learning methods, the regions of interest (ROIs) of breast lesions can first be automatically or manually acquired. For example, an ultrasound image can be input into a pre-trained ROI detection model to obtain the ROIs of breast lesions in the ultrasound image. The ROI detection model is trained based on ultrasound images annotated with ROIs of breast lesions. Alternatively, the ROIs can be obtained by detecting the operator's tracing of ROIs in the ultrasound image. Then, the ROIs of breast lesions in the ultrasound image are input into a pre-trained BI-RADS information recognition model to obtain the set of values ​​corresponding to the BI-RADS information set of the breast lesion. The BI-RADS information recognition model is trained based on ultrasound images annotated with BI-RADS information values. The BI-RADS information recognition model can employ a multi-task deep learning network, where each branch of the multi-task deep learning network is used to recognize one type of BI-RADS information; or, the BI-RADS information recognition model can employ multiple parallel deep learning networks, each of which is used to recognize one type of BI-RADS information.

[0129] In this embodiment, the BI-RADS information set includes the BI-RADS feature set and the BI-RADS classification. This means that users can modify or confirm both the BI-RADS features and the BI-RADS classification. The BI-RADS feature set can include shape type, orientation type, edge type, echo type, posterior echo type, calcification type, and blood supply type.

[0130] S403. Display the first set of values ​​on the display interface.

[0131] In this embodiment, after obtaining the first set of values, to facilitate user viewing and modification or confirmation of the first set of values, the first set of values ​​can be displayed on the display interface. For example, the names of BI-RADS information can be associated with their corresponding values.

[0132] In one optional implementation, to further facilitate user modification or confirmation of the first set of values, the ultrasound image of the subject's breast region and the first set of values ​​can be displayed on the display interface in a comparative manner. For example, the ultrasound image and the first set of values ​​can be displayed in different areas of the display interface, allowing the user to view the ultrasound image while simultaneously verifying the values ​​of each BI-RADS information in the first set of values. Please refer to [reference needed]. Figure 6 , Figure 6 This is a schematic diagram of a display interface provided in another embodiment of the present invention. Figure 6 The ultrasound image of the breast region is displayed on the left side of the display interface, and the name and corresponding value of each BI-RADS information are displayed on the right side. It should be noted that this embodiment does not limit the positional relationship and display method of the ultrasound image and the first value set on the display interface.

[0133] S404. Detect the user's modification or confirmation operation on the first set of values ​​to obtain the second set of values.

[0134] After viewing the first set of values, if a user has doubts about a value of a BI-RADS, they can modify that value using input devices such as a mouse or keyboard; if they agree with a value, they can confirm it using the same input device. In practice, dropdown menus, radio buttons, or similar methods can be used to display the value ranges for each BI-RADS for user modification or confirmation.

[0135] Understandably, the second set of values ​​can reflect not only the ultrasound image information of breast lesions, but also the user's judgment of breast lesions based on clinical experience.

[0136] S405. Display the second set of values ​​on the display interface.

[0137] To facilitate users in viewing the values ​​of each BI-RADS information after modification or confirmation, a second set of values ​​can be displayed on the display interface. For example, the second set of values ​​can be displayed in real time when the user makes modifications or confirms.

[0138] S406. Determine the value set corresponding to the BI-RADS information set of the breast lesion based on the second value set and the ultrasound image, so as to obtain the third value set.

[0139] The second set of values, obtained by modifying or confirming the values ​​of each BI-RADS information in the first set of values ​​obtained solely from ultrasound images, fully reflects the user's judgment information on breast lesions. Therefore, by combining the second set of values ​​with ultrasound images to determine the values ​​of the BI-RADS information set for breast lesions, the accuracy of BI-RADS information values ​​can be significantly improved.

[0140] In one alternative implementation, the second set of values ​​and the ultrasound image can be used as inputs to output a third set of values, which can be achieved using machine learning methods.

[0141] by Figure 6 As shown in the example, by analyzing the ultrasound images of the patient's breast area, the following results were obtained: Figure 6 The BI-RADS values ​​shown represent the first set of values. If the doctor believes the edge type value is inaccurate, they can change it from "angular" to "microlobulation, spiculation," and then confirm the other BI-RADS values. Figure 6 The "Analyze" button can start the system to re-analyze. During the re-analysis, the BI-RADS information values ​​will be updated based on the second set of values ​​and the ultrasound images, and the BI-RADS classification will be updated from 4B to 4C.

[0142] S407. Display the third set of values ​​on the display interface.

[0143] To facilitate users' viewing of the final values ​​of each BI-RADS information for breast lesions, enabling them to perform corresponding diagnostic and treatment operations based on these final values, this embodiment, after determining the final value set (i.e., the third value set) corresponding to the BI-RADS information set of the breast lesion based on the second value set and ultrasound images, can display the third value set on the display interface. Specifically, the BI-RADS information name can be associated with the BI-RADS information value for display.

[0144] The ultrasound imaging method for breast tissue provided in this embodiment acquires ultrasound images of the patient's breast region. First, based on the ultrasound images, it determines the value set corresponding to the BI-RADS information set of the breast lesion in the patient's breast region, obtaining a first value set, which is then displayed on the display interface. Next, it detects user modifications or confirmations to the first value set, obtaining a second value set, which is also displayed on the display interface. Finally, based on the second value set and the ultrasound images, it determines the value set corresponding to the BI-RADS information set of the breast lesion, obtaining a third value set, which is also displayed on the display interface. Since the second value set fully reflects the user's judgment of the breast lesion based on clinical experience, combining the second value set to determine the BI-RADS information value of the breast lesion helps improve accuracy.

[0145] When the acquired ultrasound images of the patient's breast region are multiple frames, determining the value set corresponding to the BI-RADS information set of the breast lesions in the patient's breast region based on the ultrasound images to obtain the first value set may specifically include: analyzing any one frame of the multiple ultrasound images to obtain the value set of the BI-RADS information set corresponding to that frame; and obtaining the first value set from the value set of the BI-RADS information set corresponding to the multiple ultrasound images according to a preset strategy. The specific implementation of analyzing the ultrasound images to obtain the value set of the BI-RADS information set corresponding to the ultrasound images can be referred to step S202 in the above embodiment, and will not be repeated here. The preset strategy in this embodiment may, for example, employ methods such as weighted processing, voting-majority rule, or averaging.

[0146] Based on the above embodiments, determining the value set corresponding to the BI-RADS information set of the breast lesion according to the second value set and the ultrasound image to obtain the third value set specifically includes: when the value of any BI-RADS information in the value set of the BI-RADS information set corresponding to any frame of ultrasound image is the same as the value of any BI-RADS information in the second value set, increasing the weight of any frame of ultrasound image in determining the value of any BI-RADS information in the third value set; when the value of any BI-RADS information in the value set of the BI-RADS information set corresponding to any frame of ultrasound image is different from the value of any BI-RADS information in the second value set, decreasing the weight of any frame of ultrasound image in determining the value of any BI-RADS information in the third value set.

[0147] In one optional implementation, a third value set is obtained by determining the value set corresponding to the BI-RADS information set of the breast lesion based on the second value set and the ultrasound image. Specifically, this may include: for any BI-RADS information in the BI-RADS information set, extracting the feature vector corresponding to any BI-RADS information from the ultrasound image; when any BI-RADS information has the same value in the first and second value sets, increasing the weight of the feature vector corresponding to any BI-RADS information in determining the third value set; when any BI-RADS information has a different value in the first and second value sets, decreasing the weight of the feature vector corresponding to any BI-RADS information in determining the third value set.

[0148] When the BI-RADS information has the same value in the first and second value sets, it indicates that the user has not modified the value of the BI-RADS information. This means that the evaluation result based on ultrasound images using the intelligent algorithm is consistent with the user's evaluation result based on clinical experience. In this case, the role of the feature vector corresponding to the BI-RADS information in determining the third value set should be strengthened. When the BI-RADS information has different values ​​in the first and second value sets, it indicates that the user has modified the value of the BI-RADS information. This means that the evaluation result based on ultrasound images using the intelligent algorithm is inconsistent with the user's evaluation result based on clinical experience. In this case, the role of the feature vector corresponding to the BI-RADS information in determining the third value set should be weakened.

[0149] Taking the BI-RADS information set, which includes shape type, orientation type, edge type, echo type, posterior echo type, calcification type, blood supply type, and BI-RADS grading, as an example, X represents the image information feature vector input when determining the third value set.

[0150] X=w0x0+w1x1+w2x2+w3x3+w4x4+w5x5+w6x6+w7x7;

[0151] Where x0 is the feature vector corresponding to the shape type extracted from the ultrasound image, and w0 is the weight of this feature vector. Similarly, x1, x2, x3, x4, x5, x6, and x7 are the feature vectors corresponding to the direction type, edge type, echo type, posterior echo type, calcification type, blood supply type, and BI-RADS classification extracted from the ultrasound image, respectively, with w1 to w7 being the weights of each feature vector. This embodiment does not limit the type of feature vector or the specific extraction method; feature vectors can be, for example, gradients, gray-level co-occurrence matrices, etc. The feature vectors corresponding to each BI-RADS information can be of the same type or different types.

[0152] Assuming that after obtaining the first set of values, the user modifies the values ​​for shape type, edge type, and calcification type via input device, and confirms the values ​​for other BI-RADS information, then weighted optimization can be performed by adjusting the weights as follows: w′0=w0-Δ; w′2=w2-Δ; w′5=w5-Δ; w′1=w1+Δ; w′3=w3+Δ; w′4=w4+Δ; w′6=w6+Δ; w′7=w7+Δ; X′=w′0x0+w′1x1+w′2x2+w′3x3+w′4x4+w′5x5+w′6x6+w′7x7; where Δ is the weight adjustment amount, which can be preset, such as setting Δ to 0.1. Optionally, the weights can also be normalized after weighted optimization.

[0153] The third set of values ​​is determined based on the weighted optimized image information feature vector X′. By strengthening the role of the feature vectors corresponding to the BI-RADS information confirmed by the users and weakening the role of the feature vectors corresponding to the BI-RADS information modified by the users, the accuracy of the third set of values ​​is improved.

[0154] Based on any of the above embodiments, the third value set is obtained by determining the value set corresponding to the BI-RADS information set of the breast lesion according to the second value set and the ultrasound image, and may further include:

[0155] For any BI-RADS information in the BI-RADS information set, if the value of any BI-RADS information is the same in the first value set and the second value set, increase the weight of the value of any BI-RADS information in the second value set when determining the third value set; if the value of any BI-RADS information is different in the first value set and the second value set, decrease the weight of any BI-RADS information in the second value set when determining the third value set.

[0156] When the BI-RADS information has the same value in the first and second value sets, it indicates that the user has not modified the value of the BI-RADS information. This shows that the assessment result based on ultrasound images using the intelligent algorithm is consistent with the user's assessment result based on clinical experience, which can strengthen the role of the BI-RADS information value in determining the BI-RADS grade. When the BI-RADS information has different values ​​in the first and second value sets, it indicates that the user has modified the value of the BI-RADS information. This shows that the assessment result based on ultrasound images using the intelligent algorithm is inconsistent with the user's assessment result based on clinical experience, which can weaken the role of the BI-RADS information value in determining the BI-RADS grade.

[0157] The BI-RADS information set still includes shape type, orientation type, edge type, echo type, posterior echo type, calcification type, blood supply type, and BI-RADS grade. Y represents the BI-RADS attribute feature vector input when determining the third value set.

[0158] Y=r0y0+r1y1+r2y2+r3y3+r4y4+r5y5+r6y6+r7y7;

[0159] Where y0 is the shape type value in the second value set. Similarly, y1, y2, y3, y4, y5, y6, and y7 are the direction type, edge type, echo type, posterior echo type, calcification type, blood supply type, and BI-RADS grade values ​​in the second value set, respectively. r0 to r7 are the initial weights of each BI-RADS information value. Assuming the user modifies the values ​​for shape type, edge type, and calcification type via input device, and confirms the values ​​for other BI-RADS information, then weighted optimization can be performed by adjusting the weights as follows: r′0=r0-Δ; r′2=r2-Δ; r′5=r5-Δ; r′1=r1+Δ; r′3=r3+Δ; r′4=r4+Δ; r′6=r6+Δ; r′7=r7+Δ; Y′=r′0y0+r′1y1+r′2y2+r′3y3+r′4y4+r′5y5+r′6y6+r′7y7; where Δ is the weight adjustment amount, which can be preset, such as setting Δ to 0.1. Optionally, the weights can be normalized after weighted optimization.

[0160] Then, based on the weighted optimized BI-RADS attribute feature vector Y′, the third set of values ​​is determined. By strengthening the role of BI-RADS information values ​​confirmed by users and weakening the role of BI-RADS information values ​​modified by users, the accuracy of the third set of values ​​is improved.

[0161] The above embodiments improve the accuracy of the third value set by weighting and optimizing the image information feature vector X and the BI-RADS attribute feature vector Y. Alternatively, the image information feature vector X and the BI-RADS attribute feature vector Y can be weighted and optimized simultaneously, meaning the third value set can be determined using the weighted and optimized image information feature vector X′ and the weighted and optimized BI-RADS attribute feature vector Y′. For example, X′+Y′ can be used as the feature vector to determine the third value set, where "+" indicates the fusion of the two feature vectors, such as feature vector concatenation. This embodiment does not limit the fusion method.

[0162] The above embodiments illustrate how incorporating user feedback information can improve the accuracy of auxiliary diagnosis in breast lesions. The following will use specific embodiments to illustrate how to incorporate user feedback information to improve the accuracy of auxiliary diagnosis in other lesions, such as the TI-RADS grading system for thyroid imaging reporting and data systems. Please refer to... Figure 7 , Figure 7 Another embodiment of the present invention provides an ultrasound imaging method for the breast. For example... Figure 7 As shown, the ultrasound imaging method for breasts provided in this embodiment may include:

[0163] S601. Acquire ultrasound signals of the target tissue of the subject. The ultrasound signals include at least one of the following: analog signals, digital signals, in-phase quadrature IQ signals, radio frequency (RF) signals, and signals after logarithmic compression and grayscale conversion.

[0164] In this embodiment, ultrasound signals of the target tissue of the subject can be acquired in real time, or ultrasound signals of the target tissue pre-stored in a storage medium can be read. The ultrasound signal in this embodiment can be any one or more of the following: analog signal, digital signal, in-phase quadrature IQ signal, radio frequency (RF) signal, and signal after logarithmic compression and grayscale conversion.

[0165] S602. Determine the set of values ​​corresponding to the feature set of the lesion in the target tissue based on the ultrasound signal, so as to obtain the first feature value set.

[0166] In this embodiment, existing related technologies can be used to determine the set of values ​​corresponding to the feature set of lesions in the target tissue based on ultrasound signals. For example, artificial intelligence technology can be used to determine the feature values ​​of lesions in the target tissue based on ultrasound signals. This embodiment does not limit the specific implementation method.

[0167] S603. Display the first set of feature values ​​on the display interface.

[0168] To facilitate user viewing and modification or confirmation of the first feature value set, the first feature value set can be displayed on the display interface.

[0169] S604. Detect the user's modification or confirmation operation on the first feature value set to obtain the second feature value set.

[0170] After viewing the first set of feature values, if a user has doubts about the value of a certain lesion feature, they can modify the value using an input device such as a mouse or keyboard; if they agree with the value of a certain lesion feature, they can confirm the value using the same input device. In practice, drop-down menus, radio buttons, or similar methods can be used to display the value range of each lesion feature for user modification or confirmation.

[0171] Understandably, the second set of eigenvalues ​​can reflect not only the ultrasound signal information of the lesion, but also the user's judgment of the lesion based on clinical experience.

[0172] S605. Display the second set of feature values ​​on the display interface.

[0173] To facilitate users in viewing, modifying, or confirming the values ​​of each lesion feature, a second set of feature values ​​can be displayed on the interface.

[0174] S606. Determine the set of values ​​corresponding to the feature set of the lesion based on the first feature set, the second feature set, and the ultrasound signal to obtain the third feature set.

[0175] After obtaining a first set of feature values ​​based on ultrasound signals and a second set of feature values ​​by combining user feedback information with the first set of feature values, the first set of feature values, the second set of feature values, and the ultrasound signals can be combined to determine the value set corresponding to the feature set of the lesion, so as to obtain a third set of feature values.

[0176] S607. Display the third feature value set on the display interface.

[0177] To facilitate users in viewing the final values ​​of lesion features and enabling them to perform corresponding diagnostic and treatment operations based on these values, the set of third feature values ​​of the lesion can be displayed on the interface.

[0178] The breast ultrasound imaging method provided in this embodiment acquires ultrasound signals from the target tissue of the subject. First, it determines the set of values ​​corresponding to the feature set of lesions in the target tissue based on the ultrasound signals, obtaining a first feature value set, which is then displayed on the display interface. Next, it detects user modifications or confirmations to the first feature value set, obtaining a second feature value set, which is also displayed on the display interface. Finally, it determines the set of values ​​corresponding to the feature set of the lesions based on the first, second, and ultrasound signals, obtaining a third feature value set, which is also displayed on the display interface. By utilizing not only the ultrasound signals of the target tissue but also user feedback on the lesions when determining lesion feature values, the accuracy of lesion feature value determination is improved.

[0179] Based on the above embodiments, determining the set of values ​​corresponding to the feature set of the lesion based on the first feature set, the second feature set, and the ultrasound signal to obtain the third feature set may specifically include:

[0180] Strengthen the weighting of features with identical values ​​in the first and second eigenvalue sets and their corresponding ultrasound signals when determining the third eigenvalue set; and / or,

[0181] Reduce the weight of ultrasound signals corresponding to features with different values ​​in the first and second eigenvalue sets when determining the third eigenvalue set.

[0182] When the value of a lesion feature is the same in both the first and second feature sets, it indicates that the lesion value obtained based on ultrasound signal analysis is consistent with the doctor's assessment. This suggests that the value of the lesion feature and its corresponding ultrasound signal are relatively reliable. Increasing its weight in determining the third feature set helps improve the accuracy of the third feature set. Conversely, when the value of a lesion feature is different in both the first and second feature sets, it indicates that the lesion value obtained based on ultrasound signal analysis is inconsistent with the doctor's assessment. This suggests that the ultrasound signal corresponding to the lesion feature is unreliable. Reducing the weight of the ultrasound signal corresponding to the lesion feature in determining the third feature set helps improve its accuracy. The above weighted optimization strategies can be implemented individually or in combination.

[0183] This document describes various exemplary embodiments with reference to them. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope of this document. For example, various operational steps and components for performing operational steps can be implemented in different ways depending on the specific application or considering any number of cost functions associated with the operation of the system (e.g., one or more steps can be deleted, modified, or combined with other steps).

[0184] Furthermore, as those skilled in the art will understand, the principles herein can be reflected in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to form a machine, such that instructions, which execute on the computer or other programmable data processing apparatus, can generate means for performing a specified function. These computer program instructions may also be stored in a computer-readable storage medium that can instruct the computer or other programmable data processing apparatus to operate in a particular manner, such that instructions stored in the computer-readable storage medium can form an article of manufacture, including means for implementing the specified function. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that instructions, which execute on the computer or other programmable apparatus, can provide steps for implementing the specified function.

[0185] While the principles herein have been illustrated in various embodiments, numerous modifications to the structure, arrangement, proportions, elements, materials, and components, particularly suited to specific environmental and operational requirements, may be used without departing from the principles and scope of this disclosure. These modifications and other alterations or alterations will be included within the scope of this document.

[0186] The foregoing specific descriptions have been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Therefore, considerations for this disclosure are to be illustrative rather than restrictive, and all such modifications are to be included within its scope. Similarly, advantages, other advantages, and solutions to problems with respect to various embodiments have been described above. However, benefits, advantages, solutions to problems, and any elements that produce these, or make them more explicit, should not be construed as critical, essential, or necessary. The term “comprising” and any other variations thereof as used herein are non-exclusive inclusion, meaning that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed or not part of the process, method, system, article, or apparatus. Furthermore, the term “coupled” and any other variations thereof as used herein refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections, and / or any other connections.

[0187] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. An ultrasonic imaging device, characterized in that, include: An ultrasound probe is used to emit ultrasound waves to a target tissue of a subject and receive the echo of the ultrasound waves returned by the target tissue. Based on the received echo of the ultrasound waves, an ultrasound echo signal is output, and the ultrasound echo signal carries tissue structure information of the target tissue. The transmitting circuit is used to output a corresponding transmitting sequence to the ultrasonic probe according to a set mode, so as to control the ultrasonic probe to emit corresponding ultrasonic waves; A receiving circuit is used to receive the ultrasonic echo signal output by the ultrasonic probe and output ultrasonic echo data. A display is used to output visual information; Processor, used for: Acquire multiple ultrasound images of the breast region of the subject; The intelligent analysis model for breast lesions is based on a pre-trained model to analyze any one of the multiple ultrasound images to obtain the set of values ​​for the BI-RADS feature set corresponding to the arbitrary ultrasound image. According to a preset strategy, the first feature value set corresponding to the BI-RADS feature set value of the breast lesion in the subject's breast region is obtained from the set of values ​​for the BI-RADS feature set corresponding to the multiple ultrasound images. The intelligent analysis model for breast lesions is trained based on sample ultrasound images labeled with BI-RADS feature set values. Detect user actions that modify or confirm the first feature value set to obtain the second feature value set; Determining the BI-RADS classification of the breast lesion based on the first feature set, the second feature set, and the ultrasound image includes: When the value of any BI-RADS feature in the value set of the BI-RADS feature set corresponding to any ultrasound image in the multi-frame ultrasound images is the same as the value of any BI-RADS feature in the second feature set, the weight of the ultrasound image in determining the value of any BI-RADS feature in the BI-RADS classification is increased; when the value of any BI-RADS feature in the value set of the BI-RADS feature set corresponding to any ultrasound image is different from the value of any BI-RADS feature in the second feature set, the weight of the ultrasound image in determining the value of any BI-RADS feature in the BI-RADS classification is decreased.

2. The ultrasound imaging device as described in claim 1, characterized in that, The BI-RADS feature set includes shape type, orientation type, edge type, echo type, posterior echo type, calcification type, and blood supply type.

3. The ultrasound imaging device as described in claim 1, characterized in that, The processor determines the BI-RADS classification of the breast lesion based on the first feature set, the second feature set, and the ultrasound image, including: Image information feature vectors are obtained based on the ultrasound images. ; where x i w is the feature vector corresponding to the i-th BI-RADS feature in the BI-RADS feature set extracted from the ultrasound image. i For x i The initial weights, where n is the number of BI-RADS features in the BI-RADS feature set; If the i-th BI-RADS feature has the same value in both the first feature set and the second feature set, then... Adjust x i The weights; if the i-th BI-RADS feature has different values ​​in the first feature set and the second feature set, then by Adjust x i The weights; where △ is the weight adjustment amount, For the adjusted x i The weights; Based on the feature vectors corresponding to each BI-RADS feature and their adjusted weights, the image information feature vectors used to determine the BI-RADS classification of the breast lesion are determined. ; The image information feature vector used to determine the BI-RADS classification of the breast lesion is used to... Input a pre-trained BI-RADS grading model to obtain the BI-RADS grading of the breast lesion. The BI-RADS grading model is trained based on the feature vectors of samples labeled with BI-RADS grading.

4. The ultrasound imaging device as described in claim 3, characterized in that, The processor is also used for: Obtain the attribute feature vector based on the second feature value set. ; where y i Let r be the value of the i-th BI-RADS feature in the second feature value set. i For y i The initial weights; If the i-th BI-RADS feature has the same value in both the first feature set and the second feature set, then... Adjust y i The weights; if the i-th BI-RADS feature has different values ​​in the first feature set and the second feature set, then by Adjust y i The weights; where, After adjustment y i The weights; Based on the values ​​of each BI-RADS feature in the second feature set and their corresponding adjusted weights, an attribute information feature vector for determining the BI-RADS grading of the breast lesion is determined. ; The image information feature vector used to determine the BI-RADS classification of the breast lesion is used to... and the attribute information feature vector used to determine the BI-RADS classification of the breast lesion. To merge; The fused feature vector is input into the pre-trained BI-RADS grading model to obtain the BI-RADS grading of the breast lesion.

5. The ultrasound imaging device according to any one of claims 1-4, characterized in that, Before the processor detects the user's modification or confirmation operation on the first feature value set, it is further configured to: The ultrasound image and the first set of feature values ​​are displayed in a comparative manner on the display interface.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the following ultrasound imaging method: Acquire multiple ultrasound images of the breast region of the subject; The intelligent analysis model for breast lesions is based on a pre-trained model to analyze any one of the multiple ultrasound images to obtain the set of values ​​for the BI-RADS feature set corresponding to the arbitrary ultrasound image. According to a preset strategy, the first feature value set corresponding to the BI-RADS feature set value of the breast lesion in the subject's breast region is obtained from the set of values ​​for the BI-RADS feature set corresponding to the multiple ultrasound images. The intelligent analysis model for breast lesions is trained based on sample ultrasound images labeled with BI-RADS feature set values. Detect user actions that modify or confirm the first feature value set to obtain the second feature value set; Determining the BI-RADS classification of the breast lesion based on the first feature set, the second feature set, and the ultrasound image includes: When the value of any BI-RADS feature in the value set of the BI-RADS feature set corresponding to any ultrasound image in the multi-frame ultrasound images is the same as the value of any BI-RADS feature in the second feature set, the weight of the ultrasound image in determining the value of any BI-RADS feature in the BI-RADS classification is increased; when the value of any BI-RADS feature in the value set of the BI-RADS feature set corresponding to any ultrasound image is different from the value of any BI-RADS feature in the second feature set, the weight of the ultrasound image in determining the value of any BI-RADS feature in the BI-RADS classification is decreased.

Images