Breast ultrasonic image processing method and device, model training method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a breast ultrasonic image processing method and device, a model training method and device, equipment and a storage medium. The method comprises: acquiring a breast ultrasound image; identifying lesion areas of different lesion categories in the breast ultrasound image by using at least one breast ultrasound image processing model; frame-selecting the lesion area in the breast ultrasound image and marking a lesion category corresponding to the lesion area to generate a target image; and outputting the target image. Wherein the mammary gland ultrasound image processing model is obtained by training a plurality of sample mammary gland ultrasound images and corresponding training labels; the training label comprises a lesion area selected in the sample breast ultrasound image frame and a lesion category corresponding to the lesion area. According to the technical scheme provided by the embodiment of the invention, the accuracy and comprehensiveness of identifying the lesion area and the lesion category can be improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of image processing technologies, and in particular, to a method for processing breast ultrasound images, a model training method, a device, a device, and a storage medium. Background Art

[0002] A breast ultrasound image is an image formed by scanning breast tissue using ultrasonic waves. By analyzing a breast ultrasound image, it is possible to determine whether there are benign or malignant lesions.

[0003] Currently, it is mostly necessary to manually find the lesion areas in breast ultrasound images and identify whether the lesion areas are benign or malignant lesions. Affected by the individual experience of the evaluators, the identification of the lesion areas and the lesion categories may be inaccurate. Summary of the Invention

[0004] Embodiments of the present application provide a method for processing breast ultrasound images, a model training method, a device, a device, and a storage medium, which are used to solve the problem of inaccurate identification of lesion areas and lesion categories in the prior art.

[0005] In a first aspect, a method for processing breast ultrasound images provided in an embodiment of the present application includes:

[0006] Obtain a breast ultrasound image;

[0007] Use at least one breast ultrasound image processing model to identify lesion areas of different lesion categories in the breast ultrasound image;

[0008] Frame the lesion areas in the breast ultrasound image and label the lesion categories corresponding to the lesion areas to generate a target image;

[0009] Output the target image;

[0010] The breast ultrasound image processing model is obtained by training using a plurality of sample breast ultrasound images and their corresponding training labels; the training labels include the lesion areas framed in the sample breast ultrasound images and the lesion categories corresponding to the lesion areas.

[0011] In a second aspect, a method for training a breast ultrasound image processing model provided in an embodiment of the present application includes:

[0012] Obtain a plurality of sample breast ultrasound images;

[0013] Use the plurality of sample breast ultrasound images and their corresponding training labels to train a breast ultrasound image processing model; the training labels include the lesion areas framed in the sample breast ultrasound images and the lesion categories corresponding to the lesion areas;

[0014] Among them, the breast ultrasound image processing model is used to identify the lesion areas of different lesion categories in the breast ultrasound image, and frame the lesion areas in the breast ultrasound image and label the corresponding lesion categories of the lesion areas to generate a target image.

[0015] In a third aspect, an embodiment of the present application provides a breast ultrasound image processing device, the device includes:

[0016] A first acquisition module, configured to acquire a breast ultrasound image;

[0017] An identification module, configured to use at least one breast ultrasound image processing model to identify the lesion areas of different lesion categories in the breast ultrasound image;

[0018] A generation module, configured to frame the lesion areas in the breast ultrasound image and label the corresponding lesion categories of the lesion areas to generate a target image;

[0019] An output module, configured to output the target image;

[0020] Among them, the breast ultrasound image processing model is obtained by training with a plurality of sample breast ultrasound images and their corresponding training labels; the training labels include the lesion areas framed in the sample breast ultrasound images and the corresponding lesion categories of the lesion areas.

[0021] In a fourth aspect, an embodiment of the present application provides a breast ultrasound image processing device, the device includes:

[0022] A second acquisition module, configured to acquire a plurality of sample breast ultrasound images;

[0023] A training module, configured to train a breast ultrasound image processing model with the plurality of sample breast ultrasound images and their corresponding training labels; the training labels include the lesion areas framed in the sample breast ultrasound images and the corresponding lesion categories of the lesion areas;

[0024] Among them, the breast ultrasound image processing model is used to identify the lesion areas of different lesion categories in the breast ultrasound image, and frame the lesion areas in the breast ultrasound image and label the corresponding lesion categories of the lesion areas to generate a target image.

[0025] In a fifth aspect, an embodiment of the present application provides an electronic device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the breast ultrasound image processing method described in the first aspect or the breast ultrasound image processing model training method described in the second aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which when executed by a computer, implements the breast ultrasound image processing method described in the first aspect or the breast ultrasound image processing model training method described in the second aspect.

[0027] In a seventh aspect, an embodiment of the present application provides a computer program product including a computer program / instructions, which when executed by a computer, implements the breast ultrasound image processing method described in the first aspect or the breast ultrasound image processing model training method described in the second aspect.

[0028] An embodiment of the present application obtains a breast ultrasound image; uses at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound image; boxes the lesion regions in the breast ultrasound image and labels the lesion categories corresponding to the lesion regions to generate a target image; outputs the target image; wherein the breast ultrasound image processing model is obtained by training using a plurality of sample breast ultrasound images and their corresponding training labels; the training labels include the lesion regions boxed in the sample breast ultrasound images and the lesion categories corresponding to the lesion regions. Since the breast ultrasound image processing model can learn the features of different lesion categories, using the breast ultrasound image processing model to box the lesion regions in the breast ultrasound image and label the lesion categories corresponding to the lesion regions to generate a target image can improve the accuracy and comprehensiveness of identifying lesion regions and lesion categories, and can be used to assist physician users in diagnosis.

[0029] These aspects or other aspects of the present application will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following briefly introduces the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0031] Figure 1 Shows a flowchart of an embodiment of a breast ultrasound image processing method provided by the present application;

[0032] Figure 2 Shows a flowchart of an embodiment of a breast ultrasound image processing model training method provided by the present application;

[0033] Figure 3 Shows a schematic structural diagram of an embodiment of a breast ultrasound image processing apparatus provided by the present application;

[0034] Figure 4 It shows a schematic structural diagram of another embodiment of the breast ultrasound image processing model training device provided by this application;

[0035] Figure 5 It shows a schematic structural diagram of an electronic device provided by this application. Specific embodiments

[0036] In order to enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application.

[0037] In some processes described in the specification, claims and the above-mentioned drawings of this application, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish each different operation, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., do not represent the sequence, and do not limit that "first" and "second" are different types.

[0038] As described in the background art, currently, it is mostly artificial to find the lesion area of the breast ultrasound image and identify whether the lesion area is a benign lesion or a malignant lesion. Affected by the individual experience of the evaluator, the situation of inaccurate identification of the lesion area and the lesion category will occur.

[0039] To solve the problem of inaccurate manual identification of the lesion area and the lesion category, the inventor has proposed the technical solution of this application after some research. In the embodiments of this application, a breast ultrasound image is obtained; at least one breast ultrasound image processing model is used to identify the lesion areas of different lesion categories in the breast ultrasound image; the lesion areas are framed in the breast ultrasound image and the lesion categories corresponding to the lesion areas are marked to generate a target image; the target image is output; wherein, the breast ultrasound image processing model is obtained by training with a plurality of sample breast ultrasound images and their corresponding training labels; the training labels include the lesion areas framed in the sample breast ultrasound images and the lesion categories corresponding to the lesion areas. Since the breast ultrasound image processing model can learn the characteristics of different lesion categories, using the breast ultrasound image processing model to frame the lesion areas in the breast ultrasound image and mark the lesion categories corresponding to the lesion areas to generate a target image can improve the accuracy and comprehensiveness of identifying the lesion areas and the lesion categories, and can be used to assist physician users in diagnosis.

[0040] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0041] Figure 1 It is a flowchart of an embodiment of a breast ultrasound image processing method provided by the present application. The breast ultrasound image processing method provided by the embodiments of the present application can be executed by an engine. The engine can be, for example, (ONNX Runtime, an open-source inference engine for cross-platform use). This engine can deploy a breast ultrasound image processing model and can run on different hardware platforms (such as CPUs, GPUs, FPGAs, etc.) to use the breast ultrasound image processing model, enabling automated processing, thereby reducing manual intervention by users and simplifying the operation process of identifying lesion areas and lesion categories.

[0042] The method may include the following steps:

[0043] 101: Obtain a breast ultrasound image.

[0044] A breast ultrasound image is an image formed by scanning breast tissue using ultrasonic waves and can be used to evaluate breast structure, detect breast abnormalities, guide percutaneous biopsy, and evaluate the nature of breast diseases.

[0045] 102: Use at least one breast ultrasound image processing model to identify lesion areas of different lesion categories in the breast ultrasound image.

[0046] Lesion categories may include benign lesions and malignant lesions. Benign lesions and malignant lesions have different manifestations on ultrasound images. Benign lesions appear as smooth edges, having a capsule, regular morphology, etc. on ultrasound images. Malignant lesions appear as less smooth edges, without a capsule, irregular morphology, etc. on ultrasound images.

[0047] Among them, the breast ultrasound image processing model can be a deep learning model.

[0048] The breast ultrasound image processing model can be trained using multiple sample breast ultrasound images and their corresponding training labels; the training labels can include the lesion areas outlined in the sample breast ultrasound images and the lesion categories corresponding to the lesion areas.

[0049] Among them, the training labels can be obtained by a physician outlining the lesion areas in the sample breast ultrasound images and annotating the lesion categories corresponding to the lesion areas in combination with the sample breast ultrasound image reports and pathological result reports.

[0050] Among them, the breast ultrasound image processing model can be a YOLO (You Only Look Once, a deep learning object detection algorithm) model.

[0051] 103: Select and mark the lesion area corresponding to the lesion category in the breast ultrasound image to generate a target image.

[0052] Among them, it can be to use at least one breast ultrasound image processing model to identify the lesion areas of different lesion categories in the breast ultrasound image, select and mark the lesion areas corresponding to the lesion categories in the breast ultrasound image to generate a target image.

[0053] 104: Output the target image.

[0054] Optionally, the engine for executing the breast ultrasound image processing method provided in this embodiment can be run by an electronic device, which can be a terminal device such as a PC, a laptop, a smart phone, etc., or a server. The server can be a physical server including an independent host, or can also be a virtual server, or can also be a server or server cluster in the cloud. When the breast ultrasound image processing method provided in this embodiment is executed by the server, outputting the target image can refer to the server sending the target image to the terminal device for the terminal device to display the target image. When the breast ultrasound image processing method provided in this embodiment is executed by the terminal device, outputting the target image can refer to the terminal device displaying the target image.

[0055] In the embodiment of the present application, the breast ultrasound image processing model can learn the features of different lesion categories, select and mark the lesion areas corresponding to the lesion categories in the breast ultrasound image to generate a target image, which can improve the accuracy and comprehensiveness of identifying lesion areas and lesion categories, and can be used to assist physician users in diagnosis.

[0056] In some embodiments, obtaining the breast ultrasound image may include: obtaining the breast ultrasound image; scaling the breast ultrasound image proportionally and performing color filling to reach the input image size required by the breast ultrasound image processing model.

[0057] For example, if the original size of the breast ultrasound image is 320×240 pixels and the input image size required by the breast ultrasound image processing model is 640×640 pixels, the original breast ultrasound image can be enlarged to 640×480 pixels, and color filling is performed around the enlarged image to reach 640×640 pixels.

[0058] In this way, the aspect ratio of the original breast ultrasound image can be maintained, avoiding distortion caused by directly scaling the image to the input image size required by the model, which would change the aspect ratio.

[0059] In some embodiments, using at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in a breast ultrasound image may include: based on the breast ultrasound image, using at least one breast ultrasound image processing model to predict a plurality of candidate regions and the confidence levels respectively corresponding to the plurality of candidate regions; determining the candidate regions with confidence levels higher than the confidence level threshold as the lesion regions of different lesion categories; and if the intersection over union of any two lesion regions is greater than the intersection over union threshold, removing the lesion region with a lower confidence level.

[0060] Among them, the breast ultrasound image processing model can extract features by layer-by-layer convolution and downsampling based on the breast ultrasound image, generating a plurality of feature maps with different resolutions.

[0061] The breast ultrasound image processing model can perform grid segmentation on the feature map to obtain a plurality of grid cells. Anchor points can be preset on the feature map, and an anchor point can refer to a predefined bounding box. Each grid cell will predict one or more candidate regions for each anchor point, specifically including predicting the position parameters of the candidate region (such as the offsets of the center coordinates, width, and height) and the confidence level. The confidence level can refer to the probability that a lesion exists in the candidate region.

[0062] Furthermore, the candidate regions with confidence levels higher than the confidence level threshold can be determined as the lesion regions. Using techniques such as Non-Maximum Suppression (NMS), all the lesion regions are sorted according to their confidence scores, starting from the lesion region with the highest confidence level. Calculate the intersection over union (IoU) between the lesion region with the highest confidence level and all other lesion regions. IoU is an index to measure the overlapping degree between two regions. The intersection over union of two lesion regions is the area of the overlapping part of the two regions / (the sum of the areas of the two regions minus the area of the overlapping part), and its value ranges from 0 to 1. IoU = 1 means that the two lesion regions completely overlap, while IoU = 0 means that the two lesion regions do not overlap at all. If the intersection over union of a certain lesion region with the current lesion region with the highest confidence level exceeds the preset intersection over union threshold (such as 0.5), it is considered that the lesion region is too similar to the lesion region with the highest confidence level and is redundant, so it is removed. Repeat the operation of selecting the lesion region with the highest confidence level from the unprocessed lesion regions and performing the same intersection over union comparison and removal operation on the remaining lesion regions until all the lesion regions are processed. Finally, the lesion regions with high confidence levels and low overlapping degrees with each other are left.

[0063] Among them, the above-mentioned breast ultrasound image processing model predicts multiple candidate regions and the confidence levels respectively corresponding to the multiple candidate regions according to a preset anchor template. Optionally, the breast ultrasound image processing model can directly predict multiple candidate regions and the confidence levels respectively corresponding to the multiple candidate regions without using an anchor point.

[0064] Among them, directly predicting multiple candidate regions and the confidence levels respectively corresponding to the multiple candidate regions without using an anchor point can be independent of a fixed anchor template, improving the flexibility and accuracy of recognition.

[0065] In some embodiments, as an optional method, using at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in a breast ultrasound image may include: using one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound image.

[0066] As another optional method, using at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in a breast ultrasound image may include: using multiple breast ultrasound image processing models to respectively identify lesion regions of different lesion categories in the breast ultrasound image to generate multiple recognition results.

[0067] Selecting a lesion region in the breast ultrasound image and annotating the lesion category corresponding to the lesion region to generate a target image may include: according to multiple recognition results, selecting a lesion region in the breast ultrasound image and annotating the lesion category corresponding to the lesion region to generate multiple target images.

[0068] Outputting the target image includes: outputting multiple target images.

[0069] Among them, the structures of multiple breast ultrasound image processing models can be the same or different, and the object detection algorithms used to train multiple breast ultrasound image processing models can be the same or different. For example, multiple breast ultrasound image processing models can be trained using different versions of YOLO. The ways in which multiple breast ultrasound image processing models identify lesion regions of different lesion categories in breast ultrasound images can be the same or different. For example, as described above, a breast ultrasound image processing model can identify lesion regions of different lesion categories in a breast ultrasound image by predicting multiple candidate regions and the confidence levels respectively corresponding to the multiple candidate regions. The breast ultrasound image processing model's prediction of multiple candidate regions and the confidence levels respectively corresponding to the multiple candidate regions can be predicted according to a preset anchor template or directly without an anchor. In the case where multiple breast ultrasound image processing models respectively identify lesion regions by predicting multiple candidate regions and the confidence levels respectively corresponding to the multiple candidate regions, it can be that some breast ultrasound image processing models respectively predict multiple candidate regions and the confidence levels respectively corresponding to the multiple candidate regions according to a preset anchor template, and some breast ultrasound image processing models directly predict multiple candidate regions and the confidence levels respectively corresponding to the multiple candidate regions without an anchor.

[0070] In this alternative approach, by outputting multiple target images generated by multiple breast ultrasound image processing models, physician users can comprehensively consider the recognition results of multiple breast ultrasound image processing models to assist in diagnosis, avoiding the limitations of a single breast ultrasound image processing model, thereby improving the recognition accuracy and reliability.

[0071] In some embodiments, identifying lesion regions of different lesion categories in a breast ultrasound image may include: identifying lesion regions of different lesion categories in a breast ultrasound image and determining the probability that the lesion region belongs to the lesion category.

[0072] Bounding a lesion region in a breast ultrasound image and annotating the lesion category corresponding to the lesion region to generate a target image may include: bounding a lesion region in a breast ultrasound image and annotating the lesion category corresponding to the lesion region and the probability of belonging to the lesion category to generate a target image.

[0073] In some embodiments, using at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in a breast ultrasound image may include: using multiple breast ultrasound image processing models to respectively identify lesion regions of different lesion categories in a breast ultrasound image to generate multiple recognition results;

[0074] Using multiple breast ultrasound image processing models to respectively identify the lesion regions of different lesion categories in breast ultrasound images to generate multiple recognition results may include: using multiple breast ultrasound image processing models to respectively identify the lesion regions of different lesion categories in breast ultrasound images, and determining the probability that the lesion regions belong to the lesion categories to generate multiple recognition results.

[0075] Selecting and framing the lesion regions in breast ultrasound images and annotating the corresponding lesion categories of the lesion regions to generate target images may include: according to multiple recognition results, selecting the lesion regions in breast ultrasound images to generate multiple candidate images; fusing the multiple candidate images, framing the overlapping lesion regions and annotating the corresponding lesion categories of the overlapping lesion regions to generate target images.

[0076] Fusing the multiple candidate images, framing the overlapping lesion regions and annotating the corresponding lesion categories of the overlapping lesion regions to generate target images may include: fusing the multiple candidate images, framing the overlapping lesion regions and annotating the corresponding lesion categories of the overlapping lesion regions and the probability of belonging to the lesion categories to generate target images.

[0077] Among them, each breast ultrasound image processing model can generate a candidate image. Optionally, it can be to comprehensively determine the lesion categories corresponding to the overlapping lesion regions in the target image and the probability of belonging to the lesion categories based on the lesion categories and the probabilities of belonging to the lesion categories respectively corresponding to the overlapping lesion regions in multiple candidate images. For example, three candidate images can be generated by using three breast ultrasound image processing models. A certain overlapping lesion region is 80% benign in the first candidate image, 86% benign in the second candidate image, and 83% benign in the third candidate image. For example, the average probability of this overlapping lesion region in the three candidate images can be calculated to determine the lesion category corresponding to the overlapping lesion region in the target image and the probability of belonging to the lesion category, such as (80% + 86% + 83%) / 3 = 83%. Then the lesion category corresponding to this overlapping lesion region in the target image is benign, and the benign probability is 83%.

[0078] In this embodiment, the recognition results of multiple breast ultrasound image processing models can be fused, and the overlapping lesion regions can be selected as the target lesion regions, which can reduce the possibility of misidentification by a single breast ultrasound image processing model and improve the accuracy and credibility of the target lesion regions.

[0079] Figure 2 It is a flowchart of an embodiment of a method for training a breast ultrasound image processing model provided by this application. The method may include the following steps:

[0080] 201: Obtain multiple sample breast ultrasound images.

[0081] 202: Train a breast ultrasound image processing model using multiple sample breast ultrasound images and their corresponding training labels.

[0082] Among them, the training labels may include the lesion regions outlined in the sample breast ultrasound images and the corresponding lesion categories of the lesion regions.

[0083] Among them, the breast ultrasound image processing model is used to identify the lesion regions of different lesion categories in the breast ultrasound image, and outline the lesion regions in the breast ultrasound image and label the corresponding lesion categories of the lesion regions to generate a target image.

[0084] Among them, the breast ultrasound image processing model can be a YOLO model.

[0085] Among them, during the training process, a stochastic gradient descent optimizer can be used to update the model parameters of the breast ultrasound image processing model. For example, the initial learning rate can be set to 0.01, and the learning rate can be gradually reduced to 0.001 through learning rate decay. In addition, regularization methods can be combined to adjust the weights to prevent overfitting.

[0086] In this embodiment, obtain multiple sample breast ultrasound images; train a breast ultrasound image processing model using multiple sample breast ultrasound images and their corresponding training labels; among them, the training labels include the lesion regions outlined in the sample breast ultrasound images and the corresponding lesion categories of the lesion regions, and the breast ultrasound image processing model is used to identify the lesion regions of different lesion categories in the breast ultrasound image, and outline the lesion regions in the breast ultrasound image and label the corresponding lesion categories of the lesion regions to generate a target image. Since the breast ultrasound image processing model can learn the features of different lesion categories, using the breast ultrasound image processing model to outline the lesion regions in the breast ultrasound image and label the corresponding lesion categories of the lesion regions to generate a target image can improve the accuracy and comprehensiveness of identifying lesion regions and lesion categories, and can be used to assist physician users in diagnosis.

[0087] In some embodiments, obtaining multiple sample breast ultrasound images may include: obtaining multiple sample breast ultrasound images generated by multiple types of ultrasound imaging devices. Among them, different types of ultrasound imaging devices may have differences in resolution, contrast, noise level, etc. in the generated images due to factors such as hardware differences and different setting parameters. By training using images from multiple types of ultrasound imaging devices, the adaptability of the breast ultrasound image processing model to these changes can be enhanced, which helps the model learn more extensive and diverse feature representations, thereby improving the robustness and generalization ability.

[0088] Figure 3 The following is a schematic structural diagram of an embodiment of a breast ultrasound image processing device provided in an embodiment of the present application. The device may include:

[0089] A first acquisition module 301, configured to acquire breast ultrasound images;

[0090] An identification module 302, configured to use at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound images;

[0091] A generation module 303, configured to frame the lesion regions in the breast ultrasound images and label the lesion categories corresponding to the lesion regions, so as to generate target images;

[0092] An output module 304, configured to output the target images.

[0093] Wherein, the breast ultrasound image processing model is obtained by training using a plurality of sample breast ultrasound images and their respectively corresponding training labels; the training labels include the lesion regions framed in the sample breast ultrasound images and the lesion categories corresponding to the lesion regions.

[0094] Wherein, the breast ultrasound image processing model can be a YOLO model.

[0095] In some embodiments, the identification module uses at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound images, which may include: using a plurality of breast ultrasound image processing models to respectively identify lesion regions of different lesion categories in the breast ultrasound images, so as to generate a plurality of identification results.

[0096] The generation module frames the lesion regions in the breast ultrasound images and labels the lesion categories corresponding to the lesion regions, so as to generate target images, which may include: according to the plurality of identification results, framing the lesion regions in the breast ultrasound images and labeling the lesion categories corresponding to the lesion regions, so as to generate a plurality of target images.

[0097] The output module outputs the target images, which may include: outputting a plurality of target images.

[0098] In some embodiments, the identification module uses at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound images, which may include: using a plurality of breast ultrasound image processing models to respectively identify lesion regions of different lesion categories in the breast ultrasound images, so as to generate a plurality of identification results.

[0099] The generation module frames the lesion regions in the breast ultrasound images and labels the lesion categories corresponding to the lesion regions, so as to generate target images, which may include: according to the plurality of identification results, selecting the lesion regions in the breast ultrasound images to generate a plurality of candidate images; fusing the plurality of candidate images, framing the overlapping lesion regions and labeling the lesion categories corresponding to the overlapping lesion regions, so as to generate target images.

[0100] In some embodiments, the generating module uses at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in a breast ultrasound image, which may include: based on the breast ultrasound image, using at least one breast ultrasound image processing model to predict a plurality of candidate regions and the corresponding confidence levels of the plurality of candidate regions; determining the candidate regions with confidence levels higher than the confidence level threshold as lesion regions of different lesion categories; and if the intersection over union of any two lesion regions is greater than the intersection over union threshold, removing the lesion region with a lower confidence level.

[0101] In some embodiments, the identifying module identifying lesion regions of different lesion categories in a breast ultrasound image may include: identifying lesion regions of different lesion categories in the breast ultrasound image and determining the probability that the lesion regions belong to the lesion categories.

[0102] The generating module bounding a lesion region in the breast ultrasound image and labeling the corresponding lesion category of the lesion region to generate a target image may include: bounding a lesion region in the breast ultrasound image and labeling the corresponding lesion category of the lesion region and the probability of belonging to the lesion category to generate a target image.

[0103] Figure 3 The described breast ultrasound image processing device may execute Figure 1 the breast ultrasound image processing method described in the illustrated embodiment, and its implementation principle and technical effects will not be elaborated. For the breast ultrasound image processing device in the above embodiments, the specific manners in which each module and unit perform operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0104] Figure 4 FIG. is a structural schematic diagram of an embodiment of a breast ultrasound image processing model training device provided by an embodiment of the present application. The device may include:

[0105] A second obtaining module 401, configured to obtain a plurality of sample breast ultrasound images;

[0106] A training module 402, configured to train a breast ultrasound image processing model by using the plurality of sample breast ultrasound images and the corresponding training labels respectively; the training labels include the lesion regions bounded in the sample breast ultrasound images and the corresponding lesion categories.

[0107] Wherein, the breast ultrasound image processing model is used to identify lesion regions of different lesion categories in a breast ultrasound image, and to bound a lesion region in the breast ultrasound image and label the corresponding lesion category of the lesion region to generate a target image.

[0108] Wherein, the breast ultrasound image processing model may be a YOLO model.

[0109] In some embodiments, the second acquisition module acquiring a plurality of sample breast ultrasound images may include: acquiring a plurality of sample breast ultrasound images generated by a variety of ultrasound imaging devices.

[0110] Figure 4 The breast ultrasound image processing model training device of the illustrated embodiment may execute Figure 2 The breast ultrasound image processing model training method described in the illustrated embodiment, and its implementation principle and technical effects will not be elaborated herein. For the breast ultrasound image processing model training device in the above embodiments, the specific manners in which each module and unit perform operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0111] In a possible design, Figure 3 the breast ultrasound image processing device of the illustrated embodiment or Figure 4 the breast ultrasound image processing model training device of the illustrated embodiment may be implemented as an electronic device. As Figure 5 shown, the electronic device may include a storage component 501 and a processing component 502;

[0112] The storage component 501 stores one or more computer instructions, where the one or more computer instructions are called by the processing component for execution to implement the breast ultrasound image processing method described in the illustrated embodiment as Figure 1 shown or the breast ultrasound image processing model training method described in the illustrated embodiment as Figure 2 shown.

[0113] Of course, the electronic device may necessarily further include other components, such as an input / output interface, a communication component, etc.

[0114] The input / output interface provides an interface between the processing component and a peripheral interface module, and the above peripheral interface module may be an output device, an input device, etc.

[0115] The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices, etc.

[0116] When the electronic device is a computing device, the computing device may be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device may refer to a cloud server, and the above processing component, storage component, etc. may be basic server resources leased or purchased from a cloud computing platform.

[0117] The processing components involved in the foregoing corresponding embodiments may include one or more processors to execute computer instructions to complete all or part of the steps in the above methods. Of course, the processing components may also be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for executing the above methods.

[0118] The storage component 501 is configured to store various types of data to support operations in the device. The storage component may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0119] Embodiments of the present application also provide a computer-readable storage medium storing a computer program, which when executed by a computer can implement the above Figure 1 breast ultrasound image processing method or Figure 2 breast ultrasound image processing model training method shown in the embodiments.

[0120] In addition, embodiments of the present application provide a computer program product. The computer program product includes a computer program or instructions. When the computer program or instructions are executed by a computer, the above Figure 1 breast ultrasound image processing method or Figure 2 breast ultrasound image processing model training method shown in the embodiments is implemented.

[0121] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0123] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A method for processing breast ultrasound images, characterized in that, Comprising: Obtaining a breast ultrasound image; Using at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound image; Selecting the lesion regions in the breast ultrasound image and annotating the lesion categories corresponding to the lesion regions to generate a target image; Outputting the target image; The breast ultrasound image processing model is obtained by training using a plurality of sample breast ultrasound images and their corresponding training labels; the training labels include the lesion regions selected in the sample breast ultrasound images and the lesion categories corresponding to the lesion regions.

2. The method according to claim 1, wherein The step of using at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound image includes: Using a plurality of breast ultrasound image processing models to respectively identify lesion regions of different lesion categories in the breast ultrasound image to generate a plurality of recognition results; The step of selecting the lesion regions in the breast ultrasound image and annotating the lesion categories corresponding to the lesion regions to generate a target image includes: According to the plurality of recognition results, selecting the lesion regions in the breast ultrasound image and annotating the lesion categories corresponding to the lesion regions to generate a plurality of target images; The step of outputting the target image includes: Outputting the plurality of target images.

3. The method according to claim 1, characterized in that, The step of using at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound image includes: Using a plurality of breast ultrasound image processing models to respectively identify lesion regions of different lesion categories in the breast ultrasound image to generate a plurality of recognition results; The step of selecting the lesion regions in the breast ultrasound image and annotating the lesion categories corresponding to the lesion regions to generate a target image includes: According to the plurality of recognition results, selecting the lesion regions in the breast ultrasound image to generate a plurality of candidate images; Fusing the plurality of candidate images, selecting overlapping lesion regions and annotating the lesion categories corresponding to the overlapping lesion regions to generate a target image.

4. The method according to claim 1, wherein The step of using at least one breast ultrasound image processing model to identify lesion regions of different lesion categories in the breast ultrasound image includes: Based on the breast ultrasound image, using at least one breast ultrasound image processing model to predict a plurality of candidate regions and the confidence levels respectively corresponding to the plurality of candidate regions; Determining the candidate regions with confidence levels higher than a confidence level threshold as lesion regions of different lesion categories; If the intersection over union of any two lesion regions is greater than an intersection over union threshold, removing the lesion region with a lower confidence level.

5. The method according to claim 1, wherein The step of identifying lesion regions of different lesion categories in the breast ultrasound image includes: Identifying lesion regions of different lesion categories in the breast ultrasound image and determining the probability that the lesion regions belong to the lesion categories; The step of selecting the lesion regions in the breast ultrasound image and annotating the lesion categories corresponding to the lesion regions to generate a target image includes: Selecting the lesion regions in the breast ultrasound image and annotating the lesion categories corresponding to the lesion regions and the probability of belonging to the lesion categories to generate a target image.

6. A method for training a breast ultrasound image processing model, characterized in that, Comprising: Obtaining a plurality of sample breast ultrasound images; Training a breast ultrasound image processing model using the multiple sample breast ultrasound images and their respective corresponding training labels; the training labels include the lesion regions outlined in the sample breast ultrasound images and the lesion categories corresponding to the lesion regions. Wherein, the breast ultrasound image processing model is used to identify the lesion regions of different lesion categories in a breast ultrasound image, and outline the lesion regions in the breast ultrasound image and label the lesion categories corresponding to the lesion regions to generate a target image.

7. The method according to claim 6, characterized in that, The obtaining of the multiple sample breast ultrasound images includes: Obtaining multiple sample breast ultrasound images generated by various ultrasound imaging devices.

8. An apparatus for processing breast ultrasound images, characterized in that, Including: A first obtaining module for obtaining breast ultrasound images. An identification module for identifying the lesion regions of different lesion categories in the breast ultrasound image by using at least one breast ultrasound image processing model. A generation module for outlining the lesion regions in the breast ultrasound image and labeling the lesion categories corresponding to the lesion regions to generate a target image. An output module for outputting the target image. The breast ultrasound image processing model is obtained by training using multiple sample breast ultrasound images and their respective corresponding training labels; the training labels include the lesion regions outlined in the sample breast ultrasound images and the lesion categories corresponding to the lesion regions.

9. An electronic device, characterized in that, Including: A memory, a processor, and a communication interface; wherein, executable code is stored on the memory, and when the executable code is executed by the processor, the processor executes the breast ultrasound image processing method according to any one of claims 1 to 5 or the breast ultrasound image processing model training method according to any one of claims 6 to 7.

10. A non-transitory machine-readable storage medium, characterized in that, Executable code is stored on the non-transitory machine-readable storage medium, and when the executable code is executed by the processor of an electronic device, the processor executes the breast ultrasound image processing method according to any one of claims 1 to 5 or the breast ultrasound image processing model training method according to any one of claims 6 to 7.

Images