A breast ultrasound image segmentation model construction method, segmentation method and device

Abstract

The embodiments of this specification provide a method for constructing a breast ultrasound image segmentation model, a segmentation method and a device, wherein the model includes a Vit encoder, a prompt encoder and a mask decoder; including inputting a breast ultrasound image into the Vit encoder to obtain image features; inputting the prompt code into the prompt encoder to obtain prompt features; using the LoRA module to fine-tune the multi-layer perceptron and transposed convolution layer of the mask decoder, and fine-tune the converter block and feature-image attention block of the mask decoder to output segmentation results according to image features and prompt features; iterating the above steps to construct a breast ultrasound image segmentation model. This model significantly enhances the accuracy of the segmentation model in identifying lesion areas in breast ultrasound images by fine-tuning the mask decoder; and after fine-tuning, the segmentation boundary can be adaptively adjusted through the learning of the LoRA module to avoid misjudgment caused by complex background or blurred boundaries, thereby improving the robustness of the segmentation result.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of medical image recognition technology, and in particular to a method for constructing a breast ultrasound image segmentation model, a segmentation method, and a device. Background Art

[0002] In modern medical diagnosis and treatment, medical image segmentation, as an important auxiliary diagnosis technology, can effectively support clinicians in identifying and locating lesion areas, and provide a key basis for formulating accurate treatment plans. Breast cancer is the most common malignant tumor in women worldwide. Early screening and accurate segmentation are of vital importance for the effective control and treatment of breast cancer. Breast ultrasound images have become one of the main imaging methods for breast cancer screening due to their non-invasive and safe characteristics. However, due to the low contrast, blurred boundaries, complex shapes and a large amount of noise of breast ultrasound images, traditional segmentation methods face great challenges in terms of accuracy.

[0003] At present, deep learning-based segmentation methods, such as U-Net and its variants, have been widely used in medical image segmentation. However, traditional models such as U-Net often rely on a large amount of manually annotated high-quality data to have good segmentation performance, and it is difficult to fully capture the complex lesion characteristics of breast ultrasound images. In addition, such methods have poor generalization and often need to be retrained for different lesion types or data sets, resulting in high deployment and update costs in practical applications. For example, the recently emerged large visual model (SAM) has shown strong migration capabilities in general segmentation tasks, but this model is usually mainly used for segmentation tasks of natural images or multimodal medical images. If it is directly applied to breast ultrasound image segmentation, it will be difficult to deal with its unique low contrast and unclear boundaries.

[0004] To solve the above problems, the embodiments of this specification aim to provide a breast ultrasound image segmentation model construction method, a segmentation method and a device. Summary of the invention

[0005] In view of the above-mentioned problems in the prior art, the purpose of the embodiments of this specification is to provide a method for constructing a breast ultrasound image segmentation model, a segmentation method and a device to solve the problem in the prior art that breast ultrasound images with low contrast and unclear boundaries are difficult to accurately segment.

[0006] In order to solve the above technical problems, the specific technical solutions of the embodiments of this specification are as follows:

[0007] In a first aspect, an embodiment of this specification provides a method for constructing a breast ultrasound image segmentation model, wherein the breast ultrasound image segmentation model includes a Vit encoder, a hint encoder, and a mask decoder;

[0008] Inputting the breast ultrasound image into the Vit encoder to obtain image features;

[0009] Inputting the prompt code corresponding to the breast ultrasound image into the prompt encoder to obtain a prompt feature;

[0010] Using the LoRA module, fine-tuning the weight parameters of the multilayer perceptron and the transposed convolutional layer of the mask decoder, and fine-tuning the projection layer of the converter block and the feature-image attention block of the mask decoder to output a segmentation result of the breast ultrasound image according to the image features and the prompt features;

[0011] The above steps are iterated to construct a breast ultrasound image segmentation model.

[0012] Specifically, the weight parameters of the multilayer perceptron and the transposed convolutional layer of the mask decoder are fine-tuned using the LoRA module, further comprising:

[0013] Obtaining an overall feature according to the image feature output by the Vit encoder and the prompt feature output by the prompt encoder;

[0014] Inputting the overall feature into a first LoRA block to obtain a first fine-tuning output;

[0015] inputting the overall characteristic and the first fine-tuning output into the converter block to obtain a conversion output;

[0016] Inputting the conversion output into the transposed convolution layer and the second LoRA block respectively, to obtain the transposed convolution output and the second fine-tuning output respectively;

[0017] Inputting the second fine-tuning output and the conversion output into the feature-image attention block to obtain an attention output;

[0018] The attention output processed by the multilayer perceptron and the transposed convolution output are dot-multiplied to obtain the segmentation result.

[0019] Furthermore, the projection layer of the converter block and the feature-image attention block of the mask decoder are fine-tuned using the LoRA module, including:

[0020] Freezing the weights of the projection layer;

[0021] The first LoRA block and the second LoRA block act on the projection layer of the query and value of the converter block and the feature-image attention block to obtain a weight update amount;

[0022] The weight of the projection layer is fine-tuned according to the weight update amount.

[0023] Specifically, the above steps are iterated to construct a breast ultrasound image segmentation model, including:

[0024] Acquire a training data set, wherein the training data set includes breast ultrasound images and labels corresponding thereto, wherein the breast ultrasound images include benign tumor ultrasound images and malignant tumor ultrasound images;

[0025] Inputting the training data set into the Vit encoder of the initial breast ultrasound image segmentation model and inputting the prompt code corresponding to the training data into the prompt encoder of the initial breast ultrasound image segmentation model to obtain the predicted segmentation result output by the mask decoder of the initial breast ultrasound image segmentation model;

[0026] Calculate a loss function based on the predicted segmentation result and the label;

[0027] The initial breast ultrasound image segmentation model is iteratively trained using the loss function until a preset number of iterative training times is reached, thereby obtaining a trained ultrasound image segmentation model.

[0028] Furthermore, the initial breast ultrasound image segmentation model is iteratively trained using the loss function, including:

[0029] The parameters of the Vit encoder and the hint encoder are fixed, and the parameters of the mask decoder are updated.

[0030] Preferably, before inputting the training data set into the Vit encoder of the initial breast ultrasound image segmentation model and inputting the prompt code corresponding to the training data into the prompt encoder of the initial breast ultrasound image segmentation model, the method further comprises:

[0031] Preprocess and augment the data in the acquired training data set;

[0032] The preprocessing includes at least one or more of normalization and resizing of the breast ultrasound image, and the augmentation processing includes at least one or more of random cropping, brightness adjustment and contrast transformation of the breast ultrasound image and the corresponding label.

[0033] Specifically, the loss function is:

[0034] L = (1-α) L Dice +αL BCE ;

[0035] Among them, L is the loss function, α is the weight coefficient, L Dice is the Dice loss function, L BCE is the binary cross entropy loss;

[0036] ;

[0037] ;

[0038] Among them, P i is the predicted value of the i-th pixel in the breast ultrasound image, G i is the true value of the i-th pixel in the breast ultrasound image.

[0039] In a second aspect, an embodiment of the present specification provides a method for segmenting a breast ultrasound image, the method comprising:

[0040] Acquire a breast ultrasound image to be segmented and a prompt code corresponding to the breast ultrasound image to be segmented;

[0041] Inputting the breast ultrasound image to be segmented and the prompt code into the breast ultrasound image segmentation model constructed by the breast ultrasound image segmentation model construction method described in the above technical solution;

[0042] A segmentation result of the breast ultrasound image output by the breast ultrasound image segmentation model is obtained.

[0043] In a third aspect, an embodiment of the present specification provides a device for constructing a breast ultrasound image segmentation model, wherein the breast ultrasound image segmentation model includes a Vit encoder, a prompt encoder, and a mask decoder, and the device includes:

[0044] A first input module, used for inputting a breast ultrasound image into the Vit encoder to obtain image features;

[0045] A second input module, used for inputting the prompt code corresponding to the breast ultrasound image into the prompt encoder to obtain a prompt feature;

[0046] A fine-tuning and output module, used to fine-tune the weight parameters of the multi-layer perceptron and the transposed convolution layer of the mask decoder using the LoRA module, and fine-tune the projection layer of the converter block and the feature-image attention block of the mask decoder to output the segmentation result of the breast ultrasound image according to the image features and the prompt features;

[0047] The construction module is used to iterate the above steps to construct a breast ultrasound image segmentation model.

[0048] In a fourth aspect, an embodiment of the present specification provides a breast ultrasound image segmentation device, comprising:

[0049] An acquisition module, used for acquiring a breast ultrasound image to be segmented and a prompt code corresponding to the breast ultrasound image to be segmented;

[0050] An input module, used for inputting the breast ultrasound image to be segmented and the prompt code into the breast ultrasound image segmentation model constructed by the breast ultrasound image segmentation model construction method as described in the above technical solution;

[0051] An acquisition module is used to obtain the segmentation result of the breast ultrasound image output by the breast ultrasound image segmentation model.

[0052] In a fifth aspect, an embodiment of the present specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method provided by the above technical solution when executing the computer program.

[0053] In a sixth aspect, an embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method provided by the above technical solution is implemented.

[0054] In a seventh aspect, an embodiment of the present specification provides a computer program product, comprising at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the method provided by the above technical solution.

[0055] By adopting the above technical scheme, a method for constructing a breast ultrasound image segmentation model, a segmentation method and a device provided in the embodiments of this specification can significantly enhance the accuracy of the constructed breast ultrasound image segmentation model in identifying lesion areas in breast ultrasound images with image features such as low contrast, fuzzy boundaries and variable shapes by fine-tuning the mask decoder. And the fine-tuned mask decoder can adaptively adjust the segmentation boundaries through the learning of the LoRA low-rank matrix, avoid misjudgment caused by complex background or blurred boundaries, and improve the robustness of the segmentation results. In addition, the lightweight update method based on the LoRA module can ensure the efficiency and low cost requirements of the ultrasound image segmentation model in actual deployment, and is suitable for the application of large-scale medical image segmentation tasks; it is conducive to solving the delay problem caused by insufficient computing resources during the fine-tuning process of the model, improving the efficiency of model training, and adapting to the requirements of the medical field for efficient and accurate segmentation models.

[0056] In order to make the above and other purposes, features and advantages of the embodiments of this specification more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 A schematic diagram of the steps of a method for constructing a breast ultrasound image segmentation model provided in an embodiment of this specification;

[0059] Figure 2 A schematic diagram of the structure of a breast ultrasound image segmentation model constructed in the embodiments of this specification;

[0060] Figure 3 A schematic diagram of the steps of fine-tuning the weight parameters of the multi-layer perceptron and the transposed convolutional layer of the mask decoder using the LoRA module in an embodiment of this specification is shown;

[0061] Figure 4 A schematic diagram showing the principle of fine-tuning the projection layer of the converter block and feature-image attention block of the mask decoder using the LoRA module in an embodiment of this specification;

[0062] Figure 5 A schematic diagram of the steps of iterative training to construct a breast ultrasound image segmentation model in an embodiment of this specification is shown;

[0063] Figure 6 A comparison diagram of segmentation masks obtained by processing breast ultrasound images using the breast ultrasound image segmentation model provided in the embodiment of this specification and other multiple models is shown;

[0064] Figure 7 A schematic diagram showing the steps of a breast ultrasound image segmentation method provided in an embodiment of this specification is shown;

[0065] Figure 8 A schematic diagram of the structure of a breast ultrasound image segmentation model building device provided in an embodiment of this specification is shown;

[0066] Fig. 9 A schematic diagram of the structure of a breast ultrasound image segmentation device provided in an embodiment of this specification is shown;

[0067] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of this specification is shown.

[0068] Description of the accompanying symbols:

[0069] 81. A first input module;

[0070] 82. A second input module;

[0071] 83. Fine-tuning and output module;

[0072] 84. Building blocks;

[0073] 91. Get module;

[0074] 92. Input module;

[0075] 93. Output module;

[0076] 1002. Computer equipment;

[0077] 1004. Processor;

[0078] 1006. Memory;

[0079] 1008. Driving mechanism;

[0080] 1010, input / output module;

[0081] 1012. Input device;

[0082] 1014. Output device;

[0083] 1016. Presentation equipment;

[0084] 1018. Graphical user interface;

[0085] 1020. Network interface;

[0086] 1022. Communication link;

[0087] 1024. Communication bus. DETAILED DESCRIPTION

[0088] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0089] It should be noted that the terms "first", "second", etc. in this specification and claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this specification described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0090] In order to solve the above problems, the embodiments of this specification provide a breast ultrasound image segmentation model construction method, segmentation method and device, which can solve the problem in the prior art that it is difficult to accurately segment the image due to the low contrast and unclear boundaries of the breast ultrasound image. Figure 1 This is a schematic diagram of the steps of a method for building a breast ultrasound image segmentation model provided in an embodiment of this specification. This specification provides method operation steps as described in the embodiment or flowchart, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual system or device product is executed, it can be executed in the order of the method shown in the embodiment or the accompanying drawings or in parallel.

[0091] Figure 2 FIG. 4 is a schematic diagram of the structure of the constructed breast ultrasound image segmentation model, wherein the breast ultrasound image segmentation model includes a Vit encoder, a hint encoder and a mask decoder. Figure 1 As shown, the construction method may include:

[0092] S101: Inputting a breast ultrasound image into the Vit encoder to obtain image features.

[0093] S102: Inputting the prompt code corresponding to the breast ultrasound image into the prompt encoder to obtain a prompt feature.

[0094] S103: Using the LoRA module, fine-tune the weight parameters of the multi-layer perceptron and the transposed convolutional layer of the mask decoder, and fine-tune the projection layer of the converter block and the feature-image attention block of the mask decoder to output the segmentation result of the breast ultrasound image according to the image features and the prompt features.

[0095] That is Figure 2As shown, in the embodiment of this specification, the mask decoder includes a transformer block (i.e., a Transformer block), a token-to-image attention block (T2I Attention for short), a multi-layer perceptron (MLP), and a transposed convolution layer (i.e., Transposed Convolutions);

[0096] Using the LoRA module to fine-tune the parameters of the multilayer perceptron and the transposed convolutional layer can optimize the model's segmentation effect on lesion areas with low contrast and fuzzy boundaries; and using the LoRA module to fine-tune the projection layer of the converter block and the feature-image attention block (i.e., adapter fine-tuning) can enhance the model's ability to capture the complex characteristics of breast ultrasound images.

[0097] S104: Iterate the above steps to construct a breast ultrasound image segmentation model.

[0098] That is Figure 2 As shown, in the embodiment of this specification, the constructed breast ultrasound image segmentation model is based on the SAM (Segment Anything Model) model, including a Vit encoder, a hint encoder and a mask decoder;

[0099] Vit encoder, denoted as E i , which is used to segment the received breast ultrasound image into several small patches, linearize it and add position encoding, convert the image into a series of vectors, and then extract high-dimensional image features through a multi-layer Transformer encoding network. This block processing method enables the Vit encoder to effectively capture the global information of the breast ultrasound image while retaining the local detail features, thereby generating a feature map that can characterize the content of the breast ultrasound image, providing key support for subsequent segmentation.

[0100] Prompt Encoder, denoted by E p , which is used to encode the prompt encoding input by the user and output the prompt embedding, i.e., the prompt feature. The prompt encoding can include points, boxes, or rough masks to guide the model to focus on specific areas. The prompt encoder converts the prompt encoding into a specific vector to ensure that the SAM model can focus on the target area during segmentation, thereby improving the segmentation effect.

[0101] Mask Decoder, denoted as D mThe mask decoder is a decoder module in the SAM model and is based on a multi-layer Transformer decoding structure. It is used to deeply fuse image features with cue features to generate a segmentation mask of the target area, i.e., the segmentation result. The multi-layer attention mechanism in the mask decoder enables it to flexibly focus on specific areas of the image, which is conducive to further improving the accuracy of segmentation.

[0102] The output of the Vit encoder and the output of the Hint encoder are both connected to the input of the Mask Decoder to generate a segmentation mask and output:

[0103] Mask=D m (F i , F p );

[0104] Among them, Mask is the segmentation mask finally output by the ultrasound image segmentation model, D m is the mask decoder; F i is the output of the Vit encoder, F i =E i (I), E i is the Vit encoder, I is the input breast ultrasound image; F p is the output of the prompt encoder, F p =E p (P), E p is the prompt encoder and P is the prompt code.

[0105] in, is the image feature, is the cue feature, h and w are the feature resolutions, c is the feature dimension, and k represents the number of cues.

[0106] The embodiment of this specification provides a method for constructing a breast ultrasound image segmentation model. By adding a LoRA module to fine-tune the parameters of the mask decoder and the adapter, the constructed breast ultrasound image segmentation model is improved in terms of the accuracy of the segmentation results of the lesion area in the breast ultrasound image with image features such as low contrast, fuzzy boundaries and variable shapes; and the fine-tuned mask decoder can adaptively adjust the segmentation boundary through the learning of the LoRA low-rank matrix, avoiding misjudgment caused by complex background or blurred boundaries, and improving the robustness of the segmentation results. In addition, the lightweight update method based on the LoRA module can ensure the efficiency and low cost requirements of the breast ultrasound image segmentation model in actual deployment, improve the parameterization flexibility of the model, avoid the overfitting problem caused by the large model, and is suitable for the application of large-scale medical image segmentation tasks; it is conducive to solving the delay problem caused by insufficient computing resources during the fine-tuning process of the model, improving the efficiency of model training, and adapting to the requirements of the medical field for efficient and accurate segmentation models.

[0107] Specifically, if Figure 3 As shown, in S103, using the LoRA module to fine-tune the weight parameters of the multilayer perceptron and the transposed convolution layer of the mask decoder further includes:

[0108] S301: Obtaining overall features according to the image features output by the Vit encoder and the prompt features output by the prompt encoder.

[0109] The overall feature is denoted by F in , then F in = (F i , F p ).

[0110] S302: Input the overall feature into a first LoRA block to obtain a first fine-tuning output.

[0111] The first trim output is F LoRA1 , then F LoRA1 =LoRA1(F in ), LoRA1(·) is the first LoRA block.

[0112] S303: Input the overall feature and the first fine-tuning output into the converter block to obtain a conversion output.

[0113] That is, the converter block receives F in and F LoRA1 , the conversion output is F Transformer =Transformer (F in , F LoRA1 ).

[0114] S304: Input the conversion output into the transposed convolution layer and the second LoRA block respectively to obtain a transposed convolution output and a second fine-tuning output respectively.

[0115] The output of the transposed convolution is F Conv , then F Conv =TransposedConv (F Transformer );

[0116] The second trim output is F LoRA2 , then F LoRA2 =LoRA2(F Transformer ).

[0117] S305: Input the second fine-tuning output and the conversion output into the feature-image attention block to obtain an attention output.

[0118] The attention output is F T21 , then F T21 =T2IAtt (F Transformer , F LoRA2 ).

[0119] S306: Perform a dot product on the attention output processed by the multilayer perceptron and the transposed convolution output to obtain the segmentation result.

[0120] Right now .

[0121] Among them, F MLP is the attention output after multi-layer perceptron processing, that is, F MLP =MLP (F T21 ), MLP (·) is a multi-layer perceptron, It is an element-wise dot product operation.

[0122] The breast ultrasound image segmentation method provided in the embodiment of this specification introduces the LoRA module to make subtle adjustments to the multi-layer perceptron head and transposed convolution in the mask decoder, thereby optimizing the ultrasound image segmentation model for the segmentation effect of the lesion area in the ultrasound image with low contrast and fuzzy boundaries. During the fine-tuning process, the LoRA low-rank matrix is ​​optimized while keeping the pre-trained matrix unchanged. It should be noted that the number of parameters of the LoRA module is much less than that of the original matrix, which is conducive to improving the efficiency of the fine-tuning process.

[0123] In the embodiment of this specification, both the converter block and the feature-image attention block include an attention layer, so that the principle of fine-tuning the converter block using the LoRA module and the principle of fine-tuning the feature-image attention block using the LoRA module are the same or similar. Figure 4 In the embodiment of this specification, in step S103, fine-tuning the projection layer of the converter block and the feature-image attention block of the mask decoder using the LoRA module may further include:

[0124] Freezing the weights of the projection layer;

[0125] The first LoRA block and the second LoRA block act on the projection layer of the query and value of the converter block and the feature-image attention block to obtain a weight update amount;

[0126] The weight of the projection layer is fine-tuned according to the weight update amount.

[0127] like Figure 4As shown in the figure, the LoRA module appends low-rank matrices A and B (i.e., the first LoRA block and the second LoRA block) to the projection matrix of the Transformer, and embeds them into the query, key, and value projection layers. It adopts a low-rank update method, which not only ensures the computational efficiency of fine-tuning, but also effectively enhances the adaptability of the model to specific tasks.

[0128] Specifically, for a specially encoded token (i.e. Input Token) , first freeze the Transformer layer to keep the weight matrix W fixed, then add a bypass to achieve low-rank approximation and set the parameters of the LoRA module to be updateable. The bypass includes two linear layers, namely the low-rank matrix and low-rank matrix , where r< <min{C in , C out}, get the updated weight matrix W', then:

[0129] T'=W'T;

[0130] W'=W+△W=W+BA;

[0131] Where T' is the output obtained by the projection layer operation, ; b represents the batch size, n represents the number of pixels in the image, C in and C out are the input channel dimension and the output channel dimension respectively; W' is the updated weight matrix, △W is the update amount of the weight matrix, and A and B are both low-rank matrices.

[0132] LoRA is used for the projection layer of query (Q), key (K), and value (V) to optimize the calculation of multi-head attention. By only updating the calculation of query and value, the image segmentation performance and parameter adjustment efficiency are further improved. The formula is as follows:

[0133] ;

[0134] Q=W q 'T=W q +B q A q ;

[0135] K=W k T;

[0136] V=W v 'T=W v +B v A v ;

[0137] Among them, Att(·) is the attention mechanism, softmax(·) is the normalized exponential function, C is a constant; W q ' is the weight matrix of the updated query, W q is the weight matrix of the initial query, B q is the low-rank matrix B acting on the query projection layer, A q is the projection of the low-rank matrix A on the query; W k is the initial key weight matrix; W v ' is the weight matrix of the updated value, W v is the weight matrix of the initial value, B v is the projection of the low-rank matrix B on the value, A v is the projection of the low-rank matrix A on the value.

[0138] The breast ultrasound image segmentation method provided in the embodiments of this specification introduces a LoRA module to fine-tune the mask decoder adapter, thereby enhancing the image segmentation model's ability to extract features from breast ultrasound images, especially its ability to respond to low-contrast edges and complex lesion areas.

[0139] Furthermore, if Figure 5 As shown, in the embodiment of this specification, step S104: iterating the above steps to construct a breast ultrasound image segmentation model further includes:

[0140] S501: Acquire a training data set, wherein the training data set includes breast ultrasound images and corresponding labels, wherein the breast ultrasound images include benign tumor ultrasound images and malignant tumor ultrasound images.

[0141] In some preferred embodiments, the training data set may be a BUSI data set and a Dataset B data set. The BUSI data set was first proposed by Al-Dhabyani et al. and contains 780 breast ultrasound images, including 487 benign tumor ultrasound images, 210 malignant tumor ultrasound images, and 133 lesion-free ultrasound images. These images were collected from two different models of LOGIQ E9 ultrasound equipment (standard E9 and E9 Agile systems), provided by Baheya Hospital, with an image resolution of 500×500 pixels. The Dataset B data set contains 163 images, including 109 benign tumor ultrasound images and 54 malignant tumor ultrasound images, with an average resolution of 760×570 pixels, and the images were collected by the Siemens ACUSON Sequoia C512 system. These data sets cover a variety of types of breast lesions, which can ensure the comprehensiveness of the training data, and the training data in the data sets have different contrast, resolution, and lesion characteristics, so as to ensure that the ultrasound image segmentation model can adapt to a variety of breast ultrasound images after training, thereby improving the generalization and adaptability of the ultrasound image segmentation model.

[0142] S502: Input the training data set into the Vit encoder of the initial breast ultrasound image segmentation model and input the prompt code corresponding to the training data into the prompt encoder of the initial breast ultrasound image segmentation model to obtain the predicted segmentation result output by the mask decoder of the initial breast ultrasound image segmentation model.

[0143] Specifically, each training data (i.e., each breast ultrasound image) in the training data set in the embodiment of this specification is a three-channel image, i.e., the size is 3×h×w (i.e., c is 3, h and w are feature resolutions); the hint code is a binary mask image of the same size as the corresponding training data, and identifies different semantic areas to be segmented, and the hint code can be obtained by manual processing.

[0144] S503: Calculate a loss function according to the predicted segmentation result and the label.

[0145] Preferably, in the embodiments of this specification, the loss function is:

[0146] L = (1-α) L Dice +αL BCE ;

[0147] Among them, L is the loss function, α is the weight coefficient, L Dice is the Dice loss function, L BCE is the binary cross entropy loss;

[0148] ;

[0149] ;

[0150] Among them, P i is the predicted value of the i-th pixel in the breast ultrasound image, G i is the true value of the i-th pixel in the breast ultrasound image.

[0151] Dice loss is used to measure the overlap between the predicted mask (i.e., the predicted segmentation result) and the true mask (i.e., the label corresponding to the breast ultrasound image) to optimize the regional consistency of the model. For the lesion area in the breast ultrasound image, Dice loss can help the model cover the target area more accurately and improve the coherence of the segmentation. Binary Cross-Entropy Loss (BCE) is used to optimize the classification accuracy of each pixel, which can effectively solve the problem of blurred lesion boundaries. By calculating the error between the predicted mask and the true mask at each pixel, BCE loss can refine the segmentation edge, so that the model can still maintain a high segmentation accuracy in complex morphology and low-contrast areas.

[0152] In the embodiments of this specification, by combining Dice loss and BCE loss to maximize the coverage of the lesion area and improve the accuracy of the boundary area, it is beneficial to ensure that the trained ultrasound image segmentation model has stable segmentation performance when processing breast ultrasound images.

[0153] S504: Iteratively train the initial breast ultrasound image segmentation model using the loss function until a preset number of iterative training times is reached to obtain a trained ultrasound image segmentation model.

[0154] During the training process, the predicted mask output by the model is supervised by combining these two loss functions, which helps to optimize the smoothness of the model when predicting the mask. In the iterative training process, the parameters of the Vit encoder and the hint encoder are fixed, and the parameters of the mask decoder are updated to reduce the computational overhead of training and improve the efficiency of fine-tuning.

[0155] Furthermore, before S502: inputting the training data set into the Vit encoder of the initial breast ultrasound image segmentation model and inputting the prompt code corresponding to the training data into the prompt encoder of the initial breast ultrasound image segmentation model, the method further includes:

[0156] Preprocessing and augmenting the data in the acquired training data set to enhance the generalization ability of the trained ultrasound image segmentation model;

[0157] The preprocessing includes at least one or more of normalization and resizing of the breast ultrasound image, and the augmentation processing includes at least one or more of random cropping, brightness adjustment and contrast transformation of the breast ultrasound image and the corresponding label.

[0158] The breast ultrasound image segmentation model obtained by constructing a breast ultrasound image segmentation model based on a fine-tuned visual large model provided in the embodiments of this specification is based on the SAM large model, and its ViT encoder is used to extract features from breast ultrasound images, and can obtain high-quality global image embedding. The ViT encoder divides the input breast ultrasound image into multiple small blocks and then converts them into a series of high-dimensional feature vectors. These features are further input into a multi-layer attention mechanism for processing, thereby capturing multi-scale information and complex boundary features in the image. Combined with the advantages of multi-layer feature fusion, the segmentation of breast ultrasound images is enhanced in both high-frequency and low-frequency feature extraction, especially the contour information of the lesion area and the expression of internal details are richer.

[0159] Furthermore, in the embodiments of this specification, the breast ultrasound image segmentation model also includes fine-tuning the mask decoder of the SAM large model using the introduced LoRA module, and updating the weight matrix of the mask decoder by adding a low-rank matrix, thereby retaining the feature expression of the large model while significantly reducing the computing requirements. And the LoRA fine-tuning not only reduces the consumption of computing resources, but also effectively improves the adaptability of the model, enabling it to generate accurate segmentation masks under the complex background of breast ultrasound images. In the application process, the LoRA module focuses on enhancing the recognition ability of complex shapes and fuzzy boundary areas in breast ultrasound images, so that the segmentation results have efficient training and reasoning performance while ensuring accuracy.

[0160] During fine-tuning, by freezing the weights of the ViT encoder and the hint encoder, only the mask decoder is fine-tuned, avoiding the high computational cost of fully fine-tuning all parameters of the model. This fine-tuning strategy enhances the model's specificity in processing breast ultrasound images, allowing the breast ultrasound image segmentation model to complete segmentation tasks more accurately and efficiently when applied in the medical field.

[0161] Table 1 is a data comparison table of the breast ultrasound image segmentation model constructed by the construction method provided in the embodiment of this specification and the basic SAM model after segmentation processing of the breast ultrasound image in the DataSet B data set; Table 2 is a data comparison table of the breast ultrasound image segmentation model constructed by the construction method provided in the embodiment of this specification and the basic SAM model after segmentation processing of the breast ultrasound image in the BUSI data set:

[0162] Table 1

[0163]

[0164] Table 2

[0165]

[0166] Among them, before fine-tuning represents the basic SAM model; after fine-tuning, it represents the breast ultrasound image segmentation model constructed by fine-tuning the mask decoder using the LoRA module in the embodiment of this specification. The point prompt means that the prompt encoder receives the prompt encoded as a point, and the box prompt means that the prompt encoder receives the prompt encoded as a box.

[0167] In the embodiments of this specification, the segmentation results are measured from five aspects: Dice Index, Intersection-Over-Union (IoU), Sensitivity, Precision and Specificity, where Dice Index represents the overlap between the predicted mask and the true mask, measured by the ratio of the intersection area between the predicted mask and the true mask to the total area of ​​the sample; Intersection-Over-Union (IoU), or Jaccard Index, evaluates the segmentation accuracy by the similarity of the overlapping areas; Sensitivity, i.e., True Positive Rate or Recall, reflects the ability of the model to identify the target area; Precision represents the proportion of correctly predicted foreground pixels (i.e., tumor pixels) output by the model to all predicted foreground pixels; and Specificity represents the proportion of correctly predicted background pixels (i.e., non-tumor pixels) to all actual background pixels.

[0168] Dice = 2TP / (2TP+FN+FP);

[0169] IoU = TP / (TP+FP+FN);

[0170] Sensitivity = TP / (TP+FN);

[0171] Precision = TP / (TP+FP);

[0172] Specificity=TN / (TP+FP);

[0173] Among them, TP stands for True Positive, which means the number of pixels correctly predicted as foreground (i.e., tumor); FP stands for False Positive, which means the number of background pixels incorrectly predicted as foreground; TN stands for True Negative, which means the number of pixels correctly predicted as background (i.e., non-tumor area); FN stands for False Negative, which means the number of foreground pixels incorrectly predicted as background.

[0174] By comparing the data in Table 1 and Table 2, it can be seen that the breast ultrasound image segmentation model constructed by fine-tuning the mask decoder through the LoRA model has significantly improved the accuracy and precision of the segmentation results of breast ultrasound images compared with the existing SAM large model.

[0175] Furthermore, if Figure 6 Shown is a comparison diagram of segmentation masks obtained by processing breast ultrasound images using a breast ultrasound image segmentation model constructed using the construction method provided in the embodiments of this specification and other various models.

[0176] Among them, the first column is Input, which is the input breast ultrasound image;

[0177] The second column shows the segmentation results of each breast ultrasound image using the U-Net model. The U-Net model is a classic segmentation network based on CNN (Convolutional Neural Networks). Its structure adopts a symmetrical encoder-decoder architecture and realizes pixel-level segmentation through gradual downsampling and upsampling, which can effectively capture local and global features.

[0178] The third column shows the segmentation results of each breast ultrasound image using the U-Net++ model. The U-Net++ model introduces a redesigned skip connection strategy, which increases dense connections to enhance the transmission of multi-level features, so as to better capture and integrate detail information, further improve segmentation accuracy, and can be used to process complex lesion areas.

[0179] The fourth column shows the segmentation results of each breast ultrasound image using the H2Former model. The H2Former model combines the advantages of CNN, multi-scale channel attention mechanism and Transformer to construct an efficient hierarchical hybrid visual Transformer architecture. H2Former can perform well in processing medical images with complex structures and multi-scale features, and can be applied to medical image segmentation tasks that require high precision;

[0180] The fifth column shows the segmentation results of each breast ultrasound image using the Att-Unet model. The Att-Unet model adds an attention gate mechanism to the U-Net model, making the model more sensitive when processing subtle features. The attention gate can dynamically adjust the weight of each layer of features, so as to better focus on important areas during the segmentation process, enhancing the feature sensitivity and segmentation accuracy of the model;

[0181] The sixth column shows the segmentation results of each breast ultrasound image using the AAU-net model. The AAU-net model introduces a hybrid adaptive attention module into the U-Net structure, which improves the robustness of the model and its ability to segment complex lesions such as breast lesions. The adaptive attention module enables the model to automatically allocate attention areas at different feature levels, improving the recognition effect of complex tissue structures.

[0182] The seventh column shows the segmentation results of each breast ultrasound image using the SAM (Segment Anything Model) model. The SAM model is a general large-scale segmentation model that can generate segmentation masks through various prompts such as points, boxes, and text without retraining for a specific data set. Since input prompts are required, this experiment randomly generates three points in the real value as input prompts for the model.

[0183] The eighth column shows the segmentation results of each breast ultrasound image using the MedSAM* model. The MedSAM* model is fine-tuned on a large number of medical images. MedSAM* freezes the image encoder and the prompt encoder, and only adaptively adjusts the mask decoder to meet the specific needs of medical image segmentation. MedSAM* requires box prompts. Here we introduce box prompts based on MedSAM*.

[0184] The ninth column (Ours) is the segmentation result of each breast ultrasound image obtained by using the breast ultrasound image segmentation model constructed by the embodiments of this specification.

[0185] In the segmentation results of each breast ultrasound image in the second to ninth columns, the figures circled by lines are the tumor boundaries in the breast ultrasound images. Thus, by comparing the segmentation results of breast ultrasound images by various models, it can be seen that the breast ultrasound image segmentation model constructed in the embodiments of this specification can perform more accurate edge extraction and target positioning of the lesion area when segmenting each breast ultrasound image, significantly improving the segmentation quality.

[0186] Furthermore, the embodiments of this specification also provide a method for segmenting breast ultrasound images based on a fine-tuned visual macro model, such as Figure 7As shown, it is a schematic diagram of the steps of the method for segmenting a breast ultrasound image. It should be noted that this specification provides method operation steps as described in the embodiments or flow charts, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the steps among many orders, and does not represent the only order of execution. When the system or device product is executed in practice, it can be executed in the order of the methods shown in the embodiments or the drawings or in parallel.

[0187] Specific as Figure 7 As shown, the method may include:

[0188] S701: Acquire a breast ultrasound image to be segmented and a prompt code corresponding to the breast ultrasound image to be segmented.

[0189] S702: Inputting the breast ultrasound image to be segmented and the prompt code into the breast ultrasound image segmentation model constructed by the breast ultrasound image segmentation model construction method.

[0190] S703: Obtaining the segmentation result of the breast ultrasound image output by the breast ultrasound image segmentation model.

[0191] A breast ultrasound image segmentation method provided by an embodiment of the present specification can quickly and accurately obtain the segmentation result corresponding to the breast ultrasound image by inputting the breast ultrasound image to be segmented and the corresponding prompt code into the pre-built and trained breast ultrasound image segmentation model. Among them, the mask decoder of the breast ultrasound image segmentation model is fine-tuned by the LoRA model, which can significantly enhance the segmentation accuracy of the ultrasound image segmentation model for the lesion area in the breast ultrasound image with image features such as low contrast, fuzzy boundaries and variable shapes. And the fine-tuned mask decoder can adaptively adjust the segmentation boundary through the learning of the LoRA low-rank matrix, avoid misjudgment caused by complex background or blurred boundaries, and improve the robustness of the segmentation result. In addition, the lightweight update method based on the LoRA module can ensure the efficiency and low cost requirements of the ultrasound image segmentation model in actual deployment, improve the parameterization flexibility of the model, avoid the overfitting problem caused by the large model, and is suitable for the application of large-scale medical image segmentation tasks; it is conducive to solving the delay problem caused by insufficient computing resources during the fine-tuning process of the model, improve the model training efficiency, and adapt to the medical field. The requirements for efficient and accurate segmentation models.

[0192] Based on the above-mentioned method for constructing a breast ultrasound image segmentation model, the embodiment of this specification also provides a corresponding device for constructing a breast ultrasound image segmentation model; and based on the above-mentioned method for segmenting a breast ultrasound image, the embodiment of this specification also provides a corresponding device for segmenting a breast ultrasound image. The device may include a system (including a distributed system), software (application), module, component, server, client, etc. using the method described in the embodiment of this specification and a device combined with necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiment of this specification is as described in the following embodiments. Since the implementation scheme of the device to solve the problem is similar to the method, the implementation of the specific device in the embodiment of this specification can refer to the implementation of the aforementioned method, and the repetition will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and / or hardware of predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0193] like Figure 8 As shown, an embodiment of this specification provides a breast ultrasound image segmentation model construction device, wherein the breast ultrasound image segmentation model includes a Vit encoder, a prompt encoder and a mask decoder, and the breast ultrasound image segmentation model construction device includes:

[0194] A first input module 81, used for inputting a breast ultrasound image into the Vit encoder to obtain image features;

[0195] A second input module 82, for inputting the prompt code corresponding to the breast ultrasound image into the prompt encoder to obtain a prompt feature;

[0196] A fine-tuning and output module 83, configured to fine-tune the weight parameters of the multi-layer perceptron and the transposed convolutional layer of the mask decoder using the LoRA module, and fine-tune the projection layer of the converter block and the feature-image attention block of the mask decoder, so as to output the segmentation result of the breast ultrasound image according to the image feature and the prompt feature;

[0197] The construction module 84 is used to iterate the above steps to construct a breast ultrasound image segmentation model.

[0198] like Fig. 9 As shown, a breast ultrasound image segmentation device provided in an embodiment of this specification includes:

[0199] An acquisition module 91 is used to acquire a breast ultrasound image to be segmented and a prompt code corresponding to the breast ultrasound image to be segmented;

[0200] An input module 92, used for inputting the breast ultrasound image to be segmented and the prompt code into the breast ultrasound image segmentation model constructed by the breast ultrasound image segmentation model construction method;

[0201] The output module 93 is used to obtain the segmentation result of the breast ultrasound image output by the breast ultrasound image segmentation model.

[0202] The beneficial effects obtained by the device provided in the embodiments of this specification are consistent with the beneficial effects obtained by the above method and will not be repeated here.

[0203] like Fig.10 As shown, a computer device provided in an embodiment of this specification is provided. A breast ultrasound image segmentation model construction device and a breast ultrasound image segmentation device in this specification can both be computer devices in this embodiment, and perform the above method of this specification. The computer device 1002 may include one or more processors 1004, such as one or more central processing units (CPUs), and each processing unit may implement one or more hardware threads. The computer device 1002 may also include any memory 1006, which is used to store any kind of information such as code, settings, data, etc. Non-limitingly, for example, the memory 1006 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 1002. In one case, when the processor 1004 executes an associated instruction stored in any memory or a combination of memories, the computer device 1002 can perform any operation of the associated instruction. The computer device 1002 also includes one or more drive mechanisms 1008 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0204] The computer device 1002 may also include an input / output module 1010 (I / O) for receiving various inputs (via input device 1012) and for providing various outputs (via output device 1014). A specific output mechanism may include a presentation device 1016 and an associated graphical user interface (GUI) 1018. In other embodiments, the input / output module 1010 (I / O), input device 1012, and output device 1014 may not be included, and the computer device 1002 may be used as a computer device in a network. The computer device 1002 may also include one or more network interfaces 1020 for exchanging data with other devices via one or more communication links 1022. One or more communication buses 1024 couple the components described above together.

[0205] The communication link 1022 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 1022 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0206] Corresponding to Figure 1 , Figure 3 , Figure 5 and Figure 7 In addition to the method shown, an embodiment of the specification also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are executed.

[0207] The embodiment of the present specification also provides a computer-readable instruction, wherein when the processor executes the instruction, the program therein causes the processor to execute the following Figure 1 , Figure 3 , Figure 5 and Figure 7 The method shown.

[0208] The embodiments of the present specification also provide a computer program product, including at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the following Figure 1 , Figure 3 , Figure 5 and Figure 7 The method shown.

[0209] It should be understood that in the various embodiments of this specification, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this specification.

[0210] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this specification generally indicates that the associated objects before and after are in an "or" relationship.

[0211] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0212] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0213] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0214] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this specification.

[0215] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0216] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0217] Specific embodiments are used in this specification to illustrate the principles and implementation methods of this specification. The description of the above embodiments is only used to help understand the methods and core ideas of this specification. At the same time, for those skilled in the art, according to the ideas of this specification, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this specification.

Claims

1. A method for constructing a breast ultrasound image segmentation model, characterized in that: The breast ultrasound image segmentation model includes a Vit encoder, a prompt encoder and a mask decoder; Inputting the breast ultrasound image into the Vit encoder to obtain image features; Inputting the prompt code corresponding to the breast ultrasound image into the prompt encoder to obtain a prompt feature; The weight parameters of the multilayer perceptron and the transposed convolution layer of the mask decoder are fine-tuned using the LoRA module, including: obtaining an overall feature according to the image feature output by the Vit encoder and the prompt feature output by the prompt encoder; inputting the overall feature into the first LoRA block to obtain a first fine-tuning output; inputting the overall feature and the first fine-tuning output into the converter block to obtain a conversion output; inputting the conversion output into the transposed convolution layer and the second LoRA block respectively to obtain a transposed convolution output and a second fine-tuning output respectively; inputting the second fine-tuning output and the conversion output into the feature-image attention In the force block, an attention output is obtained; the attention output processed by the multi-layer perceptron and the transposed convolution output are point-multiplied to obtain a segmentation result; and the projection layer of the converter block and the feature-image attention block of the mask decoder are fine-tuned, including: freezing the weight of the projection layer; making the first LoRA block and the second LoRA block act on the query and value projection layer of the converter block and the feature-image attention block to obtain a weight update amount; according to the weight update amount, the weight of the projection layer is fine-tuned; so as to output the segmentation result of the breast ultrasound image according to the image feature and the prompt feature; The above steps are iterated to construct a breast ultrasound image segmentation model.

2. The method for constructing a breast ultrasound image segmentation model according to claim 1, characterized in that: Iterating the above steps to construct a breast ultrasound image segmentation model further includes: Acquire a training data set, wherein the training data set includes breast ultrasound images and labels corresponding thereto, wherein the breast ultrasound images include benign tumor ultrasound images and malignant tumor ultrasound images; Inputting the training data set into the Vit encoder of the initial breast ultrasound image segmentation model and inputting the prompt code corresponding to the training data into the prompt encoder of the initial breast ultrasound image segmentation model to obtain the predicted segmentation result output by the mask decoder of the initial breast ultrasound image segmentation model; Calculate a loss function based on the predicted segmentation result and the label; The initial breast ultrasound image segmentation model is iteratively trained using the loss function until a preset number of iterative training times is reached, thereby obtaining a trained ultrasound image segmentation model.

3. The method for constructing a breast ultrasound image segmentation model according to claim 2, characterized in that: Iteratively training the initial breast ultrasound image segmentation model using the loss function further includes: The parameters of the Vit encoder and the hint encoder are fixed, and the parameters of the mask decoder are updated.

4. The method for constructing a breast ultrasound image segmentation model according to claim 2, characterized in that: Before inputting the training data set into the Vit encoder of the initial breast ultrasound image segmentation model and inputting the prompt code corresponding to the training data into the prompt encoder of the initial breast ultrasound image segmentation model, the method further includes: Preprocess and augment the data in the acquired training data set; The preprocessing includes at least one or more of normalization and resizing of the breast ultrasound image, and the augmentation processing includes at least one or more of random cropping, brightness adjustment and contrast transformation of the breast ultrasound image and the corresponding label.

5. The method for constructing a breast ultrasound image segmentation model according to claim 2, characterized in that: The loss function is: L= (1- α ) L Dice + αL BCE ； in, L is the loss function, α is the weight coefficient, L Dice for Dice Loss function, L BCE is the binary cross entropy loss; ； ； in, P i Breast ultrasound images i The predicted value of pixels, G i Breast ultrasound images i The true value of pixels.

6. A breast ultrasound image segmentation method, characterized in that: The method comprises: Acquire a breast ultrasound image to be segmented and a prompt code corresponding to the breast ultrasound image to be segmented; Inputting the breast ultrasound image to be segmented and the prompt code into the breast ultrasound image segmentation model constructed by the breast ultrasound image segmentation model construction method according to any one of claims 1 to 5; A segmentation result of the breast ultrasound image output by the breast ultrasound image segmentation model is obtained.

7. A breast ultrasound image segmentation model construction device, characterized in that: The breast ultrasound image segmentation model includes a Vit encoder, a prompt encoder and a mask decoder, and the device includes: A first input module, used for inputting a breast ultrasound image into the Vit encoder to obtain image features; A second input module, used for inputting the prompt code corresponding to the breast ultrasound image into the prompt encoder to obtain a prompt feature; A fine-tuning and output module is used to use the LoRA module to fine-tune the weight parameters of the multi-layer perceptron and the transposed convolution layer of the mask decoder, including: obtaining an overall feature according to the image feature output by the Vit encoder and the prompt feature output by the prompt encoder; inputting the overall feature into the first LoRA block to obtain a first fine-tuning output; inputting the overall feature and the first fine-tuning output into the converter block to obtain a conversion output; inputting the conversion output into the transposed convolution layer and the second LoRA block respectively, respectively obtaining a transposed convolution output and a second fine-tuning output; inputting the second fine-tuning output and the conversion output into the feature -image attention block, obtain the attention output; perform point multiplication on the attention output processed by the multilayer perceptron and the transposed convolution output to obtain the segmentation result; and fine-tune the projection layer of the converter block and the feature-image attention block of the mask decoder, including: freezing the weight of the projection layer; making the first LoRA block and the second LoRA block act on the query and value projection layer of the converter block and the feature-image attention block to obtain the weight update amount; according to the weight update amount, fine-tune the weight of the projection layer; to output the segmentation result of the breast ultrasound image according to the image feature and the prompt feature; The construction module is used to iterate the above steps to construct a breast ultrasound image segmentation model.

8. A breast ultrasound image segmentation device, characterized in that: The device comprises: An acquisition module, used for acquiring a breast ultrasound image to be segmented and a prompt code corresponding to the breast ultrasound image to be segmented; An input module, used for inputting the breast ultrasound image to be segmented and the prompt code into the breast ultrasound image segmentation model constructed by the breast ultrasound image segmentation model construction method according to any one of claims 1 to 5; An acquisition module is used to obtain the segmentation result of the breast ultrasound image output by the breast ultrasound image segmentation model.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

11. A computer program product, characterized in that The method comprises at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the method according to any one of claims 1 to 6.

Images