Ultrasonic image segmentation model training method and device

Abstract

The embodiment of the invention provides an ultrasonic image segmentation model training method and device, and the method comprises the steps: obtaining a plurality of sample images containing lesion region labeling information, inputting the sample images into a multi-scale image segmentation model, and obtaining a predicted lesion segmentation image; performing edge extraction processing on each lesion segmentation region in the predicted lesion segmentation map to obtain a corresponding edge contour region; determining an edge contour loss value and a non-edge contour loss value based on the lesion region labeling information and the edge contour region of each lesion segmentation region; and based on the edge contour loss value and the non-edge contour loss value, training the multi-scale image segmentation model to obtain a target multi-scale image segmentation model, and performing model training through integrating feature information of different scales in the ultrasonic sample image and through the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model. Therefore, the model pays more attention to key information in the sample image, and the segmentation accuracy of the image segmentation model is improved on the whole.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data processing technology, and in particular to a method and device for training an ultrasound image segmentation model. Background Art

[0002] Medical ultrasound images play an important role in medical diagnosis and treatment. Image segmentation models are widely used to process medical ultrasound images and assist doctors in diagnosis and treatment based on the processing results, thereby improving diagnosis and treatment efficiency. Typically, before using an image segmentation model, it is necessary to train it using ultrasound sample images so that the trained image segmentation model can accurately segment ultrasound images.

[0003] However, the commonly used image segmentation models currently have a single receptive field and can only focus on local information in the image, which makes the trained image segmentation model less accurate. In addition, the image segmentation model is usually trained using the loss value corresponding to the entire predicted segmentation area of the ultrasound sample image. Due to the inherent noise, low contrast and artifacts of the ultrasound sample image, the ultrasound sample image contains a large amount of non-key information, which makes the use of the loss value corresponding to the entire segmentation area for model training lead to the problem of low accuracy of image segmentation model training. Summary of the Invention

[0004] An embodiment of the present invention provides an ultrasound image segmentation model training method and device, which uses a multi-scale image segmentation model to integrate feature information of different scales in an ultrasound sample image, thereby improving the segmentation accuracy of the image segmentation model. The model is trained using edge contour loss values and non-edge contour loss values, so that the model pays more attention to key information in the sample image, further improving the segmentation accuracy of the image segmentation model.

[0005] In a first aspect, an embodiment of the present invention provides a method for training an ultrasound image segmentation model, the method comprising:

[0006] Acquire an ultrasound image sample set; wherein the ultrasound image sample set includes at least one ultrasound sample image to be used that includes lesion area annotation information;

[0007] Inputting the ultrasound sample image to be applied into a multi-scale image segmentation model to be trained for processing to obtain a predicted lesion segmentation map including at least one lesion segmentation region; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied;

[0008] For the at least one lesion segmentation region in the predicted lesion segmentation map, performing edge extraction processing on the lesion segmentation region to obtain an edge contour region corresponding to each lesion segmentation region;

[0009] Determining an edge contour loss value and a non-edge contour loss value based on the lesion region labeling information and the edge contour region corresponding to the at least one lesion segmentation region;

[0010] The multi-scale image segmentation model is trained based on the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model.

[0011] In a second aspect, an embodiment of the present invention further provides an ultrasound image segmentation model training device, the device comprising:

[0012] A sample set acquisition module is used to acquire an ultrasound image sample set; wherein the ultrasound image sample set includes at least one ultrasound sample image to be used that contains lesion area annotation information;

[0013] a predicted lesion segmentation module, configured to input the ultrasound sample image to be applied into a multi-scale image segmentation model to be trained for processing, thereby obtaining a predicted lesion segmentation map including at least one lesion segmentation region; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied;

[0014] an edge region determination module, configured to perform edge extraction processing on the at least one lesion segmentation region in the predicted lesion segmentation map to obtain an edge contour region corresponding to each lesion segmentation region;

[0015] A loss value determination module, configured to determine an edge contour loss value and a non-edge contour loss value based on the lesion region labeling information and an edge contour region corresponding to the at least one lesion segmentation region;

[0016] The model training module is used to train the multi-scale image segmentation model based on the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising:

[0018] one or more processors;

[0019] a storage device for storing one or more programs,

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the ultrasound image segmentation model training method as described in any one of the embodiments of the present invention.

[0021] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute the ultrasound image segmentation model training method as described in any one of the embodiments of the present invention.

[0022] The technical solution of an embodiment of the present invention is to obtain a plurality of ultrasound sample images to be applied including lesion area annotation information, input the ultrasound sample images to be applied into a multi-scale image segmentation model to be trained for processing, and obtain a predicted lesion segmentation map including at least one lesion segmentation area; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied, and then, for at least one lesion segmentation area in the predicted lesion segmentation map, edge extraction processing is performed on the lesion segmentation area to obtain an edge contour area corresponding to each lesion segmentation area, and further, based on the lesion area annotation information and the edge contour area corresponding to at least one lesion segmentation area, an edge contour loss value and a non-edge contour loss value are determined, thereby, based on the edge contour loss value and the non-edge contour loss value, the multi-scale image segmentation model is trained to obtain a target multi-scale image segmentation model. The technical solution provided in this embodiment adopts a multi-scale image segmentation model, which can integrate feature information of different scales in the ultrasound sample image, increase the receptive field while paying attention to local information, make full use of feature information, and improve the segmentation accuracy of the image segmentation model; and train the model by predicting the edge contour loss value and non-edge contour loss value of each lesion segmentation area in the lesion segmentation map, so that the model pays more attention to the key information in the sample image, further improving the segmentation accuracy of the image segmentation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings introduced here only illustrate some of the embodiments to be described by the present invention, and are not exhaustive. A person skilled in the art can derive other drawings based on these drawings without inventive effort.

[0024] Figure 1 A schematic diagram of a flow chart of an ultrasound image segmentation model training method provided by an embodiment of the present invention;

[0025] Figure 2 Schematic diagram of the structure of a multi-scale image segmentation model involved in an embodiment of the present invention;

[0026] Figure 3 Schematic diagram of the structure of each compression and excitation module involved in an embodiment of the present invention;

[0027] Figure 4 Schematic diagram of the structure of each multi-scale linear attention module involved in an embodiment of the present invention;

[0028] Figure 5 A schematic flow chart of another ultrasound image segmentation model training method provided by an embodiment of the present invention;

[0029] Figure 6 Schematic diagram of the implementation process of determining the edge contour area corresponding to the current lesion segmentation area involved in the embodiment of the present invention;

[0030] Figure 7 A schematic structural diagram of an ultrasound image segmentation model training device provided by an embodiment of the present invention;

[0031] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0033] Example 1

[0034] Figure 1 This is a flow chart of a method for training an ultrasound image segmentation model provided in an embodiment of the present invention. This embodiment is applicable to situations where an image segmentation model is trained based on ultrasound sample images. The method can be executed by an ultrasound image segmentation model training device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server.

[0035] like Figure 1 As shown, the ultrasound image segmentation model training method includes:

[0036] S110: Acquire an ultrasound image sample set.

[0037] The ultrasound image sample set includes at least one ultrasound sample image to be applied that includes lesion region annotation information. The ultrasound sample image to be applied is a sample image that will be used to train the image segmentation model. Each ultrasound sample image to be applied contains at least one lesion region and has corresponding lesion region annotation information. The lesion region annotation information is labeling information corresponding to the lesion region in the ultrasound sample image to be applied.

[0038] In this embodiment, an ultrasound image sample set for training a multi-scale image segmentation model can be pre-constructed and stored in a preset storage space. When the multi-scale image segmentation model needs to be trained, the ultrasound image sample set can be obtained from the preset storage space.

[0039] Next, the process of constructing the ultrasound image sample set may be described in detail. Optionally, the specific implementation steps of constructing the ultrasound image sample set may include:

[0040] S1. Acquire at least one original ultrasound image including a lesion area.

[0041] In this embodiment, the original ultrasound image may be an ultrasound image of the lesion site of the patient taken in a historical period obtained from a hospital's image database or through other means.

[0042] It should be noted that the multiple original ultrasound images obtained may have inconsistent image sizes. Here, the obtained original ultrasound images can be resampled according to a preset size. For example, all original ultrasound images can be resampled to a size of 512×512.

[0043] S2. Perform normalization processing on at least one original ultrasound image and a current original ultrasound image to obtain an ultrasound image to be used.

[0044] The ultrasound image to be used is an ultrasound image obtained by normalizing the original image.

[0045] Specifically, the normalization process for each original ultrasound image is consistent. This description uses any one of the original ultrasound images as an example. The specific implementation process of normalizing the current original ultrasound image to obtain the ultrasound image to be used may include: retrieving a predetermined first pixel mean and a first pixel variance; and processing the pixel values of all pixels in the current original ultrasound image based on the first pixel mean and the first pixel variance to obtain the ultrasound image to be used.

[0046] The first pixel mean and the first pixel variance are determined according to all pixel values of the region of interest in all original ultrasound images, where the region of interest refers to any organ and lesion area in the original ultrasound image.

[0047] In this embodiment, based on the first pixel mean and the first pixel variance, each pixel in the current original ultrasound image can be subjected to z-score normalization processing using the first pixel mean and the first pixel variance to obtain the ultrasound image to be used. For example, the first pixel mean can be expressed as x mean , the first pixel variance can be expressed as x std , for the pixel value x of the i-th pixel in the current original ultrasound image i For example, the corresponding normalized pixel value is

[0048] S3. Perform image enhancement processing on the ultrasound image to be used based on a preset image enhancement algorithm to obtain at least one ultrasound sample image to be applied.

[0049] The ultrasound sample image to be applied is a sample image obtained after image enhancement processing is performed on the ultrasound image to be used.

[0050] In this embodiment, various data augmentation methods can be used to augment ultrasound sample images to increase sample diversity and improve the generalization capability of the model. The preset image enhancement algorithm is a preset image enhancement algorithm used to augment sample images. Specifically, the preset image enhancement algorithm may include at least one of an image rotation enhancement algorithm, an image flipping enhancement algorithm, a pixel translation enhancement algorithm, a noise addition enhancement algorithm, a linear transformation enhancement algorithm, a nonlinear transformation enhancement algorithm, and a display parameter adjustment enhancement algorithm.

[0051] Specifically, each ultrasound sample image to be used can be processed using at least one preset image enhancement algorithm to obtain at least one corresponding ultrasound sample image to be applied. Based on the same processing method, a large number of ultrasound sample images to be applied can be obtained.

[0052] S4. Perform lesion region labeling processing on at least one ultrasound sample image to be applied based on a preset labeling tool to obtain lesion region labeling information corresponding to each ultrasound image to be applied.

[0053] The preset annotation tool is a user operation interface for annotating the lesion area in the ultrasound sample image to be applied.

[0054] Specifically, for any ultrasound sample image to be applied, the user can use the preset annotation tool to circle the lesion area on the sample image. The coordinates of the selected area are stored in a JSON table format, and this data content is used as the lesion area annotation information for the ultrasound image to be applied. Based on the same processing method, the lesion area annotation information corresponding to each lesion area in each ultrasound sample image to be applied can be obtained.

[0055] S5. Construct an ultrasound image sample set based on each ultrasound sample image to be applied and the corresponding lesion area annotation information.

[0056] In this embodiment, each ultrasound sample image to be applied and the corresponding lesion region annotation information may be determined as an ultrasound image sample, and thus a set of ultrasound image samples may be determined as an ultrasound image sample set.

[0057] S120 , inputting the ultrasound sample image to be applied into the multi-scale image segmentation model to be trained for processing, and obtaining a predicted lesion segmentation map including at least one lesion segmentation region.

[0058] The multiscale image segmentation model is the image segmentation model that will be trained based on the ultrasound image sample set. The model parameters in the multiscale image to be trained are the initial values. These parameters need to be modified during the model training process to obtain the target multiscale image segmentation model used to perform the ultrasound image segmentation task. The predicted lesion segmentation map is the predicted lesion region segmentation map obtained by the multiscale image segmentation model to be trained on the ultrasound sample image to be used.

[0059] In this embodiment, the structural diagram of the multi-scale image segmentation model is shown in Figure 2 .like Figure 2 As shown, the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied. Specifically, the encoder in the multi-scale image segmentation model is composed of a compression and excitation module (SE-block) and a downsampling module from top to bottom, and encodes the ultrasound sample image to be applied of size [H, W] into feature maps of [H, W], [H / 2, W / 2], [H / 4, W / 4], and [H / 8, W / 8] in sequence. Optionally, the structural diagram of each compression and excitation module can be seen in Figure 3 The decoder consists of a bottom-up multi-scale linear attention module (MLA-Block) and upsampling. The decoder can combine the feature maps of the same layer in the encoder to form a new decoding vector. Finally, the decoding vector is processed by the convolution layer to obtain the predicted lesion segmentation map corresponding to the ultrasound sample image to be applied.

[0060] Specifically, the ultrasound sample image to be applied is input into the multi-scale image segmentation model to be trained for processing, and a specific implementation method of obtaining a predicted lesion segmentation map including at least one lesion segmentation area includes:

[0061] S1. Input the ultrasound sample image to be applied into an encoder in a multi-scale image segmentation model to obtain a first-scale feature map, a second-scale feature map, a third-scale feature map, and a fourth-scale feature map corresponding to the ultrasound sample image to be applied.

[0062] In this embodiment, if Figure 2 As shown, the ultrasonic sample image to be applied is input into the encoder in the multi-scale image segmentation model. After the first compression and excitation module in the encoder processes the ultrasonic sample image to be applied, a first-scale feature map of size [H, W] is obtained; further, the first-scale feature map is downsampled and input into the second compression and excitation module in the encoder to obtain a second-scale feature map of size [H / 2, W / 2]; further, the second-scale feature map is downsampled and input into the third compression and excitation module in the encoder to obtain a third-scale feature map of size [H / 4, W / 4]; finally, the third-scale feature map is downsampled and input into the fourth compression and excitation module in the encoder to obtain a fourth-scale feature map of size [H / 8, W / 8].

[0063] S2. Input the fourth-scale feature map into the first multi-scale linear attention module in the decoder of the multi-scale image segmentation model for multi-scale decoding to obtain a first decoding vector.

[0064] In this embodiment, the decoder of the multiscale image segmentation model includes the same number of multiscale linear attention modules as the compression and excitation modules in the encoder. The first multiscale linear attention module is the multiscale linear attention module that processes the fourth-scale feature map. The first decoded vector is the output of the first multiscale linear attention module.

[0065] In this embodiment, each multi-scale linear attention module includes: a feature extraction module, a first-scale decoding module, a second-scale decoding module, a third-scale decoding module, and a fully connected layer, wherein the first-scale decoding module includes a linear attention mechanism unit, the second-scale decoding module includes a first depth-separable convolution unit, a preset convolution unit, and a linear attention mechanism unit, and the third-scale decoding module includes a second depth-separable convolution unit, a preset convolution unit, and a linear attention mechanism unit. For example, the structural diagram of each multi-scale linear attention module can be found in Figure 4 ,like Figure 4As shown in the figure, the feature extraction module includes: 3×3 depth-separable convolution (DWconv3×3), 1×1 convolution and fully connected layer (Linear Projection); the first-scale decoding module is the ReLU linear attention mechanism; the second-scale decoding module includes: 3×3 depth-separable convolution, 1×1 convolution layer and ReLU linear attention mechanism; the third-scale decoding module includes: 5×5 depth-separable convolution, 1×1 convolution layer and ReLU linear attention mechanism.

[0066] Specifically, the first-scale feature map is input into the decoder in the multi-scale image segmentation model for multi-scale decoding processing, and the specific implementation method of obtaining the first decoding vector may include: inputting the first-scale feature map into the feature extraction module of the first multi-scale linear attention module to obtain the decoding features to be processed; inputting the decoding features to be processed into the first-scale decoding module, the second-scale decoding module and the third-scale decoding module respectively to obtain the first decoding vector to be spliced, the second decoding vector to be spliced and the third decoding vector to be spliced; after splicing the first decoding vector to be spliced, the second decoding vector to be spliced and the third decoding vector to be spliced, the vectors are input into the fully connected layer to obtain the first decoding vector.

[0067] On the basis of the above examples, continue to see Figure 4 , the first scale feature map is input into the feature extraction module, and 3×3 depth separable convolution, 1×1 convolution and full connection layer (Linear Projection) processing, the feature F1 (i.e., the decoding feature to be processed) is obtained; then, the decoding feature F1 to be processed is input into the first-scale decoding module, and after being processed by the ReLU linear attention mechanism, the first decoding vector F2 to be spliced is obtained; the decoding feature F1 to be processed is input into the second-scale decoding module, and after 3×3 depth-separable convolution, 1×1 convolution and then ReLU linear attention mechanism operation, the second decoding vector F3 to be spliced is obtained; the decoding feature F1 to be processed is input into the third-scale decoding module, and after 5×5 depth-separable convolution, 1×1 convolution layer and then ReLU linear attention mechanism operation, the third decoding vector F4 to be spliced is obtained; at this time, the F2, F3, and F4 features respectively represent the feature information under different receptive fields. Further, the F2, F3, and F4 features are cascaded to obtain comprehensive information, and finally a fully connected layer is used to obtain the final feature result, which is the first decoding vector.

[0068] S3. Upsample the first decoding vector to obtain a first decoding vector to be processed, and determine a first fused feature map based on the first decoding vector to be processed and the third-scale feature map. Perform multi-scale decoding on the first fused feature map based on the second multi-scale linear attention module in the decoder to obtain a second decoding vector.

[0069] Among them, the structure of the second multi-scale linear attention module is consistent with that of the first multi-scale linear attention module.

[0070] Specifically, such as Figure 2 As shown in the figure, after obtaining the first decoded vector, the first decoded vector is upsampled to obtain a first unprocessed decoded vector. Subsequently, the first unprocessed decoded vector and the third-scale feature map are concatenated to obtain a first fused feature map. Thus, the first fused feature map is input into the second multi-scale linear attention module in the decoder for multi-scale decoding to obtain a second decoded vector. The specific processing process of the second multi-scale linear attention module is consistent with that of the first multi-scale linear attention module and will not be repeated here.

[0071] S4. Upsample the second decoded vector to obtain a second decoded vector to be processed, and determine a second fused feature map based on the second decoded vector to be processed and the second-scale feature map. Perform multi-scale decoding on the second fused feature map based on the third multi-scale linear attention module in the decoder to obtain a third decoded vector.

[0072] For details, see Figure 2 After obtaining the second decoded vector, the second decoded vector is upsampled to obtain a second unprocessed decoded vector. Subsequently, the second unprocessed decoded vector and the second-scale feature map are concatenated to obtain a second fused feature map. The second fused feature map is then input into the third multi-scale linear attention module in the decoder for multi-scale decoding, obtaining a third decoded vector. The specific processing of the third multi-scale linear attention module is consistent with that of the first multi-scale linear attention module and will not be further described here.

[0073] S5. Upsample the third decoding vector to obtain a third decoding vector to be processed, and determine a third fused feature map based on the third decoding vector to be processed and the first-scale feature map. Perform multi-scale decoding on the third fused feature map based on the fourth multi-scale linear attention module in the decoder to obtain a target decoding vector.

[0074] For details, see Figure 2After obtaining the third decoded vector, the third decoded vector is upsampled to obtain a third unprocessed decoded vector. Subsequently, the third unprocessed decoded vector and the first-scale feature map are concatenated to obtain a third fused feature map. The third fused feature map is then input into the fourth multi-scale linear attention module in the decoder for multi-scale decoding to obtain a target decoded vector. The specific processing of the fourth multi-scale linear attention module is consistent with that of the first multi-scale linear attention module and will not be further described here.

[0075] S6. Obtain a predicted lesion segmentation map including at least one lesion segmentation region based on the target decoding vector.

[0076] In this embodiment, after obtaining the target decoding vector, the target decoding vector is input into a 1×1 convolutional layer in the encoder to obtain a predicted lesion segmentation map including at least one lesion segmentation region corresponding to the ultrasound sample image to be applied.

[0077] S130 , performing edge extraction processing on at least one lesion segmentation region in the predicted lesion segmentation map to obtain an edge contour region corresponding to each lesion segmentation region.

[0078] Among them, the lesion segmentation area is the target area corresponding to the suspected lesion site in the predicted lesion segmentation map.

[0079] The edge contour area refers to the area around and along the edge of the lesion segmentation area.

[0080] In this embodiment, the predicted lesion segmentation map may include one or more lesion segmentation regions. The edge region of the lesion segmentation region in the predicted lesion segmentation map can be extracted using a preset edge extraction algorithm to obtain an edge contour region corresponding to each lesion segmentation region.

[0081] S140 : Determine an edge contour loss value and a non-edge contour loss value based on the lesion region labeling information and an edge contour region corresponding to at least one lesion segmentation region.

[0082] The edge contour loss value is used to characterize the difference between the pixel values of the edge contour area of a lesion segmentation region and the pixel values corresponding to the edge contour area in the lesion region annotation information. The non-edge contour loss value is used to characterize the difference between the pixel values of the non-edge contour area of a lesion segmentation region and the pixel values corresponding to the non-edge contour area in the lesion region annotation information.

[0083] Specifically, for any lesion segmentation area, when the corresponding edge contour area is determined, it is easy to determine the corresponding non-edge contour area. On this basis, for the edge contour area, the edge contour loss value can be determined based on the pixel value of the edge contour area of the lesion segmentation area, the pixel value corresponding to the edge contour area in the lesion area annotation information, and the edge contour area loss function; for the non-edge contour area, the non-edge contour loss value can be determined based on the pixel value of the non-edge contour area of the lesion segmentation area, the pixel value corresponding to the non-edge contour area in the lesion area annotation information, and the non-edge contour area loss function. Optionally, both the edge contour area loss function and the non-edge contour area loss function can be calculated using the cross entropy loss method.

[0084] S150 , training a multi-scale image segmentation model based on the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model.

[0085] Among them, the target multi-scale image segmentation model is the multi-scale image segmentation model that is finally trained.

[0086] The preset conditions are pre-set conditions. When training the multi-scale image segmentation model, when the training results meet the preset conditions, it indicates that the multi-scale image segmentation model training is completed.

[0087] Optionally, the preset conditions include at least one of the following situations: 1) the number of training iterations reaches a preset iteration threshold; 2) the edge contour loss value and the non-edge contour loss value are respectively less than their corresponding loss thresholds; 3) the comprehensive loss value determined by the edge contour loss value and the non-edge contour loss value is less than the comprehensive loss threshold.

[0088] In an embodiment of the present invention, the parameters of the multi-scale image segmentation model are optimized based on the edge contour loss value and non-edge contour loss value of each lesion segmentation area in the predicted lesion segmentation map. Each time the ultrasound sample image to be applied is input into the multi-scale image segmentation model, a set of edge contour loss values and non-edge contour loss values can be determined. The edge contour loss values and non-edge contour loss values determined each time can be fed back to the multi-scale image segmentation model to optimize the parameters in the multi-scale image segmentation model. As another specific implementation method, a comprehensive loss value can be determined from the edge contour loss values and non-edge contour loss values, and the comprehensive loss value can be fed back to the multi-scale image segmentation model to optimize the parameters in the multi-scale image segmentation model. Subsequently, the multi-scale image segmentation model after model optimization processes the next set of ultrasound sample images to be applied. An iteration threshold can be pre-set. When the number of times the multi-scale image segmentation model is trained based on the ultrasound sample images to be applied reaches the iteration threshold, and the edge contour loss values and non-edge contour loss values are respectively less than their corresponding loss thresholds, it can be proved that the image processing model training is complete. In another embodiment, when the number of times the multi-scale image segmentation model is trained based on the ultrasound sample image to be applied reaches an iteration threshold and the comprehensive loss value is less than the loss threshold, it can be proved that the training of the multi-scale image segmentation model is completed. At this time, the multi-scale image segmentation model obtained is the target multi-scale image segmentation model.

[0089] On the basis of the above technical solutions, after S150, it may also include: obtaining a target ultrasound image corresponding to a preset detection site; inputting the target ultrasound image into a target multi-scale image segmentation model after training, and determining a segmentation result corresponding to the suspected lesion site in the target ultrasound image according to the output of the target multi-scale image segmentation model.

[0090] The target ultrasound image is an ultrasound image to which the target multi-scale image segmentation model is to be applied for lesion area detection.

[0091] In this embodiment, the ultrasound image sample set may include ultrasound images of at least one preset detection site containing a lesion area, where the preset detection site may include the kidney, liver, lung, etc. Based on this, the target multi-scale image segmentation model ultimately trained can be used to detect suspected lesion areas in the ultrasound image corresponding to the at least one preset detection site. In addition, the ultrasound image sample set may only include ultrasound images of a certain type of preset detection site containing a lesion area. In this case, the target multi-scale image segmentation model ultimately trained can be used to detect suspected lesion areas in the ultrasound image corresponding to that type of preset detection site.

[0092] The technical solution of an embodiment of the present invention is to obtain a plurality of ultrasound sample images to be applied including lesion area annotation information, input the ultrasound sample images to be applied into a multi-scale image segmentation model to be trained for processing, and obtain a predicted lesion segmentation map including at least one lesion segmentation area; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied, and then, for at least one lesion segmentation area in the predicted lesion segmentation map, edge extraction processing is performed on the lesion segmentation area to obtain an edge contour area corresponding to each lesion segmentation area, and further, based on the lesion area annotation information and the edge contour area corresponding to at least one lesion segmentation area, an edge contour loss value and a non-edge contour loss value are determined, thereby, based on the edge contour loss value and the non-edge contour loss value, the multi-scale image segmentation model is trained to obtain a target multi-scale image segmentation model. The technical solution provided in this embodiment adopts a multi-scale image segmentation model, which can integrate feature information of different scales in the ultrasound sample image, increase the receptive field while paying attention to local information, make full use of feature information, and improve the segmentation accuracy of the image segmentation model; and train the model by predicting the edge contour loss value and non-edge contour loss value of each lesion segmentation area in the lesion segmentation map, so that the model pays more attention to the key information in the sample image, further improving the segmentation accuracy of the image segmentation model.

[0093] Example 2

[0094] Figure 5 This is a flowchart of a method for training an ultrasound image segmentation model provided in Example 2 of the present invention. Based on the previous example, S130-S150 are further refined. For specific implementations, please refer to the detailed description of the present embodiment. Technical features that are identical or similar to those in the previous example are not repeated here.

[0095] like Figure 5 As shown, the ultrasound image segmentation model training method includes:

[0096] S210 , obtaining an ultrasound image sample set; wherein the ultrasound image sample set includes at least one ultrasound sample image to be used that includes lesion region annotation information.

[0097] S220 , inputting the ultrasound sample image to be applied into the multi-scale image segmentation model to be trained for processing, and obtaining a predicted lesion segmentation map including at least one lesion segmentation region.

[0098] The multi-scale image segmentation model includes an encoder and a decoder for performing multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied.

[0099] S230 , performing dilation processing and erosion processing on the current lesion segmentation region for at least one lesion segmentation region in the predicted lesion segmentation image, respectively, to obtain a dilated image of the lesion region and an eroded image of the lesion region.

[0100] In this embodiment, in order to quickly and efficiently determine the edge contour area corresponding to the lesion segmentation area, this can be achieved by performing dilation processing and erosion processing on the lesion segmentation area. The dilation processing refers to the operation of expanding the foreground area by "filling" the background pixels in the image with structuring elements. The erosion processing refers to the operation of shrinking the foreground area by "scraping" the foreground pixels in the image with structuring elements. The dilated image of the lesion area refers to the image obtained after the current lesion segmentation area is dilated; the eroded image of the lesion area refers to the image obtained after the current lesion segmentation area is eroded.

[0101] Specifically, the processing process for each lesion segmentation region in the predicted lesion segmentation map is consistent. Here, an arbitrary lesion segmentation region is used as an example for explanation. By dilating the current lesion segmentation region, a dilated image of the lesion region corresponding to the current lesion segmentation region can be obtained; by eroding the current lesion segmentation region, an eroded image of the lesion region corresponding to the current lesion segmentation region can be obtained.

[0102] S240 : Determine an edge contour region corresponding to the current lesion segmentation region based on the lesion region expansion image and the lesion region erosion image.

[0103] In this embodiment, the schematic diagram of the implementation process of determining the edge contour area corresponding to the current lesion segmentation area is shown in FIG. Figure 6 ,like Figure 6 As shown, the edge contour area corresponding to the current lesion segmentation area can be obtained by performing difference processing on the lesion area expansion image and the lesion area erosion image.

[0104] S250 , determining an edge contour loss value based on the pixel values of the edge contour region of each lesion segmentation region, the pixel values corresponding to the edge contour region in the lesion region annotation information, and the first loss function.

[0105] In this embodiment, the first loss function is a function for calculating the loss value of the edge contour area of the lesion segmentation area. Optionally, the first loss function can be expressed as:

[0106]

[0107] Where CE_loss1 represents the edge contour loss value, n represents the total number of lesion segmentation areas, q(x i) represents the pixel value corresponding to the edge contour area pixel in the i-th lesion segmentation area, p(x i ) represents the pixel value corresponding to the edge contour area in the lesion area annotation information.

[0108] In this embodiment, on the basis of obtaining the pixel value of the edge contour area of each lesion segmentation area, the pixel value of each edge contour area and the pixel value corresponding to the edge contour area in the lesion area annotation information are substituted into the first loss function for calculation to obtain the edge contour loss value.

[0109] S260 : For each lesion segmentation region, determine a non-edge contour region of the current lesion segmentation region based on the edge contour region.

[0110] In this embodiment, for the current lesion segmentation region, when the edge contour region is known, the image region inside the edge contour region is the non-edge contour region of the current lesion segmentation region. Based on the same processing method, the non-edge contour region of each lesion segmentation region can be obtained.

[0111] S270 , determining a non-edge contour loss value based on the pixel values of the non-edge contour area of each lesion segmentation area, the pixel values corresponding to the non-edge contour area in the lesion area annotation information, and the second loss function.

[0112] In this embodiment, the second loss function is a function for calculating the loss value of the non-edge contour area of the lesion segmentation area. Optionally, the second loss function can be expressed as:

[0113]

[0114] Where CE_loss2 represents the non-edge contour loss value, n represents the total number of lesion segmentation areas, q(y i ) represents the pixel value corresponding to the pixel in the non-edge contour area of the i-th lesion segmentation area, p(y i ) represents the pixel value corresponding to the non-edge contour area in the lesion area annotation information.

[0115] In this embodiment, on the basis of obtaining the pixel value of the non-edge contour area of each lesion segmentation area, the pixel value of each non-edge contour area and the pixel value corresponding to the non-edge contour area in the lesion area annotation information are substituted into the second loss function for calculation to obtain the non-edge contour loss value.

[0116] S280: Determine a comprehensive loss value based on the first preset weight, the second preset weight, the edge contour loss value, and the non-edge contour loss value.

[0117] The first preset weight and the second preset weight are preset weight values. The first preset weight is the weight value corresponding to the edge contour loss value, and the second preset weight is the weight value corresponding to the non-edge contour loss value. The first preset weight is greater than the second preset weight. The comprehensive loss value is the weighted sum of the edge contour loss value and the non-edge contour loss value.

[0118] Specifically, the first preset weight can be expressed as W1, the second preset weight can be expressed as W2, and the comprehensive loss value can be expressed as Y=W1×CE_loss1+W2×CE_loss2.

[0119] S290. Back-propagate the comprehensive loss value to the multi-scale image segmentation model, adjust the network parameters in the multi-scale image segmentation model, and terminate the training when the preset conditions are met to obtain the target multi-scale image segmentation model.

[0120] The technical solution of the embodiment of the present invention, when determining the edge contour area of the lesion segmentation area, performs expansion processing and corrosion processing on the current lesion segmentation area respectively, and obtains the lesion area expansion image and the lesion area corrosion image. Then, based on the lesion area expansion image and the lesion area corrosion image, the edge contour area corresponding to the current lesion segmentation area is determined, which can quickly and efficiently determine the edge contour area corresponding to the lesion segmentation area. When determining the edge contour loss value and the non-edge contour loss value, the edge contour loss value and the non-edge contour loss value are determined by a pre-configured first loss function and a second loss function, thereby improving the efficiency of determining the loss value. Furthermore, a comprehensive loss value is determined based on the weighted result of the edge contour loss value and the non-edge contour loss value, and the network parameters in the multi-scale image segmentation model are trained and adjusted based on the comprehensive loss value, further improving the segmentation accuracy of the multi-scale image segmentation model.

[0121] Example 3

[0122] Figure 7 A schematic diagram of the structure of an ultrasound image segmentation model training device provided by an embodiment of the present invention is shown in FIG. Figure 7 As shown, the apparatus includes: a sample set acquisition module 310 , a lesion prediction segmentation module 320 , a margin area determination module 330 , a loss value determination module 340 and a model training module 350 .

[0123] The sample set acquisition module 310 is configured to acquire an ultrasound image sample set; wherein the ultrasound image sample set includes at least one ultrasound sample image to be used that includes lesion region annotation information;

[0124] The predicted lesion segmentation module 320 is configured to input the ultrasound sample image to be applied into a multi-scale image segmentation model to be trained for processing, thereby obtaining a predicted lesion segmentation map including at least one lesion segmentation region; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied;

[0125] An edge region determination module 330 is configured to perform edge extraction processing on the at least one lesion segmentation region in the predicted lesion segmentation map to obtain an edge contour region corresponding to each lesion segmentation region;

[0126] A loss value determining module 340 is configured to determine an edge contour loss value and a non-edge contour loss value based on the lesion region labeling information and the edge contour region corresponding to the at least one lesion segmentation region;

[0127] The model training module 350 is used to train the multi-scale image segmentation model based on the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model.

[0128] Based on the above embodiment, optionally, the ultrasound image segmentation model training device further includes a sample set construction module; the sample set construction module is used to construct an ultrasound image sample set; the sample set construction module includes:

[0129] an original image acquisition unit, configured to acquire at least one original ultrasound image including a lesion area;

[0130] a normalization processing unit, configured to perform normalization processing on the at least one original ultrasound image and the current original ultrasound image to obtain an ultrasound image to be used;

[0131] an image enhancement processing unit, configured to perform image enhancement processing on the ultrasound image to be used based on a preset image enhancement algorithm to obtain at least one ultrasound sample image to be applied;

[0132] a sample labeling unit, configured to perform lesion region labeling processing on the at least one ultrasound sample image to be applied based on a preset labeling tool, and obtain lesion region labeling information corresponding to each ultrasound sample to be applied;

[0133] The sample set construction unit is configured to construct an ultrasound image sample set based on each of the ultrasound sample images to be applied and the corresponding lesion area annotation information.

[0134] Based on the above embodiment, optionally, a normalization processing unit is specifically used to call a predetermined first pixel mean and a first pixel variance; wherein, the first pixel mean and the first pixel variance are determined based on all pixel values of the region of interest in all the original ultrasound images; for all pixel points in the current original ultrasound image, the pixel values of the pixel points are processed based on the first pixel mean and the first pixel variance to obtain the ultrasound image to be used.

[0135] Based on the above embodiment, optionally, the lesion segmentation prediction module 320 includes:

[0136] a feature map determining unit, configured to input the ultrasound sample image to be applied into an encoder in the multi-scale image segmentation model, and obtain a first-scale feature map, a second-scale feature map, a third-scale feature map, and a fourth-scale feature map corresponding to the ultrasound sample image to be applied;

[0137] a first decoding vector determining unit, configured to input the fourth-scale feature map into a first multi-scale linear attention module in a decoder of the multi-scale image segmentation model for multi-scale decoding processing to obtain a first decoding vector;

[0138] a second decoding vector determining unit, configured to perform upsampling processing on the first decoding vector to obtain a first decoding vector to be processed, determine a first fused feature map based on the first decoding vector to be processed and the third scale feature map, and perform multi-scale decoding processing on the first fused feature map based on a second multi-scale linear attention module in the decoder to obtain a second decoding vector;

[0139] a third decoding vector determining unit, configured to perform upsampling processing on the second decoding vector to obtain a second decoding vector to be processed, determine a second fused feature map based on the second decoding vector to be processed and the second scale feature map, and perform multi-scale decoding processing on the second fused feature map based on a third multi-scale linear attention module in the decoder to obtain a third decoding vector;

[0140] a fourth decoding vector determining unit, configured to perform upsampling processing on the third decoding vector to obtain a third decoding vector to be processed, determine a third fused feature map based on the third decoding vector to be processed and the first scale feature map, and perform multi-scale decoding processing on the third fused feature map based on a fourth multi-scale linear attention module in the decoder to obtain a target decoding vector;

[0141] The lesion segmentation map determining unit is configured to obtain a predicted lesion segmentation map including at least one lesion segmentation region based on the target decoding vector.

[0142] Based on the above embodiment, optionally, the decoder includes a feature extraction module, a first-scale decoding module, a second-scale decoding module, a third-scale decoding module, and a fully connected layer. The first decoding vector determination unit is specifically used to input the first-scale feature map into the feature extraction module of the first multi-scale linear attention module to obtain decoding features to be processed; input the decoding features to be processed into the first-scale decoding module, the second-scale decoding module, and the third-scale decoding module respectively to obtain a first decoding vector to be spliced, a second decoding vector to be spliced, and a third decoding vector to be spliced; wherein the first-scale decoding module includes a linear attention mechanism unit, the second-scale decoding module includes a first depth-separable convolution unit, a preset convolution unit, and the linear attention mechanism unit, and the third-scale decoding module includes a second depth-separable convolution unit, a preset convolution unit, and the linear attention mechanism unit; the first decoding vector to be spliced, the second decoding vector to be spliced, and the third decoding vector to be spliced are spliced and input into the fully connected layer to obtain a first decoding vector.

[0143] Based on the above embodiment, optionally, the edge area determination module 330 includes:

[0144] An image expansion and corrosion unit is used to perform expansion processing and corrosion processing on the current lesion segmentation area to obtain an expanded image of the lesion area and an eroded image of the lesion area;

[0145] The edge region determining unit is configured to determine an edge contour region corresponding to the current lesion segmentation region based on the lesion region expansion image and the lesion region erosion image.

[0146] Based on the above embodiment, optionally, the loss value determination module 340 includes:

[0147] an edge contour loss value determining unit, configured to determine an edge contour loss value based on pixel values of an edge contour region of each lesion segmentation region, pixel values corresponding to the edge contour region in the lesion region annotation information, and a first loss function;

[0148] A non-edge contour loss value determination unit is used to determine, for each of the lesion segmentation areas, a non-edge contour area relative to the current lesion segmentation area based on the edge contour area; and to determine the non-edge contour loss value based on the pixel value of the non-edge contour area of each lesion segmentation area, the pixel value corresponding to the non-edge contour area in the lesion area annotation information, and a second loss function.

[0149] Based on the above embodiment, optionally, the model training module 350 includes:

[0150] a comprehensive loss value determining unit, configured to determine a comprehensive loss value based on a first preset weight, a second preset weight, the edge contour loss value, and the non-edge contour loss value;

[0151] The model training unit is used to back-propagate the comprehensive loss value to the multi-scale image segmentation model, adjust the network parameters in the multi-scale image segmentation model, and terminate the training when the preset conditions are met to obtain the target multi-scale image segmentation model.

[0152] Based on the above embodiment, optionally, the ultrasound image segmentation model training device also includes a model application module; the model application module is used to obtain a target ultrasound image corresponding to a preset detection site; the target ultrasound image is input into the target multi-scale image segmentation model after training, and the segmentation result corresponding to the suspected lesion site in the target ultrasound image is determined according to the output of the target multi-scale image segmentation model.

[0153] The technical solution of an embodiment of the present invention is to obtain a plurality of ultrasound sample images to be applied including lesion area annotation information, input the ultrasound sample images to be applied into a multi-scale image segmentation model to be trained for processing, and obtain a predicted lesion segmentation map including at least one lesion segmentation area; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied, and then, for at least one lesion segmentation area in the predicted lesion segmentation map, edge extraction processing is performed on the lesion segmentation area to obtain an edge contour area corresponding to each lesion segmentation area, and further, based on the lesion area annotation information and the edge contour area corresponding to at least one lesion segmentation area, an edge contour loss value and a non-edge contour loss value are determined, thereby, based on the edge contour loss value and the non-edge contour loss value, the multi-scale image segmentation model is trained to obtain a target multi-scale image segmentation model. The technical solution provided in this embodiment adopts a multi-scale image segmentation model, which can integrate feature information of different scales in the ultrasound sample image, increase the receptive field while paying attention to local information, make full use of feature information, and improve the segmentation accuracy of the image segmentation model; and train the model by predicting the edge contour loss value and non-edge contour loss value of each lesion segmentation area in the lesion segmentation map, so that the model pays more attention to the key information in the sample image, further improving the segmentation accuracy of the image segmentation model.

[0154] The ultrasound image segmentation model training device provided in the embodiment of the present invention can execute the ultrasound image segmentation model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0155] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the embodiments of the present invention.

[0156] Example 4

[0157] Figure 8 The present invention provides a schematic structural diagram of an electronic device. Figure 8 A block diagram of an exemplary electronic device 40 suitable for implementing exemplary embodiments of the present invention is shown. Figure 8 The electronic device 40 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.

[0158] like Figure 8 As shown, electronic device 40 is a general-purpose computing device. Components of electronic device 40 may include, but are not limited to, one or more processors or processing units 401, system memory 402, and a bus 403 connecting various system components (including system memory 402 and processing unit 401).

[0159] Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0160] The electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 40, including volatile and non-volatile media, removable and non-removable media.

[0161] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 Not shown, often called a "hard drive"). Although Figure 8Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data medium interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0162] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402. Such program modules 407 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 407 generally perform the functions and / or methods of the embodiments described herein.

[0163] The electronic device 40 may also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 40, and / or any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 411. Furthermore, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 412. As shown, the network adapter 412 communicates with other modules of the electronic device 40 via the bus 403. It should be understood that although Figure 8 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0164] The processing unit 401 executes various functional applications and page processing by running the programs stored in the system memory 402, such as implementing the ultrasound image segmentation model training method provided in the embodiment of the present invention.

[0165] Example 5

[0166] An embodiment of the present invention further provides a storage medium containing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform an ultrasound image segmentation model training method, the method comprising:

[0167] Acquire an ultrasound image sample set; wherein the ultrasound image sample set includes at least one ultrasound sample image to be used that includes lesion area annotation information;

[0168] Inputting the ultrasound sample image to be applied into a multi-scale image segmentation model to be trained for processing to obtain a predicted lesion segmentation map including at least one lesion segmentation region; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied;

[0169] For the at least one lesion segmentation region in the predicted lesion segmentation map, performing edge extraction processing on the lesion segmentation region to obtain an edge contour region corresponding to each lesion segmentation region;

[0170] Determining an edge contour loss value and a non-edge contour loss value based on the lesion region labeling information and the edge contour region corresponding to the at least one lesion segmentation region;

[0171] The multi-scale image segmentation model is trained based on the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model.

[0172] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0173] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0174] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0175] The computer program code for performing the operations of the embodiments of the present invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0176] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for training an ultrasound image segmentation model, characterized in that: include: Acquire an ultrasound image sample set; wherein the ultrasound image sample set includes at least one ultrasound sample image to be used that includes lesion area annotation information; Inputting the ultrasound sample image to be applied into a multi-scale image segmentation model to be trained for processing to obtain a predicted lesion segmentation map including at least one lesion segmentation region; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied; For the at least one lesion segmentation region in the predicted lesion segmentation map, performing edge extraction processing on the lesion segmentation region to obtain an edge contour region corresponding to each lesion segmentation region; Determining an edge contour loss value and a non-edge contour loss value based on the lesion region labeling information and the edge contour region corresponding to the at least one lesion segmentation region; The multi-scale image segmentation model is trained based on the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model.

2. The method according to claim 1, characterized in that The method further includes: constructing an ultrasound image sample set, including: acquiring at least one original ultrasound image including a lesion area; For the at least one original ultrasound image, normalize the current original ultrasound image to obtain an ultrasound image to be used; performing image enhancement processing on the ultrasound image to be used based on a preset image enhancement algorithm to obtain at least one ultrasound sample image to be applied; Performing lesion region labeling processing on the at least one ultrasound sample image to be applied based on a preset labeling tool to obtain lesion region labeling information corresponding to each ultrasound image to be applied; An ultrasound image sample set is constructed based on each of the ultrasound sample images to be applied and the corresponding lesion area annotation information.

3. The method according to claim 2, characterized in that The normalizing process of the current original ultrasound image to obtain the ultrasound image to be used includes: Retrieve a predetermined first pixel mean and a first pixel variance; wherein the first pixel mean and the first pixel variance is determined based on all pixel values of the region of interest in all the original ultrasound images; For all pixel points in the current original ultrasound image, pixel values of the pixel points are processed based on the first pixel mean and the first pixel variance to obtain the ultrasound image to be used.

4. The method according to claim 1, wherein The step of inputting the ultrasound sample image to be applied into a multi-scale image segmentation model to be trained for processing to obtain a predicted lesion segmentation map including at least one lesion segmentation region comprises: Inputting the ultrasound sample image to be applied into an encoder in the multi-scale image segmentation model to obtain a first scale feature map, a second scale feature map, a third scale feature map, and a fourth scale feature map corresponding to the ultrasound sample image to be applied; Inputting the fourth-scale feature map into a first multi-scale linear attention module in a decoder of the multi-scale image segmentation model for multi-scale decoding to obtain a first decoding vector; performing upsampling processing on the first decoded vector to obtain a first decoded vector to be processed, determining a first fused feature map based on the first decoded vector to be processed and the third-scale feature map, and performing multi-scale decoding processing on the first fused feature map based on a second multi-scale linear attention module in the decoder to obtain a second decoded vector; performing upsampling processing on the second decoded vector to obtain a second decoded vector to be processed, determining a second fused feature map based on the second decoded vector to be processed and the second scale feature map, and performing multi-scale decoding processing on the second fused feature map based on a third multi-scale linear attention module in the decoder to obtain a third decoded vector; performing upsampling processing on the third decoded vector to obtain a third decoded vector to be processed, determining a third fused feature map based on the third decoded vector to be processed and the first scale feature map, and performing multi-scale decoding processing on the third fused feature map based on a fourth multi-scale linear attention module in the decoder to obtain a target decoded vector; Based on the target decoding vector, a predicted lesion segmentation map including at least one lesion segmentation region is obtained.

5. The method according to claim 4, characterized in that The decoder includes a feature extraction module, a first-scale decoding module, a second-scale decoding module, a third-scale decoding module, and a fully connected layer. The fourth-scale feature map is input into the first multi-scale linear attention module in the decoder of the multi-scale image segmentation model for multi-scale decoding processing to obtain a first decoding vector, including: Inputting the first-scale feature map into the feature extraction module of the first multi-scale linear attention module to obtain decoding features to be processed; The decoding features to be processed are respectively input into the first scale decoding module, the second scale decoding module and the third scale decoding module to obtain the first decoding vector to be spliced, the second decoding vector to be spliced and a third decoding vector to be concatenated; wherein the first-scale decoding module includes a linear attention mechanism unit, the second-scale decoding module includes a first depth-separable convolution unit, a preset convolution unit, and the linear attention mechanism unit, and the third-scale decoding module includes a second depth-separable convolution unit, a preset convolution unit, and the linear attention mechanism unit; The first decoding vector to be spliced, the second decoding vector to be spliced, and the third decoding vector to be spliced are concatenated and input into a fully connected layer to obtain a first decoding vector.

6. The method according to claim 1, characterized in that The performing edge extraction processing on the lesion segmentation region to obtain an edge contour region corresponding to each lesion segmentation region includes: Perform expansion processing and corrosion processing on the current lesion segmentation area respectively to obtain the expansion image of the lesion area and the corrosion image of the lesion area; Based on the lesion area expansion image and the lesion area erosion image, an edge contour area corresponding to the current lesion segmentation area is determined.

7. The method according to claim 1, characterized in that The determining of the edge contour loss value and the non-edge contour loss value based on the lesion region labeling information and the edge contour region corresponding to the at least one lesion segmentation region includes: Determining an edge contour loss value based on pixel values of an edge contour area of each lesion segmentation area, pixel values corresponding to the edge contour area in the lesion area annotation information, and a first loss function; For each of the lesion segmentation regions, determining a non-edge contour region of the current lesion segmentation region based on the edge contour region; A non-edge contour loss value is determined based on the pixel values of the non-edge contour area of each lesion segmentation area, the pixel values corresponding to the non-edge contour area in the lesion area annotation information, and a second loss function.

8. The method according to claim 1, characterized in that The multi-scale image segmentation model is trained based on the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model, including: Determining a comprehensive loss value based on the first preset weight, the second preset weight, the edge contour loss value, and the non-edge contour loss value; The comprehensive loss value is back-propagated into the multi-scale image segmentation model, and the network parameters in the multi-scale image segmentation model are adjusted until the training ends when the preset conditions are met, thereby obtaining the target multi-scale image segmentation model.

9. The method according to claim 1, characterized in that After obtaining the target multi-scale image segmentation model, the method further includes: Acquire a target ultrasound image corresponding to a preset detection site; The target ultrasound image is input into the trained target multi-scale image segmentation model, and a segmentation result corresponding to the suspected lesion site in the target ultrasound image is determined according to the output of the target multi-scale image segmentation model.

10. An ultrasound image segmentation model training device, characterized in that: include: A sample set acquisition module is used to acquire an ultrasound image sample set; wherein the ultrasound image sample set includes at least one ultrasound sample image to be used that contains lesion area annotation information; a predicted lesion segmentation module, configured to input the ultrasound sample image to be applied into a multi-scale image segmentation model to be trained for processing, thereby obtaining a predicted lesion segmentation map including at least one lesion segmentation region; wherein the multi-scale image segmentation model includes an encoder and a decoder that performs multi-scale decoding processing on at least one feature map output by the encoder and corresponding to the ultrasound sample image to be applied; an edge region determination module, configured to perform edge extraction processing on the at least one lesion segmentation region in the predicted lesion segmentation map to obtain an edge contour region corresponding to each lesion segmentation region; A loss value determination module, configured to determine an edge contour loss value and a non-edge contour loss value based on the lesion region labeling information and an edge contour region corresponding to the at least one lesion segmentation region; The model training module is used to train the multi-scale image segmentation model based on the edge contour loss value and the non-edge contour loss value to obtain a target multi-scale image segmentation model.

Images