Ultrasonic image ovarian lesion segmentation method, system, device and medium

Abstract

The present application discloses a method, system, device and medium for segmenting ovarian lesions in ultrasound images. The method includes inputting an ultrasound image into a coding module, outputting a multi-scale coding feature map through the coding module; determining a minimum-scale decoding feature map based on the minimum-scale coding feature map and the second-small-scale coding feature map through a pseudo-feature decoding block, and determining a fusion feature map based on the minimum-scale decoding feature map and the second-small-scale coding feature map; determining an ovarian lesion segmentation map through a decoding module based on the fusion feature map and coding feature maps of other scales except the minimum-scale coding feature map. The fusion feature map in the present application combines coding features and decoding features. Through the fusion feature map, the correlation between the features inside and outside the lesion can be learned, and the incorrect features learned by the ovarian lesion segmentation model can be corrected in time, the problems of over-segmentation and under-segmentation can be alleviated, and the accuracy of ovarian lesion segmentation can be improved.

Description

Technical Field

[0001] The present application relates to the field of ultrasound technology, and in particular to a method, system, device and medium for segmenting ovarian lesions in ultrasound images. Background Art

[0002] Ultrasound examination is one of the preferred medical imaging technologies for ovarian examination. However, in ultrasound images, the contour boundary of ovarian lesions is difficult to identify due to the similar grayscale between the ovarian lesions and the surrounding tissues, and the highly similar density values ​​can easily confuse the lesions and surrounding tissues, resulting in missed diagnosis. This places high demands on the clinical experience of ultrasound doctors and brings great clinical pressure to doctors.

[0003] In recent years, with the rapid development of deep learning technology, deep learning technology has gradually been used in the research of ovarian detection in ultrasound images. However, the existing methods of using deep learning technology to perform ovarian detection only focus on the features within the lesion area, while ignoring the correlation between the features within and outside the lesion, which affects the accuracy of ovarian lesion detection.

[0004] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0005] The technical problem to be solved by the present application is to provide a method, system, device and medium for segmenting ovarian lesions in ultrasound images in view of the deficiencies in the prior art.

[0006] In order to solve the above technical problems, the first aspect of the present application provides an ultrasound image ovarian lesion segmentation method, which uses a trained ovarian lesion segmentation model, wherein the ovarian lesion segmentation model includes an encoding module, a pseudo feature decoding block and a decoding module; the ultrasound image ovarian lesion segmentation method specifically includes:

[0007] Inputting an ultrasound image having an ovarian region into a coding module, and outputting a multi-scale coding feature map through the coding module;

[0008] Determine a minimum-scale decoding feature map based on the minimum-scale encoding feature map and the second small-scale encoding feature map by the pseudo-feature decoding block, and determine a fusion feature map based on the minimum-scale decoding feature map and the second small-scale encoding feature map;

[0009] The decoding module determines the ovarian lesion segmentation map corresponding to the ultrasound image based on the fused feature map and the coded feature maps of other scales except the coded feature map of the minimum scale.

[0010] The method for segmenting ovarian lesions in an ultrasonic image, wherein the step of determining a fusion feature map based on the minimum-scale decoding feature map and the second minimum-scale encoding feature map specifically comprises:

[0011] Determine a false positive feature map and a false negative feature map based on the minimum-scale decoding feature map and the second-small-scale encoding feature map;

[0012] Performing self-attention learning on the pseudo-positive feature map to obtain a self-attention feature, and interactively learning the pseudo-positive feature map and the pseudo-negative feature map to obtain a cross-attention feature;

[0013] A fused feature map is determined based on the self-attention feature, the cross-attention feature and the minimum-scale decoding feature map.

[0014] The method for segmenting ovarian lesions in an ultrasonic image, wherein the determining of the false positive feature map and the false negative feature map based on the minimum-scale decoding feature map and the second-small-scale encoding feature map specifically comprises:

[0015] Performing a convolution operation on the minimum-scale decoding feature map to obtain a convolution feature map;

[0016] Performing a first activation operation and a second activation operation on the convolution feature map respectively to obtain a first activation feature map and a second activation feature map, wherein the first activation operation is used to predict the area inside the pseudo lesion, and the second activation operation is used to predict the area outside the pseudo lesion;

[0017] Determine a false positive feature map based on the first activation feature map and the second small-scale encoding feature map;

[0018] A pseudo negative feature map is determined based on the second activation feature map and the second small-scale encoding feature map.

[0019] The method for segmenting ovarian lesions in ultrasonic images, wherein the self-attention learning of the pseudo-positive feature map to obtain the self-attention feature, and the interactive learning of the pseudo-positive feature map and the pseudo-negative feature map to obtain the cross-attention feature specifically includes:

[0020] constructing a first key vector, a query vector, and a value vector based on the false positive feature map, and constructing a second key vector based on the false negative feature map;

[0021] Performing self-attention learning on the first key vector, the query vector, and the value vector to obtain a self-attention feature;

[0022] The second key vector, query vector and value vector are interactively learned to obtain a cross-attention feature.

[0023] The method for segmenting ovarian lesions in ultrasonic images, wherein the ovarian lesion segmentation model further includes a Radon projection module, the Radon projection module includes a latent feature projection unit, a feature extraction unit and a back projection unit; after the multi-scale coded feature map is output by the encoding module, the method further includes:

[0024] The minimum-scale encoded feature map is subjected to Radon transformation at a preset angle by the latent feature projection unit to obtain a transformed feature map;

[0025] Extracting a feature map of the transformation feature from a channel dimension and an angle dimension by the feature extraction unit to obtain a channel feature map and an angle feature map;

[0026] The channel feature map and the angle feature map are subjected to inverse Radon transform by the inverse projection unit to output a spatial feature map, and the spatial feature map is used as the minimum-scale encoding feature map.

[0027] The method for segmenting ovarian lesions in ultrasonic images, wherein the feature extraction unit extracts the feature map of the transformation feature from the channel dimension and the angle dimension to obtain the channel feature map and the angle feature map specifically includes:

[0028] Splitting the transformed feature map into a plurality of first feature maps according to a channel dimension and into a plurality of second feature maps according to an angle dimension by the feature extraction unit;

[0029] Inputting a plurality of first feature maps into a Transformer network, and determining a channel feature map through the Transformer network;

[0030] Several second feature maps are input into the Transformer network, and the angle feature map is determined by the Transformer network.

[0031] The method for segmenting ovarian lesions in ultrasonic images, wherein the step of performing a reverse Radon transform on the channel feature map and the angle feature map by the reverse projection unit to output a spatial feature map specifically includes:

[0032] Performing reverse Radon transformation on the channel feature map and the angle feature map respectively through the reverse projection unit to obtain a reverse channel feature map and a reverse angle feature map;

[0033] The reverse channel feature map, the reverse angle feature map and the minimum scale encoding feature map are spliced ​​by the reverse projection unit to obtain a spatial feature map.

[0034] The second aspect of the present application provides an ultrasound image ovarian lesion segmentation system, which uses a trained ovarian lesion segmentation model, wherein the ovarian lesion segmentation model includes an encoding module, a pseudo feature decoding block and a decoding module; the ultrasound image ovarian lesion segmentation system specifically includes:

[0035] An acquisition module, used for acquiring an ultrasound image including an ovarian region;

[0036] A control module is used to input an ultrasound image having an ovarian region into a coding module, and output a multi-scale coding feature map through the coding module; determine a minimum-scale decoding feature map based on the minimum-scale coding feature map and the second-small-scale coding feature map through the pseudo-feature decoding block, and determine a fusion feature map based on the minimum-scale decoding feature map and the second-small-scale coding feature map; determine an ovarian lesion segmentation map corresponding to the ultrasound image through the decoding module based on the fusion feature map and coding feature maps of other scales except the minimum-scale coding feature map.

[0037] A third aspect of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the above-mentioned methods for segmenting ovarian lesions in ultrasound images.

[0038] A fourth aspect of the present application provides a terminal device, comprising: a processor and a memory;

[0039] The memory stores a computer-readable program executable by the processor;

[0040] When the processor executes the computer-readable program, the steps in any of the above-mentioned methods for segmenting ovarian lesions in ultrasound images are implemented.

[0041] Beneficial effects: Compared with the prior art, the present application provides an ultrasound image ovarian lesion segmentation method, system, device and medium, the method comprising inputting an ultrasound image with an ovarian region into a coding module, outputting a multi-scale coding feature map through the coding module; determining a minimum-scale decoding feature map based on the minimum-scale coding feature map and the second-small-scale coding feature map through the pseudo-feature decoding module, and determining a fusion feature map based on the minimum-scale decoding feature map and the second-small-scale coding feature map; determining an ovarian lesion segmentation map corresponding to the ultrasound image through the decoding module based on the fusion feature map and coding feature maps of other scales except the minimum-scale coding feature map. The fusion feature map in the present application fuses the coding feature map and the minimum-scale decoding feature map. The fusion feature map can be used to learn the correlation between the features inside and outside the lesion, and the incorrect features learned by the ovarian lesion segmentation model can be corrected in time, which can alleviate the problems of over-segmentation and under-segmentation, thereby improving the accuracy of ovarian lesion segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 This is the structural block diagram of the ovarian lesion segmentation model.

[0044] Figure 2 A flow chart of a method for segmenting ovarian lesions in ultrasound images provided in an embodiment of the present application.

[0045] Figure 3 This is the structural block diagram of the Radon projection module.

[0046] Figure 4 This is the structural block diagram of the pseudo decoding module.

[0047] Figure 5 This is a principle block diagram of the ultrasound image ovarian lesion segmentation system provided in an embodiment of the present application.

[0048] Figure 6 A functional block diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The embodiments of the present application provide a method, system, device and medium for segmenting ovarian lesions in ultrasound images. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0051] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0052] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0053] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0054] This embodiment provides an ultrasound image ovarian lesion segmentation method, using a trained ovarian lesion segmentation model, such as Figure 1As shown, the ovarian lesion segmentation model includes a coding module, a pseudo-feature decoding module and a decoding module, and the decoding block formed by the coding module, the pseudo-feature decoding module and the decoding module constitutes a U-shaped network. The coding module includes a plurality of cascaded coding units, and the decoding module includes a plurality of cascaded decoding units. The last two coding units in the cascade order are connected to the pseudo-feature decoding block, and each coding unit except the last coding unit is jump-connected to the corresponding decoding unit in the decoding module, and the pseudo-feature decoding module is connected to the front decoding unit. Among them, the coding module is used to extract features from the ultrasound image to obtain a multi-scale coding feature map, the pseudo-feature decoding module is used to determine the minimum scale decoding feature map based on the minimum scale coding feature map and the second smallest scale coding feature map, and determine the fusion feature map based on the minimum scale decoding feature map and the second smallest scale coding feature map, the decoding module is used to determine the final minimum scale decoding feature map based on the fusion feature map and the coding feature maps of other scales except the minimum scale coding feature map, and determine the ovarian lesion segmentation result based on the final minimum scale decoding feature map. In the embodiment of the present application, the pseudo-feature decoding module is used as a decoding layer in the U-shaped structure. The decoding features and the encoding features are fused through the pseudo-feature decoding module to jointly learn the features inside and outside the lesion, alleviate the problems of over-segmentation and under-segmentation, and improve the accuracy of ovarian lesion segmentation.

[0055] like Figure 2 As shown, the ultrasound image ovarian lesion segmentation method provided in the embodiment of the present application specifically includes:

[0056] S10, inputting the ultrasound image with the ovarian region into a coding module, and outputting a multi-scale coding feature map through the coding module.

[0057] Specifically, the ultrasound image may be a real-time ultrasound image collected in real time by an ultrasound device, an offline ultrasound image stored offline, or a remote ultrasound image sent by other external devices, etc. The ultrasound image includes the ovarian region, that is, the ultrasound image is obtained by performing an ovarian ultrasound examination on the patient, that is, before the ultrasound image including the ovarian region is input into the encoding module, the ultrasound image including the ovarian region is first obtained, and the obtained ultrasound image including the ovarian region may also be pre-processed, for example, clarity detection, ovarian region detection, noise removal, etc.

[0058] After obtaining the ultrasound image containing the ovarian region, the ultrasound image containing the ovarian region is input into the encoding module, and the encoding module is used to extract features of the ultrasound image containing the ovarian region, wherein the encoding module includes a plurality of encoding units, the plurality of encoding units are cascaded in sequence, and each encoding unit extracts a coded feature map of a certain scale. For example, assuming that the image size of the ultrasound image is , Indicates the image height, Indicates the image width, such as Figure 1 The coding module shown includes five coding units. The image scales extracted by each coding unit in the cascade order are (48, 1), (96, 1 / 2), (192, 1 / 4), (384, 1 / 8), and (768, 1 / 16), where: Indicates that the image width of the coded feature map is the image width of the ultrasound image , the image height of the encoded feature map is the image height of the ultrasound image (For example, 1 / 2 means ).

[0059] It should be noted that, in practical applications, the number of coding units included in the coding module and the image scale of the coding feature map extracted by each coding unit can be set according to actual needs, and no specific restrictions are made here.

[0060] In one implementation, after obtaining the multi-scale coding feature map, in order to increase the knowledge information contained in the coding feature map, the minimum-scale coding feature map can be subjected to Radon transformation to learn spatial information and supplement more information. Based on this, the ovarian lesion segmentation model also includes a Radon projection module, which is located between the coding module and the pseudo-feature decoding module; the Radon projection module includes a hidden feature projection unit, a feature extraction unit, and a reverse projection unit; the hidden feature projection unit is connected to the feature extraction unit, which is connected to the reverse projection unit, and the hidden feature projection unit is used to perform Radon transformation to obtain a transformation feature map; the feature extraction unit is used to extract features from the channel dimension and the angle dimension of the transformation feature map to obtain a channel feature map and an angle feature map; the reverse projection unit is used to perform reverse Radon transformation on the channel feature map and the angle feature map to obtain a spatial feature map.

[0061] Based on this, after the encoding module outputs the multi-scale encoding feature map, the method further includes:

[0062] S101, performing a Radon transform on the minimum-scale encoding feature map at a preset angle by the latent feature projection unit to obtain a transformed feature map;

[0063] S102, extracting a feature map of the transformation feature from a channel dimension and an angle dimension by the feature extraction unit to obtain a channel feature map and an angle feature map;

[0064] S102. Perform a reverse Radon transform on the channel feature map and the angle feature map through the reverse projection unit to output a spatial feature map, and use the spatial feature map as the minimum-scale encoding feature map.

[0065] In step S101, the transformed feature map is obtained by projecting the minimum-scale coded feature map at a preset angle. That is to say, compared with the existing Radon transform at the image level, the embodiment of the present application implements the Radon transform at the feature level, so that the expression form of the feature can be intervened in the feature learning process, and more fine-grained features can be extracted for learning, thereby improving the accuracy of the ovarian lesion segmentation results. At the same time, after the present application implements the Radon transform at the feature level, it will also perform a reverse Radon transform on the learned features in the subsequent process, and back-project the features to the spatial dimension of the same size as the minimum-scale coded feature map.

[0066] Furthermore, the preset angle is pre-set, and there may be multiple angles, for example, 0°, 45°, 90°, 135°, 180°, etc. Correspondingly, the transformation feature map can be obtained by accumulating the projections obtained by performing a Radon transform on each preset angle, that is, the minimum-scale coding feature map can be subjected to a Radon transform at each preset angle to obtain the projection corresponding to each preset angle, and then all projections are accumulated to obtain the transformation feature map. Among them, the image width of the transformation feature map is the same as the image width of the minimum-scale coding feature map, the number of channels of the transformation feature map is the same as the number of channels of the minimum-scale coding feature map, and the image height of the transformation feature map is equal to the number of preset angles. For example, Figure 3 As shown, the number of channels of the smallest scale encoding feature map is , the image width of the smallest scale encoding feature map is , the preset number of angles is , then the image scale of the transformed feature map is .

[0067] Further, in step S102, after the transformation feature map is obtained, feature extraction can be performed on the transformation feature map to learn spatial feature information. The feature extraction unit extracts features from the channel dimension and the angle dimension of the transformation feature map to obtain the channel feature map and the angle feature map, which specifically includes:

[0068] Splitting the transformed feature map into a plurality of first feature maps according to the channel dimension and into a plurality of second feature maps according to the angle dimension by the feature extraction unit;

[0069] Inputting a plurality of first feature maps into a Transformer network, and determining a channel feature map through the Transformer network;

[0070] Several second feature maps are input into the Transformer network, and the angle feature map is determined by the Transformer network.

[0071] Specifically, the number of the first feature maps is equal to the number of channels of the transformation feature map, and the number of the second feature maps is equal to the preset number of angles corresponding to the transformation feature map. After obtaining the first feature maps and the second feature maps, the first feature maps and the second feature maps are respectively input into the Transformer network, and the first feature maps are feature extracted through the Transformer network to obtain channel feature maps, and the second feature maps are feature extracted to obtain angle feature maps. The image scales of the channel feature maps and the angle feature maps are the same as the image scale of the transformation feature maps. The Transformer network includes multi-head attention, layer normalization, multi-layer perceptron and layer normalization cascaded in sequence, and the input items of the multi-head attention are fused with the output items of the multi-head attention through an adder as the input items of the layer normalization connected to the multi-head attention, and the input items of the multi-layer perceptron are fused with the output items of the multi-layer perceptron through an adder as the input items of the layer normalization connected to the multi-layer perceptron.

[0072] Further, in step S103, after obtaining the channel feature map and the angle feature map, the channel feature map and the angle feature map can be respectively subjected to reverse Radon transformation by a reverse projection unit to restore the channel feature map and the angle feature map to the feature space of the minimum scale encoding feature map. Exemplarily, the reverse Radon transformation of the channel feature map and the angle feature map by the reverse projection unit to output the spatial feature map specifically includes:

[0073] Performing reverse Radon transformation on the channel feature map and the angle feature map respectively through the reverse projection unit to obtain a reverse channel feature map and a reverse angle feature map;

[0074] The reverse channel feature map, the reverse angle feature map and the minimum scale encoding feature map are cascaded through the reverse projection unit to obtain a spatial feature map.

[0075] Specifically, if Figure 3As shown, the reverse projection unit is used to perform reverse Radon transform on the channel feature map and the angle feature map to transform the channel feature map into a reverse channel feature map, and transform the angle feature map into a reverse angle feature map, wherein the preset angle for implementing the reverse Radon transform is the same as the preset angle for implementing the Radon transform. After obtaining the reverse channel feature map and the reverse angle feature map, the reverse channel feature map, the reverse angle feature map and the minimum-scale encoding feature map are cascaded in the channel dimension, and a convolution operation is performed on the cascaded feature map to obtain a spatial feature map. The image scale of the spatial feature map is the same as the image scale of the minimum-scale encoding feature map.

[0076] S20. Determine a minimum-scale decoding feature map based on the minimum-scale encoding feature map and the second small-scale encoding feature map through the pseudo-feature decoding block, and determine a fusion feature map based on the minimum-scale decoding feature map and the second small-scale encoding feature map.

[0077] Specifically, the pseudo-feature decoding block first determines the minimum-scale decoding feature map, and then determines the fusion feature map based on the minimum-scale decoding feature map and the second small-scale encoding feature map. Correspondingly, the pseudo-decoding module includes a decoding unit and an interactive learning unit, the decoding unit is used to obtain the minimum-scale decoding feature map, the input item of the decoding unit includes the minimum-scale encoding feature map and the second small-scale encoding feature map, and the output item is the minimum-scale decoding feature map. Among them, the decoding unit includes an upsampling layer, a fusion layer and a convolution layer, the upsampling layer is connected to the fusion layer, the input item of the upsampling layer is the minimum-scale encoding feature map, the output item is the upsampling feature map, and the image scale of the upsampling feature map is the same as the image scale of the second small-scale encoding feature map; the input item of the fusion layer includes the second small-scale encoding feature map and the upsampling feature map, the output item is the intermediate feature map, the input item of the convolution layer is the intermediate feature map, and the output item is the minimum-scale decoding feature map, and the image scale of the minimum-scale decoding feature map is the same as the image scale of the second small-scale encoding feature map.

[0078] The interactive learning unit is used to determine a fused feature map based on a minimum-scale decoding feature map and a second-small-scale encoding feature map, that is, the minimum-scale decoding feature map and the second-small-scale encoding feature map can be input into the interactive learning unit, and the minimum-scale decoding feature map and the second-small-scale encoding feature map are learned by the interactive learning unit to output a fused feature map. Among them, the interactive learning unit may include an initial prediction block, an interactive learning block, and a fusion block cascaded in sequence; the initial prediction block is used to predict a pseudo-positive feature map and a pseudo-negative feature map based on the minimum-scale decoding feature map, the interactive learning block is used to self-attentionally learn and interactively learn the pseudo-positive feature map and the pseudo-negative feature map to obtain self-attention features and cross-attention features, and the fusion block is used to fuse the self-attention features, cross-attention features, and the minimum-scale decoding feature map.

[0079] Based on this, the specific process of determining the fused feature map based on the minimum-scale decoding feature map and the second small-scale encoding feature map may include:

[0080] S21, determining a false positive feature map and a false negative feature map based on the minimum-scale decoding feature map and the second-small-scale encoding feature map;

[0081] S22, performing self-attention learning on the pseudo-positive feature map to obtain a self-attention feature, and interactively learning the pseudo-positive feature map and the pseudo-negative feature map to obtain a cross-attention feature;

[0082] S23. Determine a fused feature map based on the self-attention feature, the cross-attention feature and the minimum-scale decoding feature map.

[0083] Specifically, in step S21, the false positive feature map includes feature information of the false lesion area, and the false negative feature map includes feature information outside the false lesion area. Wherein, both the false positive feature map and the false negative feature map are determined by the initial prediction block, such as Figure 4 As shown, the initial prediction block may include a first prediction branch and a second prediction branch, the first prediction branch includes a convolution layer, a first activation layer, a channel replication layer and a multiplier, the second prediction branch includes a convolution layer, a second activation layer, a channel replication layer and a multiplier, the minimum-scale decoding feature map passes through the convolution layer, the first activation layer and the channel replication layer in sequence, and then is multiplied pixel by pixel with the second small-scale encoding feature map through the multiplier to obtain a pseudo-positive feature map, the minimum-scale decoding feature map passes through the convolution layer, the second activation layer and the channel replication layer in sequence, and then is multiplied pixel by pixel with the second small-scale encoding feature map through the multiplier to obtain a pseudo-negative feature map.

[0084] Based on this, the determining of the false positive feature map and the false negative feature map based on the minimum-scale decoding feature map and the second small-scale encoding feature map specifically includes:

[0085] Performing a convolution operation on the minimum-scale decoding feature map to obtain a convolution feature map;

[0086] Performing a first activation operation and a second activation operation on the convolution feature map respectively to obtain a first activation feature map and a second activation feature map, wherein the first activation operation is used to predict the area inside the pseudo lesion, and the second activation operation is used to predict the area outside the pseudo lesion;

[0087] Determine a false positive feature map based on the first activation feature map and the second small-scale encoding feature map;

[0088] A pseudo negative feature map is determined based on the second activation feature map and the second small-scale encoding feature map.

[0089] Specifically, the convolution operation is performed through the convolution layer, the first activation operation is performed through the first activation layer, the second activation operation is performed through the second activation layer, the first activation operation is used to predict the pseudo lesion area, and the second activation operation is used to predict the pseudo lesion area, that is, the lesion area is predicted by performing the first activation operation on the minimum scale decoding feature map to obtain the first activation feature map containing the pseudo lesion area, and the second activation operation is performed on the minimum scale decoding feature map to predict the lesion area to obtain the second activation feature map containing the pseudo lesion area, and then the first activation feature map and the second activation feature map are channel-copied and multiplied pixel by pixel with the second minimum scale encoding feature, respectively, to obtain a pseudo positive feature map for reflecting the pseudo lesion internal feature and a pseudo negative feature map for reflecting the lesion external feature. The first activation operation uses a sigmoid activation function with a threshold of 0.5, and selects an area with a probability value greater than or equal to 0.5, and the second activation operation uses a sigmoid activation function with a threshold of 0.5, and selects an area with a probability value less than 0.5. Of course, in practical applications, the activation function and threshold used in the first activation operation and the second activation operation can be set according to actual needs. This is just an example without any specific limitation.

[0090] Further, in step S22, after obtaining the false positive feature map and the false negative feature map, self-attention learning is performed on the false positive feature map, and interactive learning is performed between the false positive feature map and the false negative feature map, and the relationship between the false positive feature map and the false negative feature map is extracted to obtain the relationship between the features outside the lesion and the features inside the lesion. Among them, self-attention learning is performed on the false positive feature map through the interactive learning block, and interactive learning is performed on the false positive feature map and the false negative feature map, such as Figure 4As shown, the interactive learning block includes four convolutional layers, a self-attention layer and a cross-attention layer. The four convolutional layers are used to construct a first key vector, a query vector, a value vector and a second key vector. The self-attention layer is used for self-attention learning, and the cross-attention layer is used for interactive learning.

[0091] Based on this, the self-attention learning of the pseudo-positive feature map to obtain the self-attention feature, and the interactive learning of the pseudo-positive feature map and the pseudo-negative feature map to obtain the cross-attention feature specifically includes:

[0092] constructing a first key vector, a query vector, and a value vector based on the false positive feature map, and constructing a second key vector based on the false negative feature map;

[0093] Performing self-attention learning on the first key vector, the query vector, and the value vector to obtain a self-attention feature;

[0094] The second key vector, query vector and value vector are interactively learned to obtain a cross-attention feature.

[0095] Specifically, Figure 4 As shown, the first key vector, the query vector and the value vector are obtained by performing a convolution operation on the pseudo-positive feature map, wherein the convolution parameters of the convolution operations performed on the first key vector, the query vector and the value vector may be the same. The second key vector is obtained by performing a convolution operation on the pseudo-negative feature map, and the convolution parameters of the convolution operation corresponding to the second key vector may be the same as the convolution parameters of the convolution operation corresponding to the first key vector, or may be different from the convolution parameters of the convolution operation corresponding to the first key vector. Here, the convolution parameters of the convolution operations corresponding to the first key vector, the second key vector, the query vector and the value vector are not limited and can be set according to actual needs.

[0096] After obtaining the first key vector, query vector, value vector and second key vector, self-attention learning is performed on the first key vector, query vector and value vector to perform self-attention learning on the pseudo-positive feature map, and interactive learning is performed on the second key vector, query vector and value vector to perform interactive learning on the pseudo-negative feature map. In this way, both the features within the pseudo lesion and the relationship between the features within the pseudo lesion and the features outside the pseudo lesion can be learned.

[0097] Further, in step S23, after obtaining the self-attention feature and the cross-attention feature, as Figure 4 As shown, the self-attention features, cross-attention features and the minimum-scale decoding feature map can be cascaded according to the channels, and the cascaded features are convolved to obtain a fused feature map, so that the fused feature map combines the decoding features and the encoding features.

[0098] S30. Determine, by the decoding module, an ovarian lesion segmentation map corresponding to the ultrasound image based on the fused feature map and the coded feature maps of other scales except the coded feature map of the minimum scale.

[0099] Specifically, the fusion feature map is used to replace the smallest scale decoding feature map and the encoding feature map of other scales. Figure 1 The decoded features are input into the decoding module, and the features are gradually restored to the size of the ultrasound image through the decoding module. Then, the ovarian lesion segmentation map is determined based on the decoded feature map output by the last decoding unit in the decoding module. For example, the decoded features output by the last decoding module are input into the sigmoid activation function to obtain a probability map, and then the probability map is converted into the probability map by taking the threshold. Figure 2 The ovarian lesion segmentation map was obtained by quantization.

[0100] In summary, the present embodiment provides an ultrasound image ovarian lesion segmentation method, the method comprising inputting an ultrasound image having an ovarian region into a coding module, outputting a multi-scale coding feature map through the coding module; determining a minimum-scale decoding feature map based on the minimum-scale coding feature map and the second small-scale coding feature map through the pseudo-feature decoding block, and determining a fusion feature map based on the minimum-scale decoding feature map and the second small-scale coding feature map; determining an ovarian lesion segmentation map corresponding to the ultrasound image through the decoding module based on the fusion feature map and coding feature maps of other scales except the minimum-scale coding feature map. The present application constructs a fusion feature map by fusing the coding feature map and the minimum-scale decoding feature map, and learns the correlation between the features inside and outside the lesion through the fusion feature map, so as to timely analyze and correct the incorrect features learned by the ovarian lesion segmentation model, thereby alleviating the problems of over-segmentation and under-segmentation, and further improving the accuracy of ovarian lesion segmentation.

[0101] Based on the above-mentioned ultrasound image ovarian lesion segmentation method, this embodiment provides an ultrasound image ovarian lesion segmentation system, which uses a trained ovarian lesion segmentation model, wherein the ovarian lesion segmentation model includes an encoding module, a pseudo feature decoding block and a decoding module; Figure 5 As shown, the ultrasound image ovarian lesion segmentation system specifically includes:

[0102] An acquisition module 100 is used to acquire an ultrasound image including an ovarian region;

[0103] The control module 200 is used to input an ultrasound image having an ovarian region into a coding module, and output a multi-scale coding feature map through the coding module; determine a minimum-scale decoding feature map based on the minimum-scale coding feature map and the second-small-scale coding feature map through the pseudo-feature decoding block, and determine a fusion feature map based on the minimum-scale decoding feature map and the second-small-scale coding feature map; determine an ovarian lesion segmentation map corresponding to the ultrasound image through the decoding module based on the fusion feature map and coding feature maps of other scales except the minimum-scale coding feature map.

[0104] Based on the above-mentioned method for segmenting ovarian lesions in ultrasound images, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the method for segmenting ovarian lesions in ultrasound images as described in the above-mentioned embodiment.

[0105] Based on the above ultrasound image ovarian lesion segmentation method, the present application also provides a terminal device, such as Figure 6 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, the display screen 21, the memory 22, and the communications interface 23 may communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communications interface 23 may transmit information. The processor 20 may call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0106] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0107] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0108] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0109] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for segmenting ovarian lesions in ultrasound images, characterized in that: A trained ovarian lesion segmentation model is applied, wherein the ovarian lesion segmentation model includes an encoding module, a pseudo feature decoding block, a decoding module, and a Radon projection module; the ultrasound image ovarian lesion segmentation method specifically includes: Inputting an ultrasound image having an ovarian region into a coding module, and outputting a multi-scale coding feature map through the coding module; Determine a minimum-scale decoding feature map based on the minimum-scale encoding feature map and the second small-scale encoding feature map by the pseudo-feature decoding block; Determine a false positive feature map and a false negative feature map based on the minimum-scale decoding feature map and the second-small-scale encoding feature map; Performing self-attention learning on the pseudo-positive feature map to obtain a self-attention feature, and interactively learning the pseudo-positive feature map and the pseudo-negative feature map to obtain a cross-attention feature; Determining a fused feature map based on the self-attention feature, the cross-attention feature and the minimum-scale decoding feature map; Determine, by the decoding module, an ovarian lesion segmentation map corresponding to the ultrasound image based on the fused feature map and the coded feature maps of other scales except the coded feature map of the minimum scale; The Radon projection module includes a latent feature projection unit, a feature extraction unit and a back projection unit; after the encoding module outputs the multi-scale encoding feature map, the method further includes: The minimum-scale encoded feature map is subjected to Radon transformation at a preset angle by the latent feature projection unit to obtain a transformed feature map; Extracting a feature map of the transformation feature from a channel dimension and an angle dimension by the feature extraction unit to obtain a channel feature map and an angle feature map; The channel feature map and the angle feature map are subjected to inverse Radon transform by the inverse projection unit to output a spatial feature map, and the spatial feature map is used as the minimum-scale encoding feature map.

2. The method for segmenting ovarian lesions in ultrasonic images according to claim 1, characterized in that: The determining of the false positive feature map and the false negative feature map based on the minimum-scale decoding feature map and the second-small-scale encoding feature map specifically includes: Performing a convolution operation on the minimum-scale decoding feature map to obtain a convolution feature map; Performing a first activation operation and a second activation operation on the convolution feature map respectively to obtain a first activation feature map and a second activation feature map, wherein the first activation operation is used to predict the area inside the pseudo lesion, and the second activation operation is used to predict the area outside the pseudo lesion; Determining a false positive feature map based on the first activation feature map and the second small-scale encoding feature map; A pseudo negative feature map is determined based on the second activation feature map and the second small-scale encoding feature map.

3. The method for segmenting ovarian lesions in ultrasonic images according to claim 1, characterized in that: The performing self-attention learning on the pseudo-positive feature map to obtain a self-attention feature, and interactively learning the pseudo-positive feature map and the pseudo-negative feature map to obtain a cross-attention feature specifically includes: constructing a first key vector, a query vector, and a value vector based on the false positive feature map, and constructing a second key vector based on the false negative feature map; Performing self-attention learning on the first key vector, the query vector, and the value vector to obtain a self-attention feature; The second key vector, query vector and value vector are interactively learned to obtain a cross-attention feature.

4. The method for segmenting ovarian lesions in ultrasonic images according to claim 1, characterized in that: The extracting a feature map of the transformation feature from the channel dimension and the angle dimension by the feature extraction unit to obtain a channel feature map and an angle feature map specifically includes: Splitting the transformed feature map into a plurality of first feature maps according to a channel dimension and into a plurality of second feature maps according to an angle dimension by the feature extraction unit; Inputting a plurality of first feature maps into a Transformer network, and determining a channel feature map through the Transformer network; Several second feature maps are input into the Transformer network, and the angle feature map is determined by the Transformer network.

5. The method for segmenting ovarian lesions in ultrasonic images according to claim 1, characterized in that: The performing a reverse Radon transform on the channel feature map and the angle feature map by the reverse projection unit to output a spatial feature map specifically includes: Performing reverse Radon transformation on the channel feature map and the angle feature map respectively through the reverse projection unit to obtain a reverse channel feature map and a reverse angle feature map; The reverse channel feature map, the reverse angle feature map and the minimum scale encoding feature map are spliced ​​by the reverse projection unit to obtain a spatial feature map.

6. An ultrasound image ovarian lesion segmentation system, characterized in that: A trained ovarian lesion segmentation model is applied, wherein the ovarian lesion segmentation model includes an encoding module, a pseudo feature decoding block, a decoding module, and a Radon projection module, wherein the Radon projection module includes a latent feature projection unit, a feature extraction unit, and a back projection unit; the ultrasound image ovarian lesion segmentation system specifically includes: An acquisition module, used for acquiring an ultrasound image including an ovarian region; A control module, used for inputting an ultrasound image having an ovarian region into an encoding module, and outputting a multi-scale encoding feature map through the encoding module; determining a minimum-scale decoding feature map based on the minimum-scale encoding feature map and the second-small-scale encoding feature map through the pseudo-feature decoding block, and determining a pseudo-positive feature map and a pseudo-negative feature map based on the minimum-scale decoding feature map and the second-small-scale encoding feature map; performing self-attention learning on the pseudo-positive feature map to obtain a self-attention feature, and performing interactive learning on the pseudo-positive feature map and the pseudo-negative feature map to obtain a cross-attention feature; determining a fusion feature map based on the self-attention feature, the cross-attention feature and the minimum-scale decoding feature map; determining an ovarian lesion segmentation map corresponding to the ultrasound image through the decoding module based on the fusion feature map and encoding feature maps of other scales except the minimum-scale encoding feature map; Wherein, after the multi-scale coding feature map is output by the coding module, the method further includes: The minimum-scale encoded feature map is subjected to Radon transformation at a preset angle by the latent feature projection unit to obtain a transformed feature map; Extracting a feature map of the transformation feature from a channel dimension and an angle dimension by the feature extraction unit to obtain a channel feature map and an angle feature map; The channel feature map and the angle feature map are subjected to inverse Radon transform by the inverse projection unit to output a spatial feature map, and the spatial feature map is used as the minimum-scale encoding feature map.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the ultrasound image ovarian lesion segmentation method as described in any one of claims 1-5.

8. A terminal device, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the method for segmenting ovarian lesions in ultrasound images as described in any one of claims 1 to 5 are implemented.

Images