A medical image lesion segmentation method, device and electronic equipment based on candidate edge evidence semantic assignment and three-state jump correction

Abstract

The present application provides a kind of medical image lesion segmentation method, device and electronic equipment based on candidate edge evidence semantic distribution and three-state jump correction.The method includes obtaining the medical image to be segmented and inputting lesion segmentation model, and obtaining lesion segmentation probability graph;The model uses encoder-decoder structure, and at least one jump connection layer executes the following steps: generating candidate edge evidence graph and semantic support graph;According to the pixel-by-pixel common activation relationship, pixel-by-pixel difference relationship and neighborhood difference relationship of candidate edge evidence graph and semantic support graph, generate semantic support distribution weight and semantic conflict distribution weight, and distribute candidate edge evidence graph as semantic support evidence graph and semantic conflict evidence graph;With non-candidate position weight graph, semantic support evidence graph and semantic conflict evidence graph, three-state weighting is carried out on encoding feature, detail enhancement feature and semantic reconstruction feature, and corrected jump feature is obtained to make decoder network output lesion segmentation probability graph.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing, deep learning and computer vision, specifically to a method, device and electronic device for lesion segmentation in medical images based on candidate edge evidence semantic allocation and three-state jump correction. It can be used for automatic segmentation of lesion areas in two-dimensional or three-dimensional medical images such as ultrasound, magnetic resonance, computed tomography, and dermoscopy, and is especially suitable for scenarios with blurred lesion boundaries, low local contrast, complex background tissue texture or many false edges. Background Technology

[0002] Encoder-decoder segmentation networks typically utilize skip connections to directly pass high-resolution features from the encoder to the corresponding decoding layer. Shallow encoded features retain more location, texture, and local mutation information, compensating for the loss of spatial detail caused by downsampling; however, shallow features have relatively limited ability to distinguish target categories, and tissue texture, speckle noise, imaging artifacts, and non-lesion contours in medical images may also produce strong edge responses. If passed indiscriminately, the decoder will simultaneously receive high-frequency information of real lesion boundaries and non-targets.

[0003] Existing boundary enhancement methods often directly use the boundaries generated by the labels as the supervision targets of shallow gating branches, or concatenate shallow and deep features to generate a single attention map, and then amplify or suppress the encoded features as a whole. These methods tend to merge whether a local location presents an edge with whether that edge belongs to a target lesion into the same prediction task, failing to clearly distinguish the responsibilities of candidate edge detection and target semantic discrimination. Furthermore, simple pixel-wise multiplication may incorrectly weaken the true boundary due to amplitude differences or slight spatial misalignments between shallow edges and deep semantic boundaries.

[0004] Furthermore, for shallow edges that are not supported by deep semantics, directly subtracting or shrinking the original encoded features will simultaneously weaken the organizational structure and potential weak boundaries at that location, making it difficult to balance pseudo-edge suppression with structural information preservation.

[0005] Therefore, a selective skip connection scheme is needed that can first retain shallow candidate edges, then use deep target semantics to judge the relationship of candidate evidence, and adopt different feature processing methods for the three states of non-candidate, semantic support and semantic conflict. Summary of the Invention

[0006] To overcome the problems existing in related technologies, this invention proposes a method, device, and electronic device for medical image lesion segmentation based on candidate edge evidence semantic allocation and three-state jump correction. Specifically, this invention is achieved through the following technical solution:

[0007] According to a first aspect of the embodiments of this specification, a method for medical image lesion segmentation based on candidate edge evidence semantic allocation and three-state jump correction is provided. The method includes acquiring a medical image to be segmented and inputting it into a trained lesion segmentation model, and outputting a lesion segmentation probability map of the medical image to be segmented through the lesion segmentation model. The lesion segmentation model adopts an encoder-decoder structure, wherein the encoder network of the model is used to extract multi-scale encoded features, and the decoder network of the model is used to generate decoded semantic features corresponding to each jump connection layer step by step. At least one jump connection layer of the model performs the following steps:

[0008] Step S1: Generate a single-channel candidate edge evidence map based on the encoding features of the same layer, and generate a single-channel semantic support map based on the decoding semantic features of the same layer.

[0009] Step S2: Generate semantic support allocation weights and semantic conflict allocation weights based on the pixel-wise co-activation relationship, pixel-wise difference relationship, and neighborhood difference relationship between the candidate edge evidence map and the semantic support map, and assign the candidate edge evidence map as a semantic support evidence map and a semantic conflict evidence map.

[0010] Step S3: Construct the detail enhancement feature of the encoded feature and the semantic reconstruction feature of the decoded semantic feature. Use the non-candidate position weight map, the semantic support evidence map and the semantic conflict evidence map to perform three-state weighting on the encoded feature, detail enhancement feature and semantic reconstruction feature to obtain the corrected skip feature and input it into the decoder network so that the decoder network can fuse the corrected skip feature and restore the spatial resolution, and output the lesion segmentation probability map; wherein, the non-candidate position weight map is obtained by pixel-by-pixel complementation of the candidate edge evidence map.

[0011] According to a second aspect of the embodiments of this specification, a medical image lesion segmentation device based on candidate edge evidence semantic allocation and three-state jump correction is provided, the device comprising:

[0012] The image acquisition unit is used to acquire the medical image to be segmented and input it into the trained lesion segmentation model;

[0013] An image segmentation unit is used to output a lesion segmentation probability map of the medical image to be segmented through the lesion segmentation model; the lesion segmentation model adopts an encoder-decoder structure, the encoder network of the model is used to extract multi-scale encoded features, the decoder network of the model is used to generate decoded semantic features corresponding to each skip connection layer step by step, and at least one skip connection layer of the model performs the method as described in the first aspect.

[0014] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising: a processor; and a computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in the first aspect.

[0015] The embodiments of the present invention have at least the following technical effects:

[0016] (1) The present invention separates shallow candidate edge discovery from deep target boundary semantic support. The candidate edge evidence graph does not directly accept the supervision of the real boundary and can retain all local mutations that are worth further judgment. The semantic support graph undertakes the semantic discrimination of the target boundary, avoiding duplication of the two branch tasks.

[0017] (2) The present invention uses candidate edge evidence map, semantic support map, co-activation, pixel-wise difference and neighborhood difference to describe cross-layer relationship, and then uses two-channel Softmax conservation to allocate the same candidate edge evidence map, which can reduce the sensitivity of simple pixel-wise multiplication to response amplitude and slight spatial misalignment.

[0018] (3) Instead of simply subtracting the original encoded features at the semantic conflict position, this invention uses deep semantics to reconstruct the shallow representation; at the same time, it calls the detail enhancement features at the semantic support position, thereby taking into account both pseudo-edge suppression and preservation of real weak boundary details.

[0019] (4) The non-candidate position weight graph, semantic support evidence graph, and semantic conflict evidence of the present invention Figure 3 The class weights are non-negative per pixel and sum to 1. The correction process has a clear state meaning and a bounded mixing relationship, which facilitates implementation, interpretation and ablation verification.

[0020] (5) This invention can be deployed as a pluggable skip connection module in U-Net and its encoder-decoder variants without relying on a specific backbone network; no label boundaries are required during the inference stage, making it suitable for breast ultrasound, dermoscopy and other lesion segmentation scenarios with blurred boundaries. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Some specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings in an exemplary and non-limiting manner. The same reference numerals in the drawings indicate the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0022] Figure 1This is a flowchart illustrating an exemplary embodiment of the present invention of a medical image lesion segmentation method based on candidate edge evidence semantic allocation and three-state jump correction;

[0023] Figure 2 This is a schematic diagram of the network structure of a lesion segmentation model shown in an exemplary embodiment of the present invention;

[0024] Figure 3 This is a comparative diagram of the segmentation results of the method of the present invention, U-Net, and Attention U-Net in five groups of benign breast ultrasound test cases, as illustrated in an exemplary embodiment of the present invention.

[0025] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present invention;

[0026] Figure 5 This is a block diagram illustrating a medical image lesion segmentation device based on candidate edge evidence semantic allocation and three-state jump correction, as shown in an exemplary embodiment of the present invention. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0029] The technical solution of this invention separates the edge appearance discovery and the target boundary semantic support into separate models: the candidate edge evidence map only describes whether the local position presents an edge and does not accept direct supervision from the real boundary; the semantic support map is generated by deep decoding semantics and accepts the supervision of the target boundary; then, the candidate edge evidence map is conservatively split into a semantic support evidence map and a semantic conflict evidence map through learnable relation allocation, and detail enhancement features and semantic reconstruction features are called respectively.

[0030] Figure 1This is a flowchart illustrating an exemplary embodiment of the present invention of a medical image lesion segmentation method based on candidate edge evidence semantic allocation and three-state jump correction. The method includes acquiring the medical image to be segmented and inputting it into a trained lesion segmentation model, and outputting a lesion segmentation probability map of the medical image through the lesion segmentation model. The lesion segmentation model adopts an encoder-decoder structure. The encoder network of this model is used to extract multi-scale encoded features, and the decoder network of this model is used to generate decoded semantic features corresponding to each jump connection layer step by step, such as... Figure 1 As shown, at least one skip connection layer in this model performs the following steps:

[0031] Step S1: Generate a single-channel candidate edge evidence map based on the encoding features of the same layer, and generate a single-channel semantic support map based on the decoding semantic features of the same layer.

[0032] The candidate edge evidence map is used to characterize the degree to which abrupt changes in edge or texture occur at a local location, and is not subject to direct supervision of the actual target boundary.

[0033] Step S2: Generate semantic support allocation weights and semantic conflict allocation weights based on the pixel-by-pixel co-activation relationship, pixel-by-pixel difference relationship, and neighborhood difference relationship between the candidate edge evidence map and the semantic support map, and assign the candidate edge evidence map as a semantic support evidence map and a semantic conflict evidence map.

[0034] The pixel-wise co-activation relationship is the pixel-wise product of the candidate edge evidence map and the semantic support map; the pixel-wise difference relationship is the pixel-wise absolute difference between the candidate edge evidence map and the semantic support map; the neighborhood difference relationship is the absolute difference between the local neighborhood mean of the candidate edge evidence map and the local neighborhood mean of the semantic support map.

[0035] Step S3: Construct the detail enhancement feature of the encoded feature and the semantic reconstruction feature of the decoded semantic feature. Use the non-candidate position weight map, the semantic support evidence map and the semantic conflict evidence map to perform three-state weighting on the encoded feature, detail enhancement feature and semantic reconstruction feature to obtain the corrected skip feature and input it into the decoder network so that the decoder network can fuse the corrected skip feature and restore the spatial resolution, and output the lesion segmentation probability map.

[0036] The non-candidate position weight map is obtained by pixel-by-pixel complementing of the candidate edge evidence map. The non-candidate position weights, the semantic support evidence map, and the semantic conflict evidence map are also included. Figure 3 The sum of each pixel in the semantic support evidence map and the semantic conflict evidence map is equal to 1, and the sum of the corresponding pixels in the semantic support evidence map and the semantic conflict evidence map is equal to the candidate edge evidence map.

[0037] based on Figure 1 The medical image lesion segmentation method shown in this embodiment

[0038] Next, combined Figure 2 and Figure 3 Detailed explanation of the lesion segmentation process in medical images.

[0039] (a) Data input, preprocessing and basic network.

[0040] The lesion segmentation model in this embodiment of the invention sequentially completes multi-scale feature extraction, candidate edge evidence graph generation, semantic support graph generation, relation allocation, three-state jump correction, and segmentation output.

[0041] This embodiment first obtains a medical image lesion segmentation dataset, which includes medical images. and its corresponding lesion segmentation label For medical images Size uniformity and intensity normalization are performed; during the training phase, random flipping, rotation, scaling, cropping, and intensity perturbation can be performed to obtain standardized images. :

[0042] (1)

[0043] in, This indicates that bilinear interpolation is used to interpolate medical images. Adjust to preset space size ; This indicates the image after resizing. Perform minimum-maximum intensity normalization. and They are respectively Minimum strength and maximum strength, To prevent the use of a preset positive number with a denominator of zero, this embodiment takes... .

[0044] This embodiment is... The same spatial transformation is performed synchronously, where the labels Size adjustment uses nearest neighbor interpolation to avoid generating unexpected class values. For binary classification tasks, The lesion area is set to 1, and the background area is set to 0.

[0045] In practical applications, a dataset containing medical images and pixel-level lesion masks is used. For breast ultrasound images, the images can be standardized to 256×256 or other preset sizes, and the gray values ​​can be normalized to [0,1] or mean-variance standardized.

[0046] like Figure 2 As shown, the standardized image Input encoder-decoder network. The encoder generates encoded features step by step. Coding features ... coding features The decoder generates decoding semantic features corresponding to the skip connection layers step by step. Decoding semantic features Decoding semantic features .in, Indicates the number of layers in the encoder and decoder. Represents the layer index. For the first... layer, The encoded features to be corrected , and These represent the number of channels, height, and width of the feature, respectively. To include decoding semantic features with strong target semantics, the spatial dimensions and required number of channels of the encoded features are aligned with the features through upsampling and 1×1 convolution before use.

[0047] The candidate edge evidence semantic assignment and three-state jump correction module can be set in all jump connection layers, or only in one or more high-resolution jump connection layers. Preferably, it is set in the first three high-resolution jump connection layers near the input to utilize shallow location details and control computational load.

[0048] In some embodiments, the encoder-decoder network employs a four-level encoder and a four-level decoder. Each encoder level includes convolution, normalization, activation, and downsampling, while each decoder level includes upsampling, skip feature fusion, and a convolutional block. The first three high-resolution skip connection layers are configured for implementation. Figure 1 Steps S1 to S3; the deepest skip connection can maintain regular fusion. The outputs of the first three high-resolution skip connection layers and Corrected skip characteristics with the same spatial size and number of channels No need to change the interface of subsequent decoding blocks.

[0049] (ii) Candidate Marginal Evidence Map The generation of .

[0050] In some embodiments, the generation of the candidate edge evidence map includes: channel compression and normalization of the encoded features to obtain internal features; extraction of high-frequency response, center-nearest neighbor difference, and cross-scale difference from the internal features; response normalization and learnable fusion of the high-frequency response, center-nearest neighbor difference, and cross-scale difference to obtain the single-channel candidate edge evidence map.

[0051] In some possible implementations of this embodiment, response normalization and learnable fusion are performed on the high-frequency response, center-nearest neighbor difference, and cross-scale difference, including:

[0052] Spatial response normalization is performed on the high-frequency response, center-nearest neighbor difference, and cross-scale difference, respectively. The normalized three responses are concatenated along the channel dimension and then subjected to convolution and Sigmoid activation to obtain scale selection weights. The normalized high-frequency response, the normalized center-nearest neighbor difference weighted by the scale selection weights, and the normalized cross-scale difference weighted by complementary scale selection weights are concatenated and then subjected to convolution and Sigmoid activation to obtain the candidate edge evidence map.

[0053] In some possible implementations of this embodiment, high-frequency response, center nearest neighbor difference, and cross-scale difference are extracted from the internal features, including:

[0054] The high-frequency response is obtained by applying a Laplacian convolution kernel to the internal features and taking the absolute value; the internal features are then subjected to average pooling at a first scale and average pooling at a second scale to obtain a first local basis and a second local basis; the nearest neighbor difference is obtained based on the difference between the internal features and the first local basis, and the cross-scale difference is obtained based on the difference between the first local basis and the second local basis.

[0055] Specifically, regarding the first Layer coding features Perform 1×1 convolution to compress or unify the internal channels, followed by group normalization to obtain the internal features. Group normalization is used to organize the response scales of different channels and reduce the dominance of individual channel amplitudes on subsequent high-frequency responses.

[0056] (2)

[0057] in, Indicates the kernel size as Learnable convolutions, therefore This represents a 1×1 learnable convolution; This represents the group normalization operator, which divides the channels of the internal features of each sample into a predetermined number of channels. For each group, the mean and variance are calculated in the channel dimension and spatial dimension, and then normalization and learnable affine transformation are performed. The preset number of groups.

[0058] With a fixed 3×3 Laplacian convolution kernel Channel-by-channel processing And by taking the absolute value, the high-frequency response is obtained. The parameters of the fixed convolutional kernel are not used in training. It is only used to stabilize the intensity of local mutations and does not directly represent the boundary of the target lesion.

[0059] (3)

[0060] In the formula, Indicated by As a fixed convolution kernel pair Performed channel-wise 2D convolution, = .

[0061] The first local base was obtained by using 3×3 average pooling and 5×5 average pooling with a step size of 1 while keeping the spatial dimensions constant. Second local base .by The deviation from the small neighborhood basis constitutes the center nearest neighbor difference. The difference between the first local basis and the second local basis constitutes the cross-scale difference. .

[0062] (4)

[0063] (5)

[0064] in, Indicates the kernel size as The channel-wise average pooling operator; in this embodiment, the step size is 1, and in the feature Figure 4 Zhou used Pixel padding is used to maintain the output space size. In equation (5) It indicates taking the absolute value of each element; the superscripts 3 and 5 indicate the pooling scale, and the superscript 35 indicates the difference between the two scales of 3×3 and 5×5, none of which indicate a power.

[0065] , and The natural amplitudes differ. To avoid one path dominating the fusion solely due to its larger numerical scale, spatial response normalization is performed on the non-negative responses of each path:

[0066] (6)

[0067] in, For the space response normalization operator, , and This indicates the corresponding spatial dimension of the current response graph. Let represent any nonnegative response tensor to be normalized; This means taking the mean of each sample and each channel over the two spatial dimensions of height and width. To prevent the use of a preset positive number with a denominator of zero, this embodiment takes... ; , and These are the normalized high-frequency response, center nearest neighbor difference, and cross-scale difference.

[0068] Then, the three normalized responses are concatenated along the channel dimension, and the scale-selective weight map is obtained by 1×1 convolution and sigmoid activation. :

[0069] (7)

[0070] When the value is large, increase the proportion of center-nearest neighbor differences; when the value is small, increase the proportion of cross-scale differences; high-frequency response Always retain.

[0071] The three processing results are concatenated to obtain the scale-selective fusion feature. :

[0072] (8)

[0073] splicing tensors Candidate edge evidence map compressed into a single channel by 3×3 convolution and sigmoid activation. :

[0074] (9)

[0075] in, This represents the element-wise Sigmoid activation function, whose output range is... ; Indicates splicing along the channel. This indicates element-wise multiplication.

[0076] A higher value only indicates that the location is more like a marginal or local structural abrupt change, not that it necessarily belongs to the boundary of a lesion. (Incorrect training...) To set a true boundary auxiliary loss, we can avoid the candidate edge extraction branch and the subsequent target semantic support branch from having the same responsibilities; The end-to-end gradient is obtained through the final segmentation loss.

[0077] (III) Semantic Support Graph The generation of .

[0078] In some embodiments, the generation of the semantic support graph includes: upsampling the decoded semantic features to the same spatial size as the encoded features in the same layer; and sequentially applying 3×3 convolution, ReLU activation function, 1×1 convolution and Sigmoid activation to the upsampled decoded semantic features to obtain the single-channel semantic support graph.

[0079] Specifically, the first Layer decoding semantic features Upsampling × First, neighborhood semantics are aggregated through 3×3 convolution, then compressed into a single-channel logit (logit representation) through ReLU activation and 1×1 convolution. The real-valued response before the activation function is finally processed by the Sigmoid function to obtain the semantic support graph. .

[0080] (10)

[0081] (11)

[0082] in, This indicates that the semantic features will be decoded. Upsampling to The operator used in this embodiment is bilinear interpolation; ; and These are referred to as semantic convolution features and semantic activation features, respectively.

[0083] This indicates the degree of support from the deep target semantics for the current location being considered as a target boundary region.

[0084] During the training phase, lesion labels were used. Scale alignment is performed to obtain .right Using max pooling to approximate soft expansion, for 1- Approximate soft erosion is achieved by using max pooling followed by complement set, and the difference between the two yields the target boundary label. :

[0085] (12)

[0086] in, This represents a channel-wise max pooling operator with a kernel size of 3×3, a step size of 1, and a padding of 1. This means truncating each element to... Interval.

[0087] To reduce the loss of narrow boundaries caused by downsampling, it is preferable to compute in high-resolution skip layers. For multi-scale supervision, labels can be aligned to the corresponding scales first using nearest neighbor methods, and then the boundary labels mentioned above can be calculated. The semantic support loss employs both binary cross-entropy loss and Dice loss.

[0088] (13)

[0089] in, Represents the prediction graph With target map Spatial average of pixel-wise binary cross-entropy; This represents 1 minus the Dice overlap coefficient with a smoothing term, which can be calculated separately using the following formulas:

[0090] (14)

[0091] In the formula, and These represent the height and width of the current prediction map and the target map, respectively. and These represent the pixel's position index in the height and width directions, respectively; A preset positive number is used to prevent abnormal values ​​from occurring in logarithms or divisions; These are pre-defined non-negative Dice loss weights used to adjust the relative contribution of this loss term to the single-layer semantic support loss. These weights remain fixed during a single training cycle and are not used as network parameters updated by the optimizer. Their specific values ​​can be set or adjusted based on the training data and the actual segmentation task.

[0092] Based on the above embodiments, it can be seen that in this embodiment... On the one hand, accept On the one hand, it supervises the target boundary, and on the other hand, it accepts the gradient of the final segmentation loss through subsequent relation assignment and skip feature correction, thus preserving the semantics of the target boundary and adapting to the final segmentation task.

[0093] (Four) and The relationship description and allocation of conservation evidence.

[0094] In some embodiments, the generation methods for semantic support allocation weights and semantic conflict allocation weights include:

[0095] The candidate edge evidence map, the semantic support map, the pixel-wise co-activation relationship, the pixel-wise difference relationship, and the neighborhood difference relationship are concatenated along the channel dimension to form a relation tensor. The relation tensor is then subjected to channel-wise convolution and ReLU activation functions to generate local relation features. After performing point-wise convolution on the local relation features, Softmax normalization is then performed along the channel dimension to obtain the semantic support allocation weight and the semantic conflict allocation weight.

[0096] This embodiment avoids direct setting. = ⊙ , = ⊙(1- The amplitude binding and spatial misalignment sensitivity issues caused by this will and The relationship is explicitly represented as five types of descriptive quantities.

[0097] First, calculate the common activation graph. and pixel-by-pixel difference map :

[0098] (15)

[0099] exist and At the same time, a larger value is obtained when the value is higher, so as to preserve evidence of overlap at the same location; Describes numerical discrepancies on the same pixel.

[0100] Further calculations and The 3×3 neighborhood mean is used to construct a neighborhood difference map based on the difference between the two. :

[0101] (16)

[0102] Neighborhood difference maps allow two boundary responses to be compared within a local neighborhood, which can help mitigate the positional offset of one or a few pixels.

[0103] In this embodiment, candidate strength, semantic support, co-activation, point difference, and neighborhood difference are concatenated into a relation tensor:

[0104] (17)

[0105] A 3×3 channel-wise convolution is performed on the relation tensor to observe the local patterns of each relation separately. After ReLU, the relation channels are combined by a 1×1 convolution, and two relation response values ​​are output. Softmax is performed on the channel dimension to obtain semantic support assignment weights. Weighting semantic conflicts .

[0106] (18)

[0107] (19)

[0108] in, This represents a channel-wise convolution with a kernel size of 3×3, where the number of groups is equal to the number of input channels; This is called a local feature of the relation; This means performing normalization on each spatial location of the two relation channels, so that the resulting two weights are non-negative and their sum is 1.

[0109] Multiply both weights back to the same candidate edge evidence. Semantic Support Evidence Map Semantic conflict evidence diagram :

[0110] (20)

[0111] because and Each pixel is non-negative and the sum is 1, and the evidence allocation satisfies the conservation relationship.

[0112] The above allocation process only reallocates the candidate evidence discovered in the first stage; it does not arbitrarily increase the total number of marginal entities or delete the total number of candidate entities during the relation allocation stage. Represent candidate evidence that matches the semantic support graph. This indicates evidence that there is a conflict between the appearance of the candidate edges and the semantic support graph.

[0113] (v) Supports detail enhancement, conflict semantic reconstruction and three-state jump correction.

[0114] This embodiment addresses non-candidate position weight graphs, semantic support evidence graphs, and semantic conflict evidence. Figure 3 Different feature representations are constructed or selected for each state. First, 3×3 average pooling is used to obtain the local basis of the encoded features. The detail residual is obtained by subtracting the local basis from the original encoded features. :

[0115] (twenty one)

[0116] The detailed residuals are sequentially fed into a 3×3 channel-wise convolution and a 1×1 point-wise convolution to obtain the result. The same amount of detail compensation is applied to each channel, and the detail enhancement feature is formed by summing the residuals. :

[0117] (twenty two)

[0118] (twenty three)

[0119] in, This represents a pointwise convolution with a kernel size of 1×1, used to blend channels while maintaining the same spatial dimensions; The meaning is the same as in formula (18); This is called the detail compensation amount.

[0120] For semantically conflicting locations, not directly... Perform subtraction or clear to zero. Upsampling and channel alignment features are obtained through 1×1 convolution. ,Will and The features are concatenated along the channel dimension and then generated by 3×3 convolution. This allows the deep target semantics to participate in the reorganization of the shallow representation at that location.

[0121] (twenty four)

[0122] (25)

[0123] in, This represents the decoded semantic features after channel alignment is achieved through spatial upsampling and 1×1 convolution; These are called semantic reconstruction features, and their spatial size and number of channels are consistent with those of the encoding features.

[0124] This embodiment defines three categories of weights: non-candidate, semantic support, and semantic conflict. , and :

[0125] (26)

[0126] in, The original encoded features are preserved at locations where no obvious candidate edges are found. Invoke detailed enhancement features at locations where semantic support is obtained. Invoke semantic reconstruction features at the location where semantic conflict occurs.

[0127] Based on the above three types of weights, the encoding features, detail enhancement features, and semantic reconstruction features are weighted in three states to obtain the corrected jump features. :

[0128] (27)

[0129] Three weighted single-channel spatial graphs, in relation to When channel features are multiplied, the data is broadcast along the channel dimension. achievable ,therefore It is a bounded three-state convex combination: non-candidate regions mainly retain the original features, support regions mainly adopt detailed enhancement representation, and conflict regions mainly adopt reconstruction representation involving deep semantics.

[0130] (vi) Decoding output, joint training and inference.

[0131] This embodiment will correct the skipping feature. The features are concatenated or added and fused with the upsampled features from the corresponding decoder and then fed into subsequent decoding blocks. After restoring the spatial resolution step by step, the lesion segmentation probability map is obtained by segmenting the output header and using Sigmoid or Softmax activation. .

[0132] This embodiment constructs a lesion segmentation loss. It can be composed of Dice loss and binary cross-entropy loss:

[0133] (28)

[0134] Total loss in this embodiment for:

[0135] (29)

[0136] in, and The definition is consistent with equation (14); The non-negative weights of the binary cross-entropy term in the segmentation loss, The overall weights of the multi-layer semantic support loss are used to support the overall weights. For the first The layer semantics support loss for non-negative layer weights. This represents a weighted summation of the losses of all skip connection layers that receive semantic support supervision. The weights mentioned above are preset hyperparameters determined through the validation set and are not used as network parameters in backpropagation learning. When using multi-layer supervision, the weights of each layer can be further normalized to a sum equal to 1.

[0137] In this embodiment, No settings in the middle , or The direct supervision terms are: candidate edge evidence extraction, relation assignment, and three-state correction parameters are optimized by backpropagation of the final segmentation loss; the semantic support branch is simultaneously supervised by the target boundary and constrained by the final segmentation loss.

[0138] The inference phase only requires the medical image to be segmented as input, without lesion labels or target boundary labels. The model generates these labels online based on the encoded features. Generate online based on decoded semantics Further generation , and Finally, the segmentation probability map is output, and the lesion segmentation result is obtained by thresholding or the maximum probability class.

[0139] The embodiments of the present invention have also been experimentally verified.

[0140] To verify the end-to-end feasibility and preliminary technical effectiveness of the model of this invention, a three-model comparison was conducted on the publicly available breast ultrasound image dataset BUSI. The dataset contains 647 cases in total, including 437 benign cases and 210 malignant cases; it is divided into three stratified sets according to a random seed of 2026, with 451 cases in the training set, 98 cases in the validation set, and 98 cases in the test set.

[0141] All three models use the same four-level U-Net backbone, 256×256 input size, data preprocessing, data augmentation, segmentation loss, and training budget. The comparison models include U-Net, Attention U-Net, and a network that inserts the module of this invention into the skip connections of the same backbone. The number of trainable parameters for the three models are 7,240,705, 7,327,989, and 7,877,826, respectively. The training batch size is 8, and the maximum number of training epochs is 100. Training is stopped early if the Dice on the validation set does not improve for 15 consecutive epochs after 30 epochs.

[0142] The values ​​in Table 1 are the average values ​​of the indicators for each case in the test set of 98 cases. Among them, the larger the values ​​of Dice, IoU, Precision, and Recall, the better the segmentation result. HD95 is the 95% Hausdorff distance in pixels. The smaller the value, the closer the predicted boundary is to the true boundary.

[0143] Table 1: Segmentation results of the three models on the BUSI test set

[0144]

[0145] As shown in Table 1, under the same data partitioning, preprocessing, and training budget, the average Dice, IoU, Precision, and Recall of the method of this invention are 0.760508, 0.675963, 0.807566, and 0.756032, respectively, with an average HD95 of 41.167 pixels. Compared with U-Net, Dice is improved by 0.041449, and HD95 is reduced by 9.031 pixels; compared with Attention U-Net, Dice is improved by 0.028195, and HD95 is reduced by 8.714 pixels. These results support the ability of the technology chain of this invention to complete end-to-end training and demonstrate better region overlap and boundary localization performance on this fixed test set.

[0146] Figure 3Five sets of qualitative comparisons of the same cases are presented. From left to right, each set includes the original breast ultrasound image, the actual segmentation label, the U-Net prediction result, the Attention U-Net prediction result, and the prediction result of the method of this invention. The cases shown were selected from those in the test set where the method of this invention had a Dice higher than the two comparison models and an HD95 lower than the two comparison models, taking into account both benign and malignant categories, including three sets of benign cases and two sets of malignant cases. The images are used to demonstrate typical segmentation patterns, and the overall performance is judged based on the results of all 98 test cases listed in Table 1.

[0147] This embodiment is a single fixed partition and single random seed validation on a public dataset, used to illustrate the feasibility and technical effect of the present invention. It does not constitute a conclusion on statistical significance, multicenter validation or clinical performance, nor is it intended to limit the scope of protection of the present invention.

[0148] Figure 4 This is a schematic diagram of an electronic device illustrated in this specification according to an exemplary embodiment. Please refer to... Figure 4 At the hardware level, the device includes a processor 410, an internal bus 420, a network interface 430, memory 440, a hardware acceleration device 450, and non-volatile memory 460, and may also include other hardware required for its functions. One or more embodiments of the present invention can be implemented in software, for example, the processor 410 reads the corresponding computer program from the non-volatile memory 460 into the memory 440 and then runs it. Of course, in addition to software implementation, one or more embodiments of the present invention do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the above processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0149] Figure 5 This is a block diagram illustrating a medical image lesion segmentation device based on candidate edge evidence semantic allocation and three-state jump correction, as shown in an exemplary embodiment of the present invention. The medical image lesion segmentation device can be applied to, for example... Figure 4 The electronic device shown implements the technical solution of the present invention. The medical image lesion segmentation device includes:

[0150] The image acquisition unit 510 is used to acquire the medical image to be segmented and input it into the trained lesion segmentation model;

[0151] Image segmentation unit 520 is used to output a lesion segmentation probability map of the medical image to be segmented through the lesion segmentation model; the lesion segmentation model adopts an encoder-decoder structure, the encoder network of the model is used to extract multi-scale coding features, the decoder network of the model is used to generate decoding semantic features corresponding to each skip connection layer step by step, and at least one skip connection layer of the model performs the relevant methods and steps as shown in the previous embodiment.

[0152] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0153] Accordingly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the above embodiments.

[0154] Accordingly, embodiments of the present invention also provide a computer program product configured to perform the methods described in any of the above embodiments.

[0155] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0156] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0157] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0158] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0159] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0160] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0161] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0162] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for lesion segmentation in medical images based on candidate edge evidence semantic allocation and three-state jump correction, characterized in that, The method includes acquiring a medical image to be segmented and inputting it into a trained lesion segmentation model, and outputting a lesion segmentation probability map of the medical image to be segmented through the lesion segmentation model; the lesion segmentation model adopts an encoder-decoder structure, wherein the encoder network of the model is used to extract multi-scale encoded features, and the decoder network of the model is used to generate decoded semantic features corresponding to each skip connection layer step by step, and at least one skip connection layer of the model performs the following steps: Step S1: Generate a single-channel candidate edge evidence map based on the encoding features of the same layer, and generate a single-channel semantic support map based on the decoding semantic features of the same layer. Step S2: Generate semantic support allocation weights and semantic conflict allocation weights based on the pixel-wise co-activation relationship, pixel-wise difference relationship, and neighborhood difference relationship between the candidate edge evidence map and the semantic support map, and assign the candidate edge evidence map as a semantic support evidence map and a semantic conflict evidence map. Step S3: Construct the detail enhancement feature of the encoded feature and the semantic reconstruction feature of the decoded semantic feature. Use the non-candidate position weight map, the semantic support evidence map and the semantic conflict evidence map to perform three-state weighting on the encoded feature, detail enhancement feature and semantic reconstruction feature to obtain the corrected skip feature and input it into the decoder network so that the decoder network can fuse the corrected skip feature and restore the spatial resolution, and output the lesion segmentation probability map; wherein, the non-candidate position weight map is obtained by pixel-by-pixel complementation of the candidate edge evidence map.

2. The method according to claim 1, characterized in that, Step S1 generates a single-channel candidate edge evidence map based on the coding features of the same layer, including: The encoded features are subjected to channel compression and normalization to obtain internal features; High-frequency response, center nearest neighbor difference, and cross-scale difference are extracted from the internal features respectively; The high-frequency response, center-nearest neighbor difference, and cross-scale difference are normalized and learned to be fused to obtain the candidate edge evidence map of a single channel.

3. The method according to claim 2, characterized in that, The process of response normalization and learnable fusion of the high-frequency response, center nearest neighbor difference, and cross-scale difference includes: Spatial response normalization is performed on the high-frequency response, center nearest neighbor difference, and cross-scale difference, respectively. The normalized three responses are concatenated along the channel dimension and then subjected to convolution and Sigmoid activation to obtain the scale selection weights. The normalized high-frequency response, the normalized center nearest neighbor difference weighted by the scale selection weight, and the normalized cross-scale difference weighted by the complementary scale selection weight are concatenated, and then convolved and activated by Sigmoid to obtain the candidate edge evidence map.

4. The method according to claim 2, characterized in that, The extraction of high-frequency response, center nearest neighbor difference, and cross-scale difference from the internal features includes: The high-frequency response is obtained by applying a Laplacian convolution kernel to the internal features and taking the absolute value. The internal features are subjected to average pooling at a first scale and average pooling at a second scale to obtain a first local basis and a second local basis. The center-nearest neighbor difference is obtained based on the difference between the internal features and the first local basis, and the cross-scale difference is obtained based on the difference between the first local basis and the second local basis.

5. The method according to claim 1, characterized in that, Step S1 generates a single-channel semantic support graph based on the decoding semantic features of the same layer, including: The decoded semantic features are upsampled to the same spatial size as the encoded features in the same layer; The upsampled decoded semantic features are sequentially processed through 3×3 convolution, ReLU activation function, 1×1 convolution and Sigmoid activation to obtain the semantic support map of a single channel.

6. The method according to claim 1, characterized in that: The pixel-wise co-activation relationship is the pixel-wise product of the candidate edge evidence map and the semantic support map; The pixel-by-pixel difference relationship is the absolute pixel-by-pixel difference between the candidate edge evidence map and the semantic support map; The neighborhood difference relationship is the absolute difference between the local neighborhood mean of the candidate edge evidence graph and the local neighborhood mean of the semantic support graph.

7. The method according to claim 1, characterized in that, The generation methods for semantic support allocation weights and semantic conflict allocation weights in step S2 include: The candidate edge evidence map, the semantic support map, the pixel-wise co-activation relation, the pixel-wise difference relation, and the neighborhood difference relation are concatenated along the channel dimension to form a relation tensor; The relation tensor is sequentially processed through channel-wise convolution and ReLU activation function to generate local relation features; After performing pointwise convolution on the local features of the relationship, Softmax normalization is then performed on the channel dimension to obtain the semantic support allocation weight and the semantic conflict allocation weight.

8. The method according to claim 1, characterized in that: The pixel-wise sum of the non-candidate position weight, the semantic support evidence map, and the semantic conflict evidence map is 1; The sum of corresponding pixels in the semantic support evidence map and the semantic conflict evidence map is equal to the candidate edge evidence map.

9. A medical image lesion segmentation device based on candidate edge evidence semantic allocation and three-state jump correction, characterized in that, The device includes: The image acquisition unit is used to acquire the medical image to be segmented and input it into the trained lesion segmentation model; An image segmentation unit is used to output a lesion segmentation probability map of the medical image to be segmented through the lesion segmentation model; the lesion segmentation model adopts an encoder-decoder structure, wherein the encoder network of the model is used to extract multi-scale encoded features, the decoder network of the model is used to generate decoded semantic features corresponding to each skip connection layer step by step, and at least one skip connection layer of the model performs the method as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, include: processor; as well as A computer-readable storage medium storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 8.

Images