Image boundary padding method and medical image analysis method

Abstract

The application relates to an image boundary filling method and a medical image analysis method. The method obtains an initial extended canvas of an input image, extracts an extended boundary region around the four sides of the extended canvas and splices the extended boundary region into a boundary sequence, encodes the boundary sequence and the input image respectively, extracts global context features based on original image features, fuses the global context information into the boundary sequence features through an attention interaction mechanism, reconstructs the fused boundary features, obtains extended boundary prediction data, and finally backfills the prediction data to the corresponding region of the extended canvas to generate a filling result with continuous boundaries and consistent structures. The method can be used for image extrapolation and can also be used as a boundary filling operation in a convolutional neural network, and can improve the feature expression of the boundary region in the medical image analysis scene.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image boundary filling method that can be used for image extrapolation, convolutional neural network boundary filling, and medical image analysis tasks. Background Technology

[0002] In visual tasks such as semantic segmentation, object detection, image reconstruction, and medical image analysis, convolutional neural networks often need to perform padding operations at the boundaries of images or feature maps to maintain the spatial dimensional relationship before and after convolution and ensure feature alignment in boundary regions. However, after processing such as convolutional sampling, local cropping, and ROI extraction, images or feature maps are prone to problems such as missing boundary pixels, broken edge features, or discontinuous textures. In medical imaging scenarios, these problems may further affect the analysis quality of critical areas such as organ boundaries and the periphery of lesions.

[0003] Currently, common boundary padding methods in image processing and convolutional neural networks include constant padding, copy padding, mirror padding, interpolation extension, or simple convolutional expansion. These methods are simple to implement and computationally inexpensive, but they typically only expand based on local neighborhood pixels or fixed rules, failing to fully utilize the global structural and semantic context information of the entire image or feature map. This can easily lead to boundary artifacts, amplified local noise, or unnatural edge transitions. In medical imaging scenarios, such boundary discontinuities may manifest as harsh transitions at organ edges, incomplete details at lesion boundaries, or inconsistent soft tissue textures. Summary of the Invention

[0004] Based on this, the purpose of this application is to provide an image boundary filling method and its application in medical image analysis, so as to solve the problems of traditional boundary filling lacking semantic awareness, insufficient boundary continuity, and difficulty in embedding it into neural networks as a lightweight boundary processing module, thereby improving the boundary representation quality of images or feature maps; in the medical image analysis scenario, it can improve the stability of boundary region feature extraction.

[0005] The image boundary filling method described in this application includes the following steps:

[0006] Obtain the initial expanded canvas of the input image; Extract the extended boundary regions around the initial extended canvas, and stitch the various extended boundary regions together to form an extended boundary region sequence; The extended boundary region sequence and the input image are encoded respectively to obtain the extended boundary region sequence features and the original image features; Global context features are extracted based on the original image features; the global context features are fused into the extended boundary region sequence features through an attention interaction mechanism to obtain fused boundary features; boundary features are reconstructed from the fused boundary features to obtain extended boundary prediction data; Based on the extended boundary prediction data, prediction data corresponding to the extended boundary regions around the four sides are obtained; the prediction data is filled into the corresponding regions of the initial extended canvas to obtain the output image after filling.

[0007] This application also provides a medical image analysis method, including the following steps: Acquire medical images to be analyzed; The medical image is input into the medical image analysis network; before one or more convolution processing nodes of the medical image analysis network perform convolution processing, the image boundary filling method described in any one of the embodiments of this application is executed on the medical image or intermediate feature map to be input into the convolution layer to obtain the filled feature map; Based on the filled feature map, convolutional feature extraction is performed on the corresponding convolutional layer, and the extracted features are input into the subsequent network layers of the medical image analysis network for further processing, and finally the medical image analysis results are output.

[0008] This application's embodiments precisely define the scope and data organization of boundary processing by acquiring the initial expanded canvas of the input image and extracting its surrounding expanded boundary regions, which are then stitched together into a sequence. The expanded boundary region sequence, the input image, and the original boundary regions are each segmented and embedded, converting data from different regions into feature representations with a unified dimension and discriminative power. Furthermore, global contextual features are extracted based on the original image features, enabling the extraction of global structural information, texture contextual information, and boundary transition information from the entire image or feature map. These are then fused into the expanded boundary region sequence features through an attention interaction mechanism, giving the boundary features contextual awareness of the overall image content. Further, the fused boundary features are reconstructed and backfilled into the expanded canvas, resulting in an output image with better boundary continuity and stronger structural consistency. In medical image analysis scenarios, this output can provide more stable feature representations for areas such as organ boundaries and lesion edges, thereby improving the boundary processing quality of subsequent segmentation, detection, or reconstruction models.

[0009] To better understand and implement this application, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the image boundary filling method according to an embodiment of this application; Figure 2 This is a flowchart illustrating the medical image analysis method according to an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Wherein, when the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0012] It should be understood that the embodiments described below do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0013] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application are also intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, in the description of this application, unless otherwise stated, “a plurality” means two or more. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items, for example, A and / or B, which can represent: A alone, A and B together, and B alone; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship.

[0014] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms, and these terms are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Depending on the context, the word "if" as used in this application can be interpreted as "when," "when," or "in response to determination."

[0015] Please refer to Figure 1 The image boundary filling method described in this application includes the following steps: S101: Obtain the initial extended canvas of the input image; S102: Extract the extended boundary regions around the initial extended canvas, and stitch the various extended boundary regions together to form an extended boundary region sequence; S103: Encode the extended boundary region sequence and the input image respectively to obtain the extended boundary region sequence features and the original image features; S104: Extract global context features based on the original image features; fuse the global context features into the extended boundary region sequence features through an attention interaction mechanism to obtain fused boundary features; reconstruct the boundary features of the fused boundary features to obtain extended boundary prediction data; S105: Based on the extended boundary prediction data, obtain the prediction data corresponding to the extended boundary regions around the four sides respectively; fill the prediction data into the corresponding region of the initial extended canvas to obtain the output image after filling.

[0016] This application's embodiments precisely define the scope and data organization of boundary processing by acquiring the initial expanded canvas of the input image and extracting its surrounding expanded boundary regions, which are then stitched together into a sequence. The expanded boundary region sequence, the input image, and the original boundary regions are each segmented and embedded, converting data from different regions into feature representations with a unified dimension and discriminative power. Furthermore, global contextual features are extracted based on the original image features, enabling the extraction of global structural information, texture contextual information, and boundary transition information from the entire image or feature map. These are then fused into the expanded boundary region sequence features through an attention interaction mechanism, giving the boundary features contextual awareness of the overall image content. Further, the fused boundary features are reconstructed and backfilled into the expanded canvas, resulting in an output image with better boundary continuity and stronger structural consistency. In medical image analysis scenarios, this output can provide more stable feature representations for areas such as organ boundaries and lesion edges, thereby improving the boundary processing quality of subsequent segmentation, detection, or reconstruction models.

[0017] The image boundary filling method described in this application is executed by a computer device. The following provides a detailed description of each step.

[0018] For step S101, obtain the initial extended canvas of the input image.

[0019] The input image is the original tensor data to be expanded and filled, serving as the initial input to the entire processing flow. Specifically, it can be image data or an intermediate feature map output by a convolutional neural network, typically in a B×C×H×W format, containing batch size, number of channels, height, and width information. It forms the basis for boundary generation and filling. In one embodiment, the input image refers to the image or feature map of a medical image. This image data, containing key information about the medical image, is obtained after preprocessing operations such as convolutional sampling, local cropping, and ROI region extraction during medical image processing. It forms the basis for subsequent boundary filling processing.

[0020] The initial expanded canvas is a temporary canvas data larger than the input image after initial boundary expansion to the target width. In other words, it's an image obtained by expanding the input image (e.g., adding a pixel area of ​​a certain width around it). The purpose of this expansion is to provide more surrounding information for boundary processing.

[0021] This step obtains the initial expanded canvas of the input image after initial boundary expansion, providing a foundation for subsequent boundary sequence modeling and boundary prediction data backfilling. Specifically, the input image is read in tensor format B×C×H×W. The target boundary expansion width d is determined, which can be the convolution padding width or the image extrapolation width. Layer-by-layer iterative boundary initialization expansion is performed on the input image: using the input image as the initial feature map, a ring of feature units is added around the perimeter in each round. Optionally, each round of expansion obtains initial pixels through boundary neighborhood weighted convolution, and then refines them through bi-branch multi-scale convolution to obtain an updated feature map. Iteration continues until the expansion width reaches d, resulting in an initial expanded canvas of size B×C×(H+2d)×(W+2d).

[0022] In one embodiment, step S101, obtaining the initial expanded canvas of the input image, includes: Step S1011: Obtain the input image and the target boundary expansion width.

[0023] The target boundary expansion width is a predefined expansion parameter used to specify the number of pixels or feature blocks to be expanded around the input image to supplement the contextual information of the boundary region. For example, when the expansion width is 16 pixels, 16 columns or rows of feature data need to be added to the top, bottom, left, and right sides of the feature map.

[0024] Step S1012: Expand the input image based on the target boundary expansion width to obtain the corresponding initial expanded canvas.

[0025] The expanded boundary regions in the initial expanded canvas are the boundary regions to be further optimized or reconstructed. Boundary expansion refers to the operation of adding feature data around the input image through methods such as padding, mirroring, or copying. The purpose is to expand the spatial range of the feature map and provide richer contextual information for subsequent boundary processing (such as feature fusion and prediction).

[0026] This embodiment supports setting the expansion width of the input image. By obtaining the target boundary expansion width and performing the corresponding boundary expansion operation, the task requirements are effectively met.

[0027] In one embodiment, step S1012, which involves performing layer-by-layer boundary initialization expansion on the input image based on the target boundary expansion width to obtain the corresponding initial expanded canvas, includes: Step S10121: The input image is used as the current image to be expanded; an iterative expansion step is performed based on the current image to be expanded: convolutional inference calculation is performed based on the edge pixels of the current image to be expanded to obtain peripheral expanded pixels; the peripheral expanded pixels are merged with the current image to be expanded to generate an updated current image to be expanded; the iterative expansion step is continued until the boundary expansion width of the current image to be expanded matches the target boundary expansion width, and the current image to be expanded is determined as the initial expansion canvas; the expansion boundary region of the initial expansion canvas is used to carry the backfilling of subsequent boundary prediction data.

[0028] Convolutional inference computation is a computational method that uses convolution operations to extract and predict edge information. Specifically, based on the pixels in the edge's neighborhood, weighted convolution and multi-scale convolutional inference are used to estimate the values ​​of the surrounding extended pixels, thereby achieving layer-by-layer boundary initialization.

[0029] The term "peripheral expansion pixels" refers to the newly added edge pixels / feature units during boundary expansion. These are new pixel values ​​or feature values ​​generated at the edges of the input image through convolutional inference calculations, used to fill the boundary region. In this embodiment, during each iteration of expansion, a ring of new pixels calculated by edge convolution is merged with the current feature map to achieve size expansion. This can preserve the local feature information of the original edge to a certain extent, avoiding discontinuities or artificial artifacts at the expanded boundary.

[0030] This embodiment achieves boundary expansion through iterative convolutional inference, significantly improving the rationality and naturalness of the expanded region and laying a high-quality foundation for subsequent boundary processing steps. Traditional methods (such as zero-padding and mirror padding) generate expanded pixels using only simple rules, which can easily lead to discontinuous boundaries or texture distortion. This embodiment, however, captures local edge features and contextual patterns through convolutional kernels, generating only a single layer of outer pixels in each iteration, gradually approaching the target width. This ensures that the expanded region is highly consistent with the original feature map in terms of texture, gradient, and semantics. The final generated initial expanded canvas retains the core information of the original feature map while providing richer contextual support through intelligently filled boundary regions, enabling subsequent steps to extract features or predict boundaries based on more complete spatial information.

[0031] In one embodiment, step S10121, which involves performing convolutional inference calculations based on the edge pixels of the current image to be expanded to obtain the peripheral expanded pixels, includes: Step S101211: Based on the edge pixels of the current image to be expanded, an initial estimation result is obtained through boundary neighborhood weighted convolution; boundary refinement is performed through bi-branch multi-scale convolution to obtain the boundary refinement result.

[0032] Boundary neighborhood weighted convolution is a convolution operation designed for edge pixels of feature maps. By dynamically adjusting the weight distribution of the convolution kernel in the edge region, it enhances the ability to capture local edge features while suppressing interference from invalid information outside the boundary, generating an initial pixel estimate that better matches the true edge distribution.

[0033] Bi-branch multi-scale convolution is a multi-scale feature extraction structure consisting of two parallel convolutional branches. One branch uses a small-scale convolutional kernel (such as 3×3) to capture local details of the edges, while the other branch uses a large-scale convolutional kernel (such as 5×5 or 7×7) to extract the macroscopic context of the edges. By fusing the outputs of the two branches, refined modeling of boundary features is achieved.

[0034] The initial estimation result is the output of the boundary neighborhood weighted convolution, which reflects the preliminary mapping of the local neighborhood features of edge pixels in the extended region, but may have problems such as blurred details or discontinuous boundaries due to ignoring multi-scale context.

[0035] Boundary refinement is the output of bi-branch multi-scale convolution. By fusing local details with macro context, it corrects blurry or broken boundaries in the initial estimation results, generating more accurate and continuous boundary pixel values.

[0036] Step S101212: The initial estimation result is fused with the boundary refinement result to obtain the peripheral extended pixels.

[0037] The initial estimation results are fused with the boundary refinement results through weighted summation or an attention mechanism to generate the final peripheral extended pixels. The fused pixel values ​​preserve the continuity of local edge texture while conforming to the overall boundary trend of the organ or lesion.

[0038] This embodiment significantly improves the rationality and accuracy of the extended boundary by combining the weighted convolution of the boundary neighborhood with the bi-branch multi-scale convolution, providing high-quality input for subsequent boundary processing steps.

[0039] In one embodiment, step S1011, obtaining the target boundary expansion width, includes: Step S10111: Obtain the boundary padding width of the convolution process.

[0040] Boundary padding width refers to the extra pixels (such as zero padding or mirror padding) that need to be added around the boundaries of the input image in a convolutional neural network to maintain the same spatial dimensions as the input and output feature maps. This padding width is determined by parameters such as the kernel size, stride, and whether padding is used. For example, when the kernel size is 3×3, the stride is 1, and "same" padding is used, the boundary padding width is typically 1 pixel to ensure that the output feature map size is the same as the input.

[0041] Step S10112: Determine the target boundary expansion width of the input image based on the boundary fill width.

[0042] The target boundary expansion width is the total number of pixel layers of the input image boundary that needs to be expanded in this embodiment. Its value is dynamically determined based on the boundary padding width of the convolution processing. It aims to provide a clear expansion range for subsequent boundary expansion operations, ensuring that the expanded feature map can meet the input size requirements of convolution calculation or adapt to the contextual information requirements of specific tasks (such as large target detection and high-resolution segmentation).

[0043] This embodiment targets the convolution processing scenario. It determines the target boundary expansion width of the input image based on the boundary padding width of the convolution processing, ensuring that the expanded feature map can meet the input size requirements of subsequent convolution calculations and avoiding feature loss or calculation errors due to size mismatch (such as the output feature map size being too small to cover the target area).

[0044] For step S102, the extended boundary regions around the initial extended canvas are extracted, and the various extended boundary regions are spliced ​​together to form an extended boundary region sequence.

[0045] The extended boundary region refers to the edge regions in the initial extended canvas that extend beyond the input image and require focused modeling and generation. These are specific areas located around the initial extended canvas, extended from the input image, and are the key parts requiring filling. In one embodiment, the four edge bands (top, right, bottom, and left) with a width equal to the extended width, extracted from the initial extended canvas, are used as the extended boundary region.

[0046] An extended boundary region sequence is a data structure that reorganizes two-dimensional extended boundary regions into a one-dimensional continuous form, facilitating sequence modeling. Specifically, it is a sequence structure formed by piecing together the extended boundary regions around the initial extended canvas in a certain order (such as clockwise or counterclockwise). This structure facilitates the serialization of boundary regions and the extraction of their inherent sequential features. In one embodiment, four extended boundary regions are pieced together in a clockwise order and flattened along the boundary length to form a continuous sequence, transforming the two-dimensional boundary into a one-dimensional sequence.

[0047] This step aims to extract the boundaries to be modeled and convert them into a sequence, reducing the dimensionality of the modeling. The extended boundary regions are extracted from the initial extended canvas; these are the target areas to be filled. These extended boundary regions are then concatenated into a sequence, making the information within the boundary regions more organized and concentrated, facilitating subsequent unified analysis and processing, and uncovering the inherent connections and patterns between the boundary regions.

[0048] In one embodiment, step S102, which involves extracting the extended boundary regions around the initial extended canvas and concatenating these extended boundary regions into a sequence, includes: Step S1021: Extract four non-overlapping expansion boundary regions (top, right, bottom, and left) from the initial expanded canvas, with a width equal to the target boundary expansion width; wherein, the right and left expansion boundary regions are transposed to unify the travel direction of the four expansion boundary regions. Step S1022: The four extended boundary regions are spliced ​​together along the boundary length direction in the order of top, right, bottom, and left to obtain a continuous sequence of extended boundary regions.

[0049] The boundary length direction refers to the direction along which the extended boundary region extends. Flattening the spliced ​​boundary region into a continuous sequence along this direction makes the information in the boundary region more orderly and compact, which is convenient for subsequent operations such as feature extraction and fusion.

[0050] An extended boundary region sequence refers to a continuous sequence obtained by stitching together the extended boundary regions around the initial extended canvas and then flattening them along the length of the boundary. This sequence integrates and arranges the information of the boundary regions in an orderly manner, which is beneficial for uncovering the potential relationships and feature patterns between the boundary regions, and provides more effective data support for subsequent boundary filling and image inpainting.

[0051] This embodiment implements the serialization processing of the extended boundary regions around the initial extended canvas. Accurately extracting the extended boundary regions from the initial extended canvas ensures the integrity and accuracy of the boundary information. Stitching the boundary regions preserves their relative positional relationships and spatial structure information, avoiding the loss or error of boundary information due to messy stitching. Flattening the stitched boundary regions into a continuous sequence along the boundary length makes the information in the boundary regions more ordered and compact, facilitating subsequent operations such as feature extraction and fusion.

[0052] For step S103, the extended boundary region sequence and the input image are encoded respectively to obtain the extended boundary region sequence features and the original image features.

[0053] Encoding is the process of converting image or region data into a unified feature representation that the model can process. It is used to standardize data format, extract structural information, and support subsequent contextual modeling and feature interaction. Methods include, but are not limited to, patch embedding, convolutional embedding, linear mapping, feature flattening, and pixel-wise encoding, which can transform a two-dimensional region into a one-dimensional feature sequence or vector, achieving a mapping from region to feature representation. In one embodiment, patch embedding encoding is used. Patch embedding encoding is an encoding operation that divides the image / boundary region into blocks and maps them to high-dimensional feature vectors. Specifically, it divides the image region into several small blocks, and then performs an encoding operation on each small block, converting it into a fixed-dimensional feature vector. This embodiment uses patch embedding to divide the boundary and original image into blocks and map them into vectors, transforming them into token features that the model can process, preparing for contextual interaction.

[0054] The extended boundary region sequence is a high-dimensional feature representation obtained through block embedding encoding. Encoded from the extended boundary region sequence, it serves as the query feature for attention interactions, carrying initial information about the boundary to be optimized.

[0055] The original image features are the overall content feature representation obtained by block embedding encoding of the input image. They are obtained by encoding the original input image and contain semantic and structural information of the main image content, providing a global reference for boundary generation.

[0056] This step aims to convert images / sequences into high-dimensional features that the model can process. Encoding the extended boundary region sequence allows for the extraction of local feature information of the boundary region; encoding the input image yields the original image features that reflect the overall characteristics of the entire image.

[0057] In one embodiment, step S103, which involves performing block-based embedding encoding on the extended boundary region sequence and the input image to obtain the extended boundary region sequence features and the original image features, further includes: Step S1031: Determine the original boundary region of the input image; encode the original boundary region to obtain the original boundary region features.

[0058] The original boundary region is the inherent boundary region formed by the edge positions of the input image itself. It is a transition region connecting the main body region of the original image and the extended boundary region, and is used to provide boundary transition auxiliary information in the subsequent boundary reconstruction stage.

[0059] The original boundary region feature is a feature representation that reflects the original boundary region features after encoding the original boundary region of the input image.

[0060] This step, in addition to performing block embedding encoding on the extended boundary region sequence, can extract local feature information of the boundary region. Block embedding encoding is performed on the input image to obtain the original image features. Furthermore, block embedding encoding is performed on the original boundary region of the input image to obtain the original boundary region features, which are used to understand the original state and features of the input image boundary.

[0061] Step S104, the step of extracting global context features based on the original image features, further includes: Step S1041: Extract boundary transition auxiliary features based on the original boundary region features; the boundary transition auxiliary features are used to fuse with the fused boundary features during the boundary feature reconstruction stage.

[0062] Boundary transition auxiliary features are obtained by sequential expansion of the original boundary region features and are used to enhance the continuous transition between the extended boundary and the edge of the original image during the boundary reconstruction stage.

[0063] This step utilizes original image features and original boundary region features to play different roles: original image features are used to extract global contextual features reflecting the overall content and spatial structure; original boundary region features are used to extract auxiliary boundary transition features to preserve the texture, gradient, and structural continuity at the input image or the original edges of the input image. These two features work synergistically in the subsequent attention fusion and boundary reconstruction stages, ensuring that the generated boundary conforms to the global content semantics and seamlessly connects with the original edges. For example, in medical imaging, this allows for more accurate identification of the overall location and shape of human organs, as well as the original features of organ boundaries, providing a more reliable basis for subsequent boundary filling.

[0064] This embodiment performs block-based embedding encoding on the extended boundary region sequence, the input image, and the original boundary region, comprehensively extracting feature information at different levels. The extended boundary region sequence features help accurately process local features of the boundary region; the original image features reflect the overall image features, providing a foundation for global analysis; and the original boundary region features preserve the original boundary information, providing auxiliary features for boundary transition during the boundary reconstruction stage. This embodiment fully considers the overall structure of the image and the original boundary features, better reflecting the prior features of the global image. Ultimately, it significantly improves the accuracy and reliability of medical image boundary filling.

[0065] For step S104, global context features are extracted based on the original image features; the global context features are fused into the extended boundary region sequence features through an attention interaction mechanism to obtain fused boundary features; the fused boundary features are reconstructed to obtain extended boundary prediction data.

[0066] Global context features are features extracted from the entire input image that reflect the overall semantic information of the image. They include specific prior features such as human tissue distribution and texture distribution, which play an important guiding role in accurately filling boundary regions.

[0067] Attention interaction is a mechanism for feature fusion that assigns different weights based on the importance of different features, selectively fusing global context features into extended boundary region sequence features to enhance the expressive power of boundary features. In this embodiment, boundary features are used as queries and global features as keys. Through attention weight allocation, global information is injected into the boundary features to achieve semantic alignment.

[0068] Boundary feature reconstruction refers to the decoding process of restoring the fused boundary features into boundary data that can be directly backfilled. In this embodiment, through multi-level expansion, upsampling, and feature refinement, the high-dimensional fused features are restored into predicted data that matches the expanded boundary size.

[0069] This step extracts global contextual features from the original image, capturing the overall features and semantic information of the entire image. This provides specific prior features, such as the distribution of human tissue and texture in the image, for subsequent boundary region filling. An attention interaction mechanism is used to fuse the global contextual features into the extended boundary region sequence features. This allows the boundary features to incorporate global information, enhancing their representational power. Different attention weights are assigned based on the importance of different features, enabling targeted fusion. Finally, boundary feature reconstruction is performed on the fused boundary features to recover the complete feature information of the boundary region, generating predicted data for the extended boundary.

[0070] In one embodiment, step S104, which involves extracting global context features based on the original image features, includes: Step S1042: Perform sequence expansion and long-range dependency modeling on the original image features to obtain global context features that match the sequence length of the expanded boundary region.

[0071] This step performs sequence expansion on the original image features and performs long-range dependency modeling using a state-space sequence model to obtain global context features that match the sequence length of the expanded boundary region.

[0072] Sequence expansion refers to the dimensional expansion of the original image features and the original boundary region features, giving them richer information in the sequence dimension and providing a more sufficient data foundation for subsequent long-range dependency modeling. This is similar to adding more elements or dimensions to a data sequence to better capture the relationships between features. In this embodiment, sequence expansion stretches the length and maps the dimensions of the original image features and the original boundary features to match the boundary sequence length, providing a unified format for state-space modeling.

[0073] State-space sequence models are used to efficiently model long sequence dependencies, capturing long-range dependencies between elements in a sequence. Specifically, state-space sequence models maintain a state space, continuously updating the state as they process sequence data, thus remembering previously processed information and associating it with currently processed information. In one embodiment, Mamba2 is preferred over the traditional Transformer for long sequence feature extraction with lower computational cost, making it suitable for modeling long-range dependencies in boundary sequences. In medical image processing, complex, long-distance relationships may exist between different regions of an image; state-space sequence models can effectively model these relationships, thereby extracting more accurate global contextual features.

[0074] The global context features are obtained from the original image features through sequence expansion and long-range dependency modeling, and are used to provide global structural constraints for the generation of extended boundaries.

[0075] In this step, the original image features are input into the state space sequence model for sequence expansion and long-range dependency modeling. This allows the original image features to match the expanded boundary region sequence in terms of sequence length, thereby obtaining global context features. Global context features are used to provide overall structure and semantic constraints.

[0076] Step S1041, which involves extracting boundary transition auxiliary features based on the original boundary region features, includes: Step S10411: Perform sequence expansion on the original boundary region features to obtain boundary transition auxiliary features that match the size of the fused boundary features.

[0077] Boundary transition auxiliary features are used to preserve the texture, gradient, and structural continuity at the original boundary. Both types of features are used together in the subsequent boundary reconstruction stage to generate extended boundary prediction data.

[0078] This embodiment performs sequence expansion on the original image features and original boundary region features, enriching the information content of the features and providing a more sufficient data foundation for long-range dependency modeling. By using a state-space sequence model, it effectively captures the long-range dependencies between elements in the sequence, extracting more comprehensive and accurate global context features. These global context features not only consider the overall structure and organization of the image but also take into account the original features of the boundaries, thus better reflecting the prior features of the input image.

[0079] In one embodiment, step S104, which fuses the global context features into the extended boundary region sequence features through an attention interaction mechanism to obtain fused boundary features, includes: Step S1043: Using the extended boundary region sequence features as query features and the global context features as key and value features, perform multi-head cross-attention interaction to obtain attention fusion results.

[0080] Query features refer to the features that act as the "questioning" subject in the attention interaction mechanism. This step refers to the extension of the boundary region sequence features, which are used to actively retrieve relevant information from the key-value features in order to obtain global context information related to themselves. It is the active party in feature fusion.

[0081] Key features and value features refer to the features that serve as an "information repository" in attention interactions; in this step, it refers to global context features. These features store global semantic information about the entire image (such as the distribution of human tissues and texture patterns), and provide supplementary information to the query features through matching with them.

[0082] Multi-head cross-attention interaction is a feature fusion mechanism that processes the interaction between query features and key-value features in parallel through multiple independent attention heads. Each attention head focuses on a different feature subspace, enabling it to capture the correlation between query features and global context features from multiple dimensions, thus enhancing the comprehensiveness and depth of feature fusion. In this embodiment, multi-head cross-attention interaction implements weighted fusion across feature sources in parallel across multiple subspaces, using extended boundary sequence features as the Query and global context features as the Key and Value, to achieve cross-domain feature fusion.

[0083] This step uses extended boundary region sequence features as query features and global context features as key and value features, achieving deep interaction through a multi-head cross-attention mechanism. Each attention head independently calculates the similarity between the query feature and the key and value features, generating attention weights, and then weightedly fusing information from the key and value features. For example, in medical imaging, if the query feature corresponds to the edge region of an organ, the global context in the key and value features may contain information such as the overall morphology of the organ and the distribution of neighboring tissues. Multi-head cross-attention can mine the association between this information and the query feature from different angles (such as morphology, texture, and spatial relationships), so that the fusion result contains both local details of the boundary and global semantic information, avoiding boundary breaks or distortions caused by the lack of local information.

[0084] Step S1044: Add the attention fusion result to the global context features after residual mapping, and perform layer normalization to obtain the fusion boundary features.

[0085] This step performs residual normalization on the fusion result obtained from multi-head cross-attention. Specifically, the fusion result is added to the global context features after residual mapping, and then layer normalization is performed to obtain the fusion boundary features. This process preserves the structural information in the context branches of the entire image while stabilizing the distribution of the fusion features. In another embodiment, the fusion result can also be residually fused with the extended boundary region sequence features or their mapping features.

[0086] This embodiment significantly improves the accuracy and reliability of medical image boundary filling through multi-head cross-attention interaction and residual normalization processing. Multi-head cross-attention mines the correlation between extended boundary region sequence features and global context features from multiple dimensions, ensuring that the fused boundary features contain both local boundary details and global semantic information. This effectively solves problems such as organ edge breakage and lesion boundary distortion caused by the lack of local information in traditional methods. Residual normalization processing ensures the stability and detail preservation of boundary filling by retaining the original boundary features and standardizing the fusion result, avoiding abrupt boundary transitions or texture distortion that may occur during the filling process. The final fused boundary features more accurately reflect the true boundary morphology, providing high-quality feature input for subsequent boundary feature reconstruction and filling operations, and significantly improving the quality of medical images.

[0087] In one embodiment, step S104, which involves reconstructing the fused boundary features to obtain extended boundary prediction data, includes: Step S1045: Perform multi-level block expansion, upsampling processing and layer-by-layer feature refinement on the fused boundary features and the boundary transition auxiliary features respectively to obtain the first boundary reconstruction features and the boundary transition reconstruction features.

[0088] Multi-level block expansion refers to the operation of dividing the fused boundary features and boundary transition auxiliary features into multiple sub-blocks and expanding their spatial dimensions step by step. This embodiment uses multi-level block expansion to gradually increase the spatial size of the boundary features for both paths, matching the target expanded boundary resolution. Block processing avoids detail blurring caused by global expansion, while multi-level expansion gradually restores the spatial structure information of the boundary, such as the transition from coarse-grained morphology to fine-grained texture at the organ edge.

[0089] Upsampling is a scaling operation that increases the spatial size of feature maps. In this embodiment, the fused boundary features and boundary transition auxiliary features are expanded in blocks to enlarge the size of the boundary features, providing a spatial basis for layer-by-layer feature refinement and final output. In medical imaging, upsampling can restore low-resolution boundary features to a size that matches the original expanded boundary region, preserving the subtle morphology of lesion edges.

[0090] Layer-by-layer feature refinement refers to the progressive optimization of two expanded features through multiple convolutions, activation functions, and normalization operations. Each layer of refinement focuses on feature adjustments at different scales. For example, the first layer may enhance the continuity of the boundary contour, while the next layer refines texture details, ensuring that the fused boundary features maintain global semantics and the boundary transition auxiliary features maintain local edge continuity, ultimately making the features closer to the real boundary shape. In one embodiment, boundary features are optimized layer by layer through modules such as the VSSD Block (VisualState Space Duality Block) to improve detail continuity and naturalness.

[0091] Step S1046: The first boundary reconstruction feature and the boundary transition reconstruction feature are fused to obtain the joint boundary reconstruction feature.

[0092] This step aggregates the first boundary reconstruction feature, which carries global semantic information, with the boundary transition reconstruction feature, which carries local continuity information of the original boundary, to form a joint boundary reconstruction feature that combines global structural consistency and local edge smoothness. Optional fusion methods include element-wise addition fusion, channel concatenation followed by convolutional mapping fusion, and attention-weighted fusion. Element-wise addition fusion directly adds the two features element-wise at the same spatial location and channel dimension, achieving a simple superposition of global semantics and local transition information. Channel concatenation followed by convolutional mapping fusion first concatenates the two features along the channel dimension, then performs channel dimensionality reduction and feature remapping through 1×1 convolution, achieving adaptive weighted integration of information. Attention-weighted fusion learns the weight distribution of the two features through an attention network, and then performs a weighted summation of the first boundary reconstruction feature and the boundary transition reconstruction feature according to their weights, thereby strengthening important features and suppressing redundant information.

[0093] Step S1047: Continue to perform upsampling and layer-by-layer feature refinement on the joint boundary reconstruction features to obtain refined boundary features.

[0094] Refining boundary features involves refining and fusing them separately through two paths, followed by further multi-level refinement to achieve detailed boundary features that can be directly mapped to the output. After multiple levels of expansion, separate refinement, joint fusion, and further refinement, the features are convolutionally mapped to obtain predicted data that matches the expanded boundary, which is then used to backfill the output.

[0095] Step S1048: Map the refined boundary features to extended boundary prediction data that matches the extended boundary region in both spatial size and number of channels.

[0096] Feature mapping adjusts the refined boundary features to the same spatial size and number of channels as the expanded boundary region through linear or nonlinear transformations, ensuring that the predicted data after fusing global semantics and local transition constraints can be directly used for backfilling operations, achieving seamless integration.

[0097] This embodiment uses multi-level block expansion and upsampling to progressively restore boundary details from the fused boundary features and boundary transition auxiliary features, respectively. Layer-by-layer feature refinement optimizes the two boundary features from coarse to fine, effectively combining global semantic information with local edge continuity information; the final feature mapping ensures seamless connection between the predicted data and the original boundary region. This series of operations makes the joint boundary features highly consistent with real medical images in terms of morphology, texture, and spatial structure, effectively solving problems such as stiff organ edges, distorted lesion boundaries, soft tissue texture distortion, abrupt boundary transitions, and texture misalignment in traditional methods.

[0098] For step S105, based on the extended boundary prediction data, the prediction data corresponding to the extended boundary regions around the four sides are obtained respectively; the prediction data is filled into the corresponding region of the initial extended canvas to obtain the output image after filling.

[0099] This step aims to restore the sequence prediction results to two-dimensional boundaries and fill them in. Specifically, the predicted data of the extended boundary is matched with the surrounding extended boundary regions, and the predicted data corresponding to each region is obtained. Then, these predicted data are filled back into the corresponding regions of the initial extended canvas, completing the filling of the medical image boundaries and obtaining a high-quality feature map after filling. This feature map has more complete boundary information and more accurate feature representation.

[0100] In one embodiment, step S105, which involves obtaining the prediction data corresponding to the four surrounding extended boundary regions based on the extended boundary prediction data, includes: Step S1051: According to the length range of the four extended boundary regions (upper, right, lower, and left) recorded during the splicing of the extended boundary region sequence, the extended boundary prediction data is decomposed into four corresponding prediction boundary data; wherein, the right and left prediction boundary data are subjected to a reverse transpose process, which is the opposite of the transpose process.

[0101] Based on the length ranges of the four boundary regions (top, right, bottom, and left) recorded during boundary packing, the expanded boundary prediction data is sequentially decomposed into top boundary prediction data, right boundary prediction data, bottom boundary prediction data, and left boundary prediction data. For the right and left boundary prediction data, a reverse transpose operation is performed, the opposite of the transpose operation in the packing stage, to restore their spatial orientation in the 2D expanded canvas. This operation ensures that each decomposed data block strictly corresponds to the original boundary region positions around the initial expanded canvas, avoiding filling misalignment due to disordered order.

[0102] Step S1052: Based on the four predicted boundary data, determine the predicted data corresponding to each of the extended boundary regions.

[0103] The four independent predicted boundary data blocks obtained from the decomposition are directly used as the predicted data for the corresponding expanded boundary regions. For example, the decomposed "upper boundary predicted data" directly corresponds to the boundary region at the top of the initial expanded canvas, and the "left boundary predicted data" corresponds to the left region. This one-to-one mapping relationship ensures that subsequent filling operations can accurately fill the original missing areas with the predicted data, achieving seamless integration of boundary repair.

[0104] This embodiment achieves the corresponding backfilling of the extended boundary prediction data to the four boundary regions of the initial extended canvas through length interval-based decomposition and inversion processing. This processing avoids positional mismatches during the boundary sequence restoration process, ensuring that the predicted boundary data is consistent with the spatial position and direction of travel of the four extended boundary regions (top, right, bottom, and left). Ultimately, after the processed predicted data is filled into the initial extended canvas, it can significantly improve the boundary integrity, morphological accuracy, and texture realism of medical images, thereby helping to improve the boundary recognition quality and result stability of medical image analysis models.

[0105] This application also provides a medical image analysis method, including the following steps: S201: Acquire medical images to be analyzed.

[0106] Raw medical images (such as 3D voxel data or 2D slice images) are obtained from medical imaging equipment (such as CT, MRI, X-ray) or image databases and preprocessed into feature map forms suitable for deep learning processing.

[0107] S202: Input the medical image into the medical image analysis network; before one or more convolution processing nodes of the medical image analysis network perform convolution processing, perform the image boundary filling method described in any one of the embodiments of this application on the medical image or intermediate feature map to be input into the convolution layer to obtain the filled feature map.

[0108] Convolutional operations are performed on the padded medical images to extract high-level semantic features. Since the padded boundary regions combine local boundary information and global contextual information, the convolutional kernels can obtain more continuous neighborhood inputs when computing near the boundary regions, thereby improving the quality of boundary region feature extraction.

[0109] S203: Based on the filled feature map, perform convolutional feature extraction of the corresponding convolutional layer, and input the extracted features into the subsequent network layers of the medical image analysis network for further processing, and finally output the medical image analysis result.

[0110] The extracted feature maps are further processed according to the specific medical task (such as segmentation, detection, and classification). In segmentation tasks, the feature maps are input into a decoder to generate pixel-level segmentation masks. For example, in brain tumor segmentation, the padded feature maps help the decoder more accurately locate the tumor edges. In detection tasks, candidate boxes are generated on the feature maps using a region proposal network, and the target locations are classified and regressed. For example, in breast calcification detection, expanded boundary regions can reduce the missed detection of tiny calcifications near the image edges. Finally, structured analysis results, such as segmentation masks or detection box coordinates and categories, are generated and visualized by overlaying them onto the original image for doctors' reference.

[0111] The medical image analysis method proposed in this embodiment integrates the aforementioned image boundary filling method as a pluggable boundary processing module into the medical image processing workflow through intelligent boundary filling-feature extraction-image analysis. This module can be applied to the original input image or to intermediate feature maps in the network, solving the feature extraction deviation problem caused by information loss or unreasonable filling during boundary processing in traditional methods. This improves the accuracy and robustness of medical image analysis. This method is particularly suitable for scenarios in medical images with complex target boundaries and strong contextual dependencies (such as tumor segmentation and organ detection).

[0112] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and this application also intends to include these modifications and variations.

Claims

1. An image boundary filling method, characterized in that, Includes the following steps: Obtain the initial expanded canvas of the input image; Extract the extended boundary regions around the initial extended canvas, and stitch the various extended boundary regions together to form an extended boundary region sequence; The extended boundary region sequence and the input image are encoded respectively to obtain the extended boundary region sequence features and the original image features; Global context features are extracted based on the features of the original image; The global context features are fused into the extended boundary region sequence features through an attention interaction mechanism to obtain fused boundary features; the fused boundary features are then reconstructed to obtain extended boundary prediction data. Based on the extended boundary prediction data, prediction data corresponding to each of the extended boundary regions is obtained; the prediction data is then filled into the corresponding region of the initial extended canvas to obtain the output image after filling.

2. The image boundary filling method according to claim 1, characterized in that, The steps for obtaining the initial expanded canvas of the input image include: Obtain the input image and the target boundary expansion width; The input image is expanded layer by layer based on the target boundary expansion width to obtain the corresponding initial expanded canvas.

3. The image boundary filling method according to claim 2, characterized in that, The steps of performing layer-by-layer boundary initialization expansion on the input image based on the target boundary expansion width to obtain the corresponding initial expanded canvas include: The input image is used as the current image to be expanded; an iterative expansion step is performed based on the current image to be expanded: convolutional inference calculation is performed on the edge pixels of the current image to be expanded to obtain the peripheral expansion pixels; the peripheral expansion pixels are merged with the current image to be expanded to generate an updated current image to be expanded; the iterative expansion step is continued until the boundary expansion width of the current image to be expanded matches the target boundary expansion width, and the current image to be expanded is determined as the initial expansion canvas.

4. The image boundary filling method according to claim 3, characterized in that, The step of performing convolutional inference calculations based on the edge pixels of the current image to be expanded to obtain the peripheral expanded pixels includes: Based on the edge pixels of the current image to be expanded, an initial estimation result is obtained through boundary neighborhood weighted convolution; boundary refinement is performed through bi-branch multi-scale convolution to obtain the boundary refinement result. The initial estimation result is fused with the boundary refinement result to obtain the peripheral extended pixels.

5. The image boundary filling method according to claim 1, characterized in that, The step of extracting the extended boundary regions around the initial extended canvas and concatenating the various extended boundary regions into an extended boundary region sequence includes: From the initial expanded canvas, extract four non-overlapping expanded boundary regions: top, right, bottom, and left, with a width equal to the target boundary expansion width. The right and left expanded boundary regions are transposed to unify the travel direction of the four expanded boundary regions. The four extended boundary regions are spliced ​​together along the boundary length direction in the order of top, right, bottom, and left to obtain a continuous sequence of extended boundary regions.

6. The image boundary filling method according to claim 5, characterized in that, The step of obtaining the prediction data corresponding to each of the extended boundary regions based on the extended boundary prediction data includes: According to the length range of the four extended boundary regions (upper, right, lower, and left) recorded during the splicing of the extended boundary region sequence, the extended boundary prediction data is decomposed into four corresponding prediction boundary data; wherein, the right and left prediction boundary data are subjected to a reverse transpose process, which is the opposite of the transpose process. Based on the four predicted boundary data, the predicted data corresponding to each of the extended boundary regions is determined.

7. The image boundary filling method according to claim 1, characterized in that, The step of encoding the extended boundary region sequence and the input image respectively to obtain the extended boundary region sequence features and the original image features further includes: Determine the original boundary region of the input image; encode the original boundary region to obtain the original boundary region features; The step of extracting global context features based on the original image features also includes: Sequence expansion and long-range dependency modeling are performed on the original image features to obtain global context features that match the sequence length of the expanded boundary region; The original boundary region features are sequentially expanded to obtain a boundary transition auxiliary feature whose size matches that of the fused boundary feature; the boundary transition auxiliary feature is used to fuse with the fused boundary feature during the boundary feature reconstruction stage.

8. The image boundary filling method according to claim 7, characterized in that, The step of fusing the global context features into the extended boundary region sequence features through an attention interaction mechanism to obtain the fused boundary features includes: Using the extended boundary region sequence features as query features and the global context features as key and value features, multi-head cross-attention interaction is performed to obtain attention fusion results. The attention fusion result is added to the global context features after residual mapping, and then layer normalization is performed to obtain the fusion boundary features.

9. The image boundary filling method according to claim 7 or 8, characterized in that, The step of reconstructing the fused boundary features to obtain extended boundary prediction data includes: Multi-level block expansion, upsampling, and layer-by-layer feature refinement are performed on the fused boundary features and the boundary transition auxiliary features respectively to obtain the first boundary reconstruction features and the boundary transition reconstruction features. The first boundary reconstruction feature is fused with the boundary transition reconstruction feature to obtain a joint boundary reconstruction feature; The joint boundary reconstruction features are further subjected to upsampling and layer-by-layer feature refinement to obtain refined boundary features; The refined boundary features are mapped to extended boundary prediction data that match the extended boundary region in both spatial size and number of channels.

10. A medical image analysis method, characterized in that, Includes the following steps: Acquire medical images to be analyzed; The medical image is input into the medical image analysis network; before one or more convolution processing nodes of the medical image analysis network perform convolution processing, the image boundary filling method according to any one of claims 1 to 9 is performed on the medical image or intermediate feature map to be input into the convolution layer to obtain the filled feature map. Based on the filled feature map, convolutional feature extraction is performed on the corresponding convolutional layer, and the extracted features are input into the subsequent network layers of the medical image analysis network for further processing, and finally the medical image analysis results are output.

Images