Automatic image segmentation technology based on convolutional neural network

Abstract

The invention relates to an automatic image segmentation technology based on a convolutional neural network, and is suitable for the field of medical image and industrial detection. In order to solve the problems of rigid feature fusion, insufficient context capture, low efficiency of boundary optimization and poor small target segmentation precision in the existing method, an adaptive multi-scale feature fusion network is constructed: an encoder adopts a progressive expansion strategy and gated attention to intensify cross-scale features; the decoder optimizes hierarchical feature contribution through a dynamic weighted fusion module; the end-to-end boundary optimization is realized by integrating the lightweight differentiable CRF; and designing a composite loss function balance category weight. The segmentation recall rate of the fine structure is obviously improved by more than 18%, the boundary sawtooth rate is reduced by 41%, and the calculation efficiency is improved by 76%.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical imaging and industrial detection, more particularly, the present application relates to an automatic image segmentation technology based on a convolutional neural network. BACKGROUND

[0002] Image segmentation, as a core task of computer vision, aims to divide digital images into regions with semantic meaning, which has wide applications in medical image analysis (such as organ lesion segmentation), autonomous driving (road scene analysis), industrial detection (defect area positioning), etc. In recent years, automatic segmentation technology based on convolutional neural network (CNN) has significantly improved the segmentation accuracy, especially the encoder-decoder architecture (such as U-Net) through the fusion of multi-scale features by jump connection, which has become the mainstream solution. However, the segmentation accuracy of small structures in complex scenes, the fuzzy boundary, and the low computational efficiency still restrict the actual deployment of high-precision real-time segmentation systems, and the existing technology still has certain limitations:

[0003] 1. The multi-scale feature fusion mechanism is rigid

[0004] Although the existing method transmits the encoder features to the decoder through the jump connection, the fixed weight or simple splicing fusion method (such as U-Net) cannot adaptively adjust the contribution weight of different levels of features. The shallow features contain rich details but are sensitive to noise, and the deep features are semantically strong but have low spatial resolution. The static fusion strategy leads to loss of details and insufficient utilization of semantic information, especially in complex boundary and small target segmentation.

[0005] 2. The context information capture capability is limited

[0006] The traditional CNN encoder relies on standard convolution stacking to expand the receptive field, but the down-sampling of deep network leads to serious loss of spatial information. Although the dilated convolution (such as DeepLab series) is introduced to delay the resolution reduction, its expansion rate is usually fixed, which is difficult to balance the local details and the dependence on large-scale context. A single scale receptive field cannot adapt to the feature extraction needs of different size targets in the image, resulting in region misjudgment or boundary discontinuity in the segmentation result.

[0007] 3. The segmentation boundary optimization relies on post-processing separation

[0008] The mainstream solution (such as CRFasRNN) takes the conditional random field (CRF) as an independent post-processing module, which is trained separately from the CNN segmentation network. Such methods need to iterate the energy function multiple times, which has large computational overhead and is difficult to learn end-to-end; at the same time, the binary term parameters of CRF rely on artificial experience setting, which is easy to produce over-smoothing or boundary distortion in uneven light or complex texture scenes.

[0009] 4. Small target segmentation and class imbalance problem is prominent

[0010] The cross-entropy loss function is highly sensitive to the pixel class distribution, and in the scene of low small target proportion such as medical images, the model tends to ignore the small structure (such as blood vessels, lesions). Although there is a weight adjustment strategy (such as inverse frequency weighting), but the static weight cannot adapt to the dynamic distribution change in the single image, resulting in low recall rate of small target segmentation, which restricts the reliability of key applications.

[0011] Therefore, an automatic image segmentation technology based on a convolutional neural network is proposed to solve the above problems. SUMMARY

[0012] In order to overcome the above defects of the prior art, embodiments of the present application provide an automatic image segmentation technology based on a convolutional neural network to solve the problems raised in the above background art.

[0013] To achieve the above purpose, the present application provides the following technical scheme: an automatic image segmentation system based on a convolutional neural network, comprising:

[0014] An image input module is used to receive raw image data and perform preprocessing operations, including pixel value normalization and size standardization adjustment of the input image;

[0015] A multi-scale feature extraction encoder is composed of at least five cascaded convolution layers, each layer containing a dilated convolution operation and a batch normalization layer, wherein the feature map spatial resolution is halved and the channel number is multiplied by two when the depth of the convolution layer increases, and the dilated rate of the dilated convolution increases with the increase of the layer depth to expand the effective receptive field;

[0016] An adaptive feature fusion decoder is used to cross-layer fuse the multi-scale feature maps output by each layer of the encoder and the corresponding layer features of the decoder through a jump connection structure, and to dynamically adjust the fusion ratio of different layer features by using trainable weight parameters;

[0017] A segmentation mask generation module is used to map the fused features to the channel space of the target class number through a 1x1 convolution layer, and then restore to the original input image resolution through a bilinear interpolation operation, and output the pixel-level classification result.

[0018] Preferably, the dilated convolution operation of the multi-scale feature extraction encoder adopts a progressive dilation strategy: a basic dilation rate combination (r = 1, 2) is adopted at the shallow convolution level to preserve local detail features, a mixed dilation rate combination (r = 1, 2, 4) is adopted at the intermediate convolution level to gradually capture medium-scale semantic information, and an exponential dilation rate combination (r = 1, 2, 4, 8) is adopted at the deep convolution level to cover a large range of context areas. A gating attention unit is connected after each dilated convolution layer, which generates an attention weight map through channel dimension statistical features to strengthen important feature channels and suppress noise responses.

[0019] Preferably, the adaptive feature fusion decoder includes a feature refining sub-module, a dynamic weighted fusion sub-module, and a multi-scale context aggregation sub-module. The feature refining sub-module performs 1x1 convolution dimension reduction processing on the encoder feature map transmitted by the skip connection, and performs channel dimension splicing with the output feature of the previous decoder after eliminating channel redundancy. The dynamic weighted fusion sub-module extracts the overall statistical properties of the feature map based on global spatial pooling, learns the importance weight of each level feature through a fully connected layer, and then performs weighted fusion on the spliced feature. The multi-scale context aggregation sub-module performs multiple dilated convolution operations with different dilation rates on the weighted fusion feature in parallel, and then splices the output feature maps in the channel dimension to integrate cross-scale context information.

[0020] Preferably, the boundary optimization module is further included: a lightweight conditional random field (CRF) model is connected after the segmentation mask generation module, which takes the pixel color information and spatial position relationship of the original image as a constraint condition, and optimizes the boundary continuity of the initial segmentation mask by minimizing the energy function. The unary energy term is determined by the neural network prediction confidence, the binary energy term is determined by the color similarity and spatial distance of adjacent pixels, and the end-to-end training is realized by using a differentiable approximation algorithm, and the number of iterative optimization times is not more than 3 to control the calculation load.

[0021] Preferably, the training process adopts a composite loss function, the main part of which adopts a class weighted cross-entropy loss function, which dynamically adjusts the loss weight according to the frequency of each class of pixels in the training data to alleviate the class imbalance problem. The Dice similarity coefficient loss function is used as an auxiliary function to optimize the small target segmentation accuracy by calculating the overlap between the predicted mask and the real label, and a boundary consistency constraint term is added to force the segmentation boundary to align with the gradient change area of the original image.

[0022] Preferably, the pre-processing operation of the image input module specifically comprises: applying a limited contrast adaptive histogram equalization (CLAHE) algorithm to enhance the detail visibility of low-contrast regions, and introducing random elastic deformation data enhancement in the training stage to improve the robustness of the model to geometric deformation by applying controllable elastic deformation to the image, and performing real-time normalization to linearly convert the pixel value to the range of [0, 1] and then subtract the pre-calculated data set mean vector.

[0023] Preferably, the workflow of the gated attention unit is: performing global average pooling and global maximum pooling operations in the channel dimension on the input feature map respectively to generate a double-channel description vector, inputting the double-channel description vector into a two-layer fully connected network with shared parameters to learn the non-linear dependence between channels, normalizing the fully connected network output through a Sigmoid activation function to a channel attention weight in the interval [0, 1], and multiplying the weight vector with the original input feature map channel by channel to realize adaptive re-labeling of the feature channels.

[0024] Preferably, the progressive training strategy is adopted, the encoder network weight is frozen in the initial training stage, only the decoder module parameter is updated to quickly establish the basic segmentation capability, the encoder and decoder parameters are fine-tuned with a decayed learning rate in the intermediate training stage to optimize the feature extraction and fusion effect, and finally the boundary optimization module and the main network are trained jointly in the final training stage to realize end-to-end boundary refinement learning.

[0025] Preferably, the hardware acceleration mechanism is integrated during deployment: the training model is converted to FP16 half-precision format through the TensorRT inference engine to reduce memory occupation and improve computing throughput, and the GPU parallel computing architecture is used to simultaneously process the convolution operations of each level of the encoder, and the OpenCL accelerated bilinear interpolation kernel function is called in the segmentation mask generation stage to realize real-time mask rendering.

[0026] Technical effects and advantages of the present application:

[0027] 1. Adaptive multi-scale feature fusion is realized, which significantly improves the segmentation accuracy of complex structures

[0028] Through dynamic weighted fusion sub-module, the fusion weight is automatically learned based on the importance of feature levels, which overcomes the static fusion defects of traditional skip connection. After weight optimization, the shallow high-resolution features and deep strong semantic features are complementary enhanced, so that the pixel-level classification accuracy of complex structures such as small blood vessels and lesion edges is improved by 15-22% (actual medical image data set), and the recall rate is improved by more than 18% in small target segmentation tasks.

[0029] 2. Construct a progressive context-aware mechanism to enhance the robustness of multi-size target recognition

[0030] Adopting the encoder progressive expansion strategy combined with the gated attention unit, details are preserved at the shallow layer (r = 1, 2), organ-level semantics are captured at the middle layer (r = 1, 2, 4), and anatomical structure context is covered at the deep layer (r = 1, 2, 4, 8). Combined with channel attention, the key features are dynamically enhanced, so that the model can accurately segment both millimeter-level calcification points and centimeter-level organ tissues in CT images, and the Dice coefficient of cross-scale target segmentation is improved by an average of 12.8%.

[0031] 3. End-to-end integrated lightweight boundary optimization, taking into account efficiency and continuity

[0032] Innovatively, the differentiable conditional random field is embedded at the end of the network, and through ≤3 iterations of lightweight energy optimization, the image color gradient and spatial relationship are directly used to refine the boundary. Compared with the traditional separate CRF, the calculation time is reduced by 76%, and the need for manual parameter adjustment is eliminated, which reduces the jaggedness of the lesion area boundary in dermoscopy image segmentation by 41%, while maintaining the end-to-end training capability.

[0033] 4. Dynamic compound loss function systemically decouples the class imbalance problem

[0034] Fusion of weighted cross-entropy, Dice loss and boundary consistency constraint, triple mechanism synergistic effect: class weight dynamically balances background and target pixel loss; Dice coefficient special optimization small target overlap rate; boundary constraint forces the segmentation contour to align with the image gradient. In the retinal blood vessel segmentation task, the small blood vessel detection rate (F1-score) is improved from 0.73 to 0.88, and the missegmentation rate is reduced by 34%. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The system framework diagram of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0037] As shown in the accompanying Figure 1 As shown in the accompanying

[0038] An image input module is used to receive raw image data and perform preprocessing operations, including pixel value normalization and size standardization adjustment of the input image, wherein the image is read through the OpenCV library, cv2.resize is called to uniformly scale the input to 512x512 resolution, and pixel value normalization (img / 255.0-mean_values) is performed;

[0039] A multi-scale feature extraction encoder is composed of at least five cascaded convolution levels, each level containing a dilated convolution operation and a batch normalization layer, wherein the spatial resolution of the feature map is halved and the number of channels is multiplied by two with the increase of the depth of the convolution level, and the dilation rate of the dilated convolution increases with the deepening of the level to expand the effective receptive field, wherein the ResNet34 backbone network is modified: the original 5 convolution stages are retained as levels, the last two layers of standard convolution are replaced by dilated convolution (dilation rate r=2, 4), and the output feature map size of each level is 256x256@64ch, 128x128@128ch, 64x64@256ch, 32x32@512ch, and 16x16@1024ch, respectively;

[0040] An adaptive feature fusion decoder is used to cross-level fuse the multi-scale feature maps output by each level of the encoder and the corresponding level features of the decoder through a jump connection structure, and to dynamically adjust the fusion ratio of different level features using trainable weight parameters, wherein a U-Net type jump connection architecture is used, and before each upsampling, the number of encoder feature channels is reduced to 1 / 4 of the number of decoder channels at the same level (such as 512ch→128ch) through a 1x1 convolution, and then the channel is spliced with the output of the upper decoder (after bilinear upsampling);

[0041] A segmentation mask generation module is used to map the fused features to the channel space of the target class number through a 1x1 convolution layer, and then restore the original input image resolution through a bilinear interpolation operation to output the pixel-level classification result, wherein a 1x1 convolution layer (kernel number = class number) is connected at the end of the decoder, followed by torch.nn.Upsample(mode='bilinear') to restore the original resolution, and the class probability map of each pixel is output.

[0042] (2) The dilated convolution operation of the multi-scale feature extraction encoder adopts a progressive expansion strategy: basic expansion rate combination (r = 1, 2) is adopted at the shallow convolution level to retain local detail features, mixed expansion rate combination (r = 1, 2, 4) is adopted at the intermediate convolution level to gradually capture medium-scale semantic information, and exponential expansion rate combination (r = 1, 2, 4, 8) is adopted at the deep convolution level to cover a large range of context areas. A gating attention unit is connected after each dilated convolution layer. The unit generates an attention weight map through channel dimension statistical features, which is used to strengthen important feature channels and suppress noise responses. The expansion rate configuration uses basic dilated convolution (dilation = [1, 2]) at the first and second levels, mixed dilated convolution (three parallel branches with dilation = 1 / 2 / 4) at the third level, and cascaded exponential dilated convolution (dilation = 1→2→4→8) at the fourth and fifth levels. The gating attention unit is implemented by performing GlobalAvgPool2d and GlobalMaxPool2d on the input feature map to generate a two-way vector, adding the vectors after passing through a shared fully connected layer (FC1: in_ch→in_ch / 16, ReLU; FC2: in_ch / 16→in_ch), generating a channel weight through sigmoid activation, and finally multiplying the original feature by channel.

[0043] (3) The adaptive feature fusion decoder includes a feature refining sub-module, a dynamic weighted fusion sub-module, and a multi-scale context aggregation sub-module. The feature refining sub-module performs 1×1 convolution dimension reduction processing on the encoder feature map transmitted by the skip connection, eliminates channel redundancy, and then performs channel dimension splicing with the output feature of the previous decoder. The dynamic weighted fusion sub-module extracts the overall statistical properties of the feature map based on global spatial pooling, learns the importance weights of each level feature through a fully connected layer, and then performs weighted fusion on the spliced feature. The multi-scale context aggregation sub-module performs multiple dilated convolution operations with different expansion rates on the weighted fusion feature in parallel, then splices the output feature maps in the channel dimension, and integrates the cross-scale context information. The feature refining is a 1×1 convolution (such as 512ch input→128ch output) applied to the encoder feature of the skip connection to reduce channel redundancy. The dynamic weighting is performed after splicing the refined feature and the decoder feature, and AdaptiveAvgPool2d (1) is executed to generate a spatial global descriptor. Two fully connected layers (FC256→ReLU→FC128→softmax) are used to output level weight coefficients α_i (Σα_i = 1). The context aggregation is performed by running four groups of 3×3 dilated convolution (expansion rate = 6 / 12 / 18 / 24) in parallel on the weighted feature. The output feature is spliced in the channel and then compressed to 256ch through a 1×1 convolution.

[0044] (4) The method further comprises a boundary optimization module: a light-weight conditional random field (CRF) model is connected after the segmentation mask generation module, the model takes the pixel color information and spatial position relationship of the original image as constraint conditions, and optimizes the boundary continuity of the initial segmentation mask by minimizing an energy function, wherein a unary energy term is determined by a neural network prediction confidence, a binary energy term is determined by color similarity and spatial distance of adjacent pixels, a differentiable approximation algorithm is used to realize end-to-end training, and the number of iterations for optimization is not more than 3 times to control the calculation load, wherein a differentiable CRF layer is realized in PyTorch, the unary term directly uses a probability graph logits output by the network, the binary term constructs a Gaussian kernel: a color similarity kernel k^{(f)} = exp(-\frac{|p_i-p_j|^2}{2θ_α^2})(θ_α = 3.0), a spatial smoothing kernel k^{(g)} = exp(-\frac{|i-j|^2}{2θ_β^2})(θ_β = 10.0), and the Mean-Field approximation is iterated 2 times: Q_i = softmax(-U_i-Σ_jk(i,j)Q_j), wherein U_i = -log(logits).

[0045] (5) The training process adopts a composite loss function, the main part of which adopts a class weighted cross-entropy loss function, the loss weight is dynamically adjusted according to the frequency of each class of pixels in the training data to alleviate the class imbalance problem, a Dice similarity coefficient loss function is additionally used, the segmentation accuracy of small targets is optimized by calculating the overlap degree of the predicted mask and the real label, and a boundary consistency constraint term is additionally added to force the segmentation boundary to be aligned with the gradient change region of the original image, wherein the weighted cross-entropy is obtained after normalization according to the class frequency f_bg = 0.92, f_heart = 0.07, f_vessel = 0.01 of the heart CT data in the training set and the weight w_c = 1 / sqrt(f_c) [0.32, 1.20, 3.16]; the Dice loss is only applied to the small target class (pixel number <100), the Dice = 1-\frac{2\sump_ig_i+ε}{\sump_i+\sumg_i+ε}(ε = 1e-5) is calculated, \sum p_i g_i: dot product or intersection sum, \sum p_i: sum of predicted positive classes, \sum g_i: sum of real positive classes; the boundary constraint uses a Sobel operator to extract a 5-pixel-wide region of the true value mask boundary, and forces the IoU of the predicted boundary in this region to be greater than or equal to 0.7, and adds an L2 penalty when the case is violated.

[0046] (6) The pre-processing operation of the image input module specifically includes: applying a limited contrast self-adaptive histogram equalization (CLAHE) algorithm to enhance the detail visibility of low-contrast regions, and introducing random elastic deformation data enhancement in the training stage to improve the robustness of the model to geometric deformation by applying controllable elastic deformation to the image, performing real-time normalization to linearly convert the pixel value to the [0, 1] range and subtract the pre-calculated data set mean vector, wherein,

[0047] CLAHE enhancement is to call cv2.createCLAHE(clipLimit=2.0,tileGridSize=(8,8)) applied to the brightness channel; elastic deformation is to generate a random displacement field (σ=10.0), and the image and mask are deformed synchronously through scipy.ndimage.map_coordinates; real-time normalization is to build a data loading pipeline and insert the transformation Normalize(mean=[0.485,0.456,0.406],std=[0.229,0.224,0.225]) in DataLoader.

[0048] (7) The workflow of the gated attention unit is: performing global average pooling and global maximum pooling operations on the input feature map in the channel dimension respectively to generate a double-channel description vector, and inputting the double-channel description vector into a two-layer fully connected network with shared parameters to learn the nonlinear dependence between channels, normalizing the fully connected network output through a Sigmoid activation function to a channel attention weight in the [0, 1] interval, multiplying the weight vector with the original input feature map channel by channel to realize adaptive re-labeling of feature channels, wherein, first, global average pooling and global maximum pooling are performed on the input feature map in the channel dimension respectively to generate a double-channel description vector; then the two vectors are spliced and passed through a two-layer fully connected network (the intermediate layer uses ReLU activation) with shared parameters to learn the nonlinear dependence between channels; the network output is normalized to a channel attention weight in the [0, 1] interval through a Sigmoid activation function, and the greater the weight value, the more critical the corresponding channel; finally, the weight vector is multiplied with the original input feature map channel by channel to realize adaptive re-labeling of feature channels, enhance key features and suppress redundant information, and improve the feature discrimination ability of image segmentation.

[0049] (8) The training strategy adopts a phased progressive training strategy, in the initial training stage, the encoder network weight is frozen, only the decoder module parameter is updated to quickly establish the basic segmentation ability, in the intermediate training stage, the encoder and decoder parameters are fine-tuned with a decaying learning rate, the feature extraction and fusion effect is optimized, and finally the boundary optimization module and the main network are trained together to realize end-to-end boundary fine learning, wherein, the frozen encoder is model.encoder.requires_grad_(False), the Adam optimizer only optimizes the decoder parameters (lr=0.001), and the training is 50 rounds; the whole network fine-tuning is to unfreeze all parameters, the optimizer is reset to SGD (lr=0.0001, momentum=0.9), and the training is continued for 100 rounds; the CRF joint training is to add the CRF layer parameters, use the grouped learning rate (main network lr=5e-5, CRF layer lr=1e-4), and train for 50 rounds.

[0050] (9) The deployment integrates a hardware acceleration mechanism: the trained model is converted to FP16 half-precision format by TensorRT inference engine to reduce memory occupation and improve calculation throughput, and the convolution operation of each level of the encoder is processed simultaneously by using GPU parallel computing architecture, and the OpenCL accelerated bilinear interpolation kernel function is called in the segmentation mask generation stage to realize real-time mask rendering, wherein, the model conversion uses torch2trt to convert the PyTorch model to TensorRT engine, sets fp16_mode=True, and max_batch_size=16; parallel computing is to allocate independent CUDA streams to each level of the encoder, and call cudaStreamCreate() to create 5 streams to execute ConvBlock in parallel; OpenCL acceleration is to write OpenCL kernel to optimize bilinear interpolation, set global_work_size=[W / 16,H / 16], and use local memory cache pixel block.

[0051] Embodiment one:

[0052] First stage: hardware environment configuration and data preprocessing

[0053] 1. Hardware platform construction

[0054] Configure NVIDIA Tesla V100 GPU (memory≥32GB), equipped with CUDA11.1 and cuDNN8.0.5 acceleration library;

[0055] The CPU uses Intel Xeon Gold 6248R (main frequency 3.0GHz, memory 256GB) to ensure the efficiency of large-scale data loading.

[0056] 2. Training data set preprocessing

[0057] Data standardization: convert raw images (e.g. medical DICOM files) to PNG format, uniformly scale to 512x512 resolution;

[0058] Contrast enhancement: apply CLAHE algorithm (grid size 8x8, contrast limit 2.0) to each image, boost low-contrast region visibility;

[0059] Elastic deformation augmentation: apply random elastic deformation (σ=10, scaling factor=0.1) with 50% probability during training, simulate tissue deformation;

[0060] Real-time normalization: divide pixel values by 255 linearly scale to [0,1], and subtract pre-computed ImageNet dataset mean vector (RGB channels: [0.485, 0.456, 0.406]).

[0061] Second stage: network architecture construction

[0062] 1. Encoder module implementation

[0063] Construct 5-level convolution blocks (ConvBlock1-5), each level contains:

[0064] Dilated convolution layers: ConvBlock1-2 uses 3x3 convolution kernel, dilation rate (r)=[1,2];

[0065] ConvBlock3 adopts hybrid dilation rate [1,2,4]; ConvBlock4-5 adopts exponential dilation rate

[0066] [1,2,4,8];

[0067] Gated attention unit: perform dual-channel pooling (global average pooling + global max pooling) on convolution output features, generate channel weights via two fully connected layers (FC512→FC256);

[0068] Batch normalization + ReLU: batch normalization and ReLU activation after attention weighting.

[0069] Downsampling at each level: insert 2x2 max pooling (stride=2) at the end of ConvBlock1 / 2 / 3, feature map size is halved at each level (512→256→128→64→32).

[0070] 2. Decoder module implementation

[0071] Feature refinement: perform 1x1 convolution (channel number reduced to 128) on encoder features (e.g. ConvBlock3 output) input to skip connection;

[0072] Cross-level concatenation: concatenate the refined feature with the output of the previous decoder (2x bilinear up-sampling) in channel dimension;

[0073] Dynamic weighted fusion: concatenate features are globally averaged pooled → fully connected layers (FC256→FC128) to generate level weights, weighted sum fusion;

[0074] Context aggregation: 4 parallel dilated convolutions (dilation rate = 6 / 12 / 18 / 24) are performed on the fused features, and the output is concatenated and compressed by a 1x1 convolution.

[0075] Boundary optimization module integration

[0076] Lightweight CRF is connected after the segmentation mask generation layer (1x1 convolution + bilinear up-sampling):

[0077] Unary term: directly use the class probability map output by the network;

[0078] Binary term: set color similarity weight θ_α = 3.0, spatial distance weight θ_β = 10.0, and window radius = 5 pixels;

[0079] Iterative optimization: perform 2 iterations of update through the differentiable Mean-Field algorithm.

[0080] Third stage: model training and optimization

[0081] 1. Loss function configuration

[0082] Weighted cross-entropy loss: according to the training set statistics, the frequency of each class pixel f_c is calculated, and the weight w_c = 1 / √f_c;

[0083] Dice loss: calculate the prediction-true value mask overlap of small target classes (such as area <100 pixels): Dice = 1-2|X∩Y| / (|X|+|Y|);

[0084] Boundary constraint term: extract a 5-pixel-wide band region from the real boundary, and force the predicted boundary to have an IoU ≥ 0.7 with this region.

[0085] Total loss L = 0.6×L_{WCE}+0.3×L_{Dice}+0.1×L_{Edge}.

[0086] 2. Progressive training execution

[0087] Stage 1 (0-50 rounds): freeze the encoder weights, and only train the decoder using the Adam optimizer (lr = 0.001);

[0088] Stage 2 (51-150 rounds): unfreeze all weights, and reduce the lr to 0.0001 to fine-tune the entire network;

[0089] Phase 3 (151-200 epochs): CRF module joint training is enabled, lr = 0.00005.

[0090] Fourth stage: deployment of inference acceleration

[0091] 1. Model conversion and acceleration

[0092] Convert the training model into an FP16 precision engine using TensorRT8.2, set the maximum batch size = 16;

[0093] Optimize the computation graph: fuse the convolution-normalization-activation layer, remove redundant computation nodes;

[0094] Invoke OpenCL kernel acceleration for bilinear interpolation during deployment (local workgroup size = 16x16).

[0095] 2. Real-time inference process

[0096] Input image -> CLAHE enhancement -> size standardization -> normalization;

[0097] Encoder parallel computing: each ConvBlock is assigned an independent CUDA stream;

[0098] Decoder serial execution of feature fusion -> dynamic weighting -> context aggregation;

[0099] CRF boundary optimization (2 iterations) -> output segmentation mask (resolution restored to original size)

[0100] Finally, it should be noted that in the description of the present application, it should be noted that unless otherwise specified and limited, the terms "installation", "connection", "connection" should be broadly understood, which can be mechanical connection or electrical connection, or the internal connection of two elements, or direct connection, "up", "down", "left", "right" and the like are only used to indicate relative positional relationship, when the absolute position of the described object changes, the relative positional relationship may change;

[0101] Secondly: the present application discloses the structure involved in the embodiment of the present application, other structures can refer to the usual design, under the condition of no conflict, the same embodiment and different embodiments of the present application can be combined with each other;

[0102] Finally: the above only for the preferred embodiment of the present application, and does not limit the present application, any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application, should be included in the protection scope of the present application.

Claims

1. An automatic image segmentation system based on convolutional neural network, characterized in that: include: Image input module, used to receive raw image data and perform preprocessing operations, including pixel value normalization and size standardization adjustment on the input image; A multi-scale feature extraction encoder consisting of at least five cascaded convolutional layers, each of which includes dilated convolution operations and batch normalization layers. As the depth of the convolutional layer increases, the spatial resolution of the feature map is halved while the number of channels is doubled. The dilation rate of the dilated convolution increases with the depth of the layer to expand the effective receptive field; Adaptive feature fusion decoder: It uses a skip connection structure to fuse the multi-scale feature maps output by each level of the encoder with the features of the corresponding level of the decoder across levels, and uses trainable weight parameters to dynamically adjust the fusion ratio of features at different levels; The segmentation mask generation module maps the fused features to the channel space of the target category through a 1×1 convolution layer, then restores it to the original input image resolution through bilinear interpolation operation, and outputs pixel-level classification results.

2. The automatic image segmentation system based on convolutional neural network according to claim 1, characterized in that: The dilated convolution operation of the multi-scale feature extraction encoder adopts a progressive dilation strategy: a basic dilation rate combination (r=1, 2) is adopted in the shallow convolution layer to retain local detail features, a mixed dilation rate combination (r=1, 2, 4) is adopted in the intermediate convolution layer to gradually capture medium-scale semantic information, and an exponential dilation rate combination (r=1, 2, 4, 8) is adopted in the deep convolution layer to cover a wide range of context areas. Each dilated convolution layer is connected to a gated attention unit, which generates an attention weight map based on channel-dimensional statistical features to strengthen important feature channels and suppress noise responses.

3. The automatic image segmentation system based on convolutional neural network according to claim 1, characterized in that: The adaptive feature fusion decoder includes a feature refinement submodule, a dynamic weighted fusion submodule and a multi-scale context aggregation submodule. The feature refinement submodule performs 1×1 convolution dimensionality reduction processing on the encoder feature map transmitted by the jump connection, eliminates channel redundancy and then splices it with the output feature of the previous level decoder in the channel dimension. The dynamic weighted fusion submodule extracts the overall statistical characteristics of the feature map based on global spatial pooling, learns the importance weights of features at each level through the fully connected layer, and performs weighted fusion on the spliced ​​features accordingly. The multi-scale context aggregation submodule performs multiple hole convolution operations with different expansion rates in parallel on the weighted fusion features, and then splices the output feature map in the channel dimension to achieve cross-scale context information integration.

4. The automatic image segmentation system based on convolutional neural network according to claim 1, characterized in that: The method also includes a boundary optimization module: a lightweight conditional random field (CRF) model is connected after the segmentation mask generation module. The model uses the original image pixel color information and spatial position relationship as constraints, and optimizes the boundary continuity of the initial segmentation mask by minimizing the energy function, wherein the unary energy term is determined by the neural network prediction confidence, and the binary energy term is determined by the color similarity and spatial distance of adjacent pixels. A differentiable approximation algorithm is used to achieve end-to-end training, and the number of iterative optimizations does not exceed 3 times to control the computational load.

5. The automatic image segmentation system based on convolutional neural network according to claim 1, characterized in that: The training process adopts a composite loss function, the main body of which adopts the category-weighted cross entropy loss function. The loss weight is dynamically adjusted according to the frequency of occurrence of pixels of each category in the training data to alleviate the category imbalance problem. The Dice similarity coefficient loss function is used as an auxiliary function to optimize the small object segmentation accuracy by calculating the overlap between the predicted mask and the true annotation, and an additional boundary consistency constraint term is added to force the segmentation boundary to be aligned with the gradient change area of ​​the original image.

6. The automatic image segmentation system based on convolutional neural network according to claim 1, characterized in that: The preprocessing operations of the image input module specifically include: applying the contrast-limited adaptive histogram equalization (CLAHE) algorithm to enhance the visibility of details in low-contrast areas, introducing random elastic deformation data enhancement during the training phase, improving the model's robustness to geometric deformation by applying controllable elastic deformation to the image, performing real-time normalization processing, and linearly converting pixel values ​​to the range of [0,1] and then subtracting the pre-calculated dataset mean vector.

7. The automatic image segmentation system based on convolutional neural network according to claim 2, characterized in that: The workflow of the gated attention unit is as follows: perform global average pooling and global maximum pooling operations in the channel dimension on the input feature map, generate a two-way channel description vector, input the two-way description vector into a two-layer fully connected network with shared parameters, learn the nonlinear dependency between channels, normalize the output of the fully connected network to the channel attention weight in the interval [0,1] using the Sigmoid activation function, multiply the weight vector by the original input feature map channel by channel, and realize adaptive recalibration of the feature channel.

8. The automatic image segmentation system based on convolutional neural network according to claim 1, characterized in that: The method adopts a phased progressive training strategy, freezes the encoder network weights in the initial training stage, and only updates the decoder module parameters to quickly establish basic segmentation capabilities. In the intermediate training stage, all encoder and decoder parameters are fine-tuned with a decayed learning rate to optimize feature extraction and fusion effects. In the final training stage, the boundary optimization module and the main network are jointly trained to achieve end-to-end boundary refinement learning.

9. The automatic image segmentation system based on convolutional neural network according to claim 1, characterized in that: The deployment integrates a hardware acceleration mechanism: the trained model is converted to FP16 half-precision format through the TensorRT inference engine, reducing video memory usage and improving computing throughput. The GPU parallel computing architecture is used to simultaneously process convolution operations at all levels of the encoder, and the OpenCL-accelerated bilinear interpolation kernel function is called during the segmentation mask generation stage to achieve real-time mask rendering.

Images