A neural network method for image domain sparse feature segmentation is constructed based on soft threshold and CNN module alternate iteration

Abstract

The application discloses a neural network method for image domain sparse feature segmentation based on soft threshold and CNN module alternating iteration, and the core of the method is that a 1-norm and a learnable network double driving mechanism are adopted to quickly and accurately extract a target and a structure with sparse features. l 1-norm and a learnable network double driving mechanism, which can quickly and accurately extract a target and a structure with sparse features. A variational decomposition model is constructed based on target sparse representation, a deep learning network denoising module is introduced in an alternating minimization method, and an algorithm iteration process is expanded into a network. Then, the extracted sparse features are input into a subsequent connected lightweight U-Net, so that image segmentation is realized. Through retinal blood vessel segmentation and road crack detection verification, the network architecture proposed in the application has a lightweight feature, and compared with a standard U-Net, the number of parameters is smaller, and the performance can be equivalent to or even better than that of the standard U-Net.

Description

Technical Field

[0001] This invention relates to the field of image segmentation, specifically focusing on a neural network algorithm for sparse feature segmentation in the image domain, based on the iterative construction of a soft thresholding module and a CNN module. This algorithm aims to improve the efficiency and accuracy of image segmentation tasks for targets with sparse structures, providing a more efficient and accurate solution to the problem of sparse feature segmentation in images. Background Technology

[0002] Image segmentation, a crucial component of intermediate-level visual tasks, serves as a bridge between low-level visual operations and high-level visual applications. Sparse feature segmentation in the image domain is a special type of image segmentation task aimed at extracting spatially sparse targets from an image. This problem has wide applications, such as retinal vessel segmentation, road crack detection, and the discovery of subtle defects in equipment. Since the emergence of computer vision in the 1980s, numerous traditional and deep learning-based methods have been developed. Early image segmentation primarily relied on low-level image features such as pixel intensity, color, and texture, leading to traditional methods like thresholding, region growing, graph theory, and support vector machines (SVM), achieving good segmentation results. Traditional methods have clear mathematical principles, strong interpretability, and high algorithmic efficiency, but they have limitations when processing image data with complex backgrounds or noise, and most algorithm parameters require manual adjustment. In recent years, the rise of deep learning has brought revolutionary changes to image segmentation, including sparse feature segmentation. The emergence of Convolutional Neural Networks (CNNs), along with the U-Net and YOLO series algorithms, has enabled the transformation of image segmentation from traditional hand-crafted features to deep features. These modern methods have improved segmentation accuracy through multimodal fusion, multi-scale feature extraction, and self-attention mechanisms. Deep learning models can automatically learn high-level image features and achieve high-precision segmentation even in complex scenes, but their interpretability is weak, and most current neural network models have numerous parameters, high computational complexity, and are difficult to train.

[0003] To combine the advantages of traditional methods with modern deep learning methods, iterative unfolding techniques have emerged. It integrates the iterative process of traditional optimization algorithms with deep learning network structures, preserving the interpretability of traditional optimization algorithms while fully leveraging the powerful modeling capabilities of deep learning to more effectively solve various complex optimization problems. According to compressed sensing theory, L1 regularization can effectively characterize sparse signals. Recently, some work has indeed considered building neural network methods based on L1 regularization and iterative unfolding for sparse feature segmentation in the image domain. However, these algorithms rely on an iterative process without convergence guarantees and require backpropagation of a system of linear equations, making training difficult and resulting in unstable predictions.

[0004] Based on the above facts, a neural network segmentation algorithm based on alternating soft thresholding and CNN modules is proposed. By using sparse optimization and iterative expansion techniques, interpretability is increased, model parameters are reduced, and the step of solving large linear equations is eliminated, thereby improving the efficiency of model training and prediction while maintaining or improving segmentation performance. This is a key method to solve the problems of existing technologies. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing deep neural network segmentation models, which often suffer from numerous parameters, difficulty in training, and unstable predictions. This invention proposes a method based on alternating iterations of soft thresholding and CNN modules to construct a lightweight neural network model for sparse feature segmentation in the image domain. By effectively decomposing sparse image features and dense background features, the model enables a lightweight segmentation network, reducing network parameters while improving segmentation accuracy and stability.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A neural network algorithm for sparse feature segmentation in the image domain is constructed by iteratively alternating between soft thresholding and CNN modules, including the following five steps:

[0008] (a) Collect data and create datasets: Collect images and gold standard images with sparse structure; randomly divide the collected datasets into training and test sets according to a certain ratio and perform preprocessing;

[0009] (b) Establishing a decomposition model and image segmentation neural network: An iterative unfolding network (alternating between soft thresholding and CNN modules) is constructed based on a variational image decomposition model and an alternating iterative algorithm, and connected to a lightweight segmentation module to form an image segmentation neural network. The network input is the image to be segmented with sparse structural features, and the network output is the segmented sparse feature mask image;

[0010] (c) Design the loss function: Introduce mean squared error and l1 sparsity metric into the loss function;

[0011] (d) Training the network model: The image segmentation neural network constructed in step (b) is optimized on the training and validation sets using a fully supervised learning method to obtain the optimal network parameters;

[0012] (e) Testing the network model: After training, the network obtained in step (d) is tested on the test set to obtain a sparse feature segmentation mask, which is then compared with the gold standard to verify the effectiveness of the reconstruction method.

[0013] Furthermore, the present invention uses the data collected in step (a) to train the network model established in steps (b)-(d), and uses step (e) to perform sparse feature segmentation in the image domain; the specific implementation of the preprocessing of the dataset in step (a) includes normalization, random horizontal flipping, random vertical flipping, and cutting image patches, which improves training efficiency and increases the sample size.

[0014] Furthermore, the variational image decomposition model in step (b) of this invention is:

[0015]

[0016] The model takes the input image Divided into dense characteristics and sparsity features ,in It is the regularization term for dense features. The solution is obtained using an alternating iterative algorithm, with the following steps:

[0017]

[0018] The above steps can be rewritten in the form of proximity operator and soft threshold shrinkage operator.

[0019]

[0020] Next, dense features The update step is expanded into a network, that is, the neighbor operator is transformed into a learnable module. Recorded as Simultaneously, learnable parameters are introduced for the update step of sparse features. That is, the parameters are updated independently in each unfolded layer, using the same initialization. The parameters of the entire iterative unfolded network are... .

[0021] Furthermore, the network proposed in step (b) of this invention It consists of two parts: an iterative unfolding neural network at the front end and a lightweight segmentation module, miniUNet, at the back end. The iterative unfolding neural network can be divided into dense features. and sparsity features Two parts: The first part is the feature learning module. The input is the input image. and sparsity features Output dense features The second module is the soft thresholding shrinkage operator module, which takes the input image as input. and dense features Output sparse features Dense characteristics Feature learning module for sub-problems It is an M+2 layer convolutional neural network. The input layer contains one 2D convolutional layer and one activation layer. The 2D convolutional layer has 6 input channels and 32 output channels, with a total of 192 convolutional kernels. The M intermediate layers each contain one 2D convolutional layer, one batch normalization layer, and one activation layer. The 2D convolutional layer has 32 input channels and 32 output channels, with a total of 1024 convolutional kernels. The batch normalization layer has 32 channels. The output layer contains only one 2D convolutional layer with 32 input channels and 3 output channels, with a total of 96 convolutional kernels. All 2D convolutional kernels used are 3x3 in size with a stride of 1. The activation function used in all activation layers is ReLU. The backend miniUNet processes the extracted sparse features. Further processing is required. It's a lightweight UNet network that includes two downsampling steps, with 3 input channels and 16 initial feature channels. This step proposes an image segmentation network. The total input is an image with sparse structural features, and the total output is a sparse feature mask image after segmentation.

[0022] Furthermore, the loss function in step (c) of this invention is:

[0023]

[0024] in and These are the outputs of the lightweight segmentation module, namely the sparse feature mask image after segmentation, and the sparse features output by the iteratively unfolded network, respectively. The gold standard image with labels. For image segmentation network parameters, For weights; MSE represents mean squared error, defined as:

[0025]

[0026] SpaErr represents the sparse error, defined as:

[0027]

[0028] This step utilizes information from the sparse features of the image to characterize the sparsity; it minimizes the loss function. The goal is to minimize the error between the segmented sparse feature mask and the gold standard, as well as the l1 norm of the sparse features.

[0029] Furthermore, step (d) of this invention employs a fully supervised training method, using image patches with a certain type of sparse structure as network input and the corresponding gold standard of sparse feature structure as labels to train the network. After initializing the network, the ADAM algorithm is used to optimize the network parameters. The optimal network parameters are obtained after training.

[0030]

[0031] Specifically, the testing process in step (e) is implemented by feeding the data from the test set into the image segmentation network trained in step (d) one by one. The image to be segmented As the network input, the network output is the segmented sparse feature map.

[0032]

[0033] The method proposed in this invention has been compared with traditional segmentation networks, demonstrating its superiority. This invention employs an iterative design of a lightweight image segmentation network model based on alternating soft thresholding and CNN modules. Through effective decomposition of image sparse and dense features, it reduces network parameters while eliminating the need to solve large linear equation systems, thereby improving segmentation accuracy and stability, and increasing the interpretability of the network model.

[0034] Attached image content

[0035] Figure 1 This is an overall flowchart of the method proposed in this invention.

[0036] Figure 2 The iterative unfolding neural network model proposed in this invention; the CNN architecture diagram in DFE; SFE is the soft thresholding operator.

[0037] Figure 3 This is the loss function curve of the network model proposed in this invention trained on the retinal vessel segmentation test set.

[0038] Figure 4 This is a comparison of the results of the network model proposed in this invention with the traditional UNet segmentation network on the retinal vessel segmentation test set.

[0039] Figure 5 This is a graph showing the change in the loss function of the network model proposed in this invention during training on a road crack detection test set.

[0040] Figure 6 This is a comparison of the results of the network model proposed in this invention with the traditional UNet segmentation network on the road crack detection test set. Detailed Implementation

[0041] The objectives, technical solutions, and advantages of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] This invention proposes a lightweight image segmentation network model based on an iterative design of soft thresholding and CNN modules. Through effective decomposition of sparse and dense image features, the subsequent segmentation network becomes lightweight, reducing network parameters while improving the segmentation accuracy of sparse structural features. Specifically, it integrates the advantages of mathematical modeling and data-driven methods, incorporating mathematical image sparse priors into the segmentation network structure for image feature extraction, significantly improving the efficiency and accuracy of image segmentation. The overall flowchart of this invention is shown below. Figure 1 As shown, the specific steps include:

[0043] (a) Data collection: Collect images with sparse feature structures that include the gold standard. Randomly divide the collected images with sparse feature structures into training set and test set according to a certain proportion and perform preprocessing. Preprocessing methods include normalization, random horizontal flipping, random vertical flipping, and image patch cutting.

[0044] (b) Model building: according to Figure 2 The structure shown is an iterative unfolding neural network (alternating between soft thresholding and CNN modules), which is then connected to a lightweight segmentation module to form an image segmentation network. Its input is the image to be segmented, which has sparse structural features, and the network output is a segmented sparse feature mask. The image segmentation network can be divided into two parts: a front-end and a back-end. The front-end contains the iterative unfolding neural network for sparse feature extraction; the back-end is the lightweight segmentation module for outputting the sparse feature mask.

[0045] In an embodiment of the present invention, an iterative unfolding network (alternating between soft thresholding and CNN modules) is constructed based on image decomposition and alternating iterative algorithms, and then connected to a lightweight segmentation network to form an image segmentation network. The network input is the image to be segmented with sparse structural features, and the network output is the segmented sparse feature mask image.

[0046] In an embodiment of the present invention, the variational image decomposition model is as follows:

[0047]

[0048] The model takes the input image Divided into dense characteristics and sparsity features ,in It is the regularization term for dense features. The solution is obtained using an alternating iterative algorithm, with the following steps:

[0049]

[0050] The above steps can be rewritten in the form of proximity operator and soft threshold shrinkage operator.

[0051]

[0052] Next, dense features The update step is expanded into a network, that is, the neighbor operator is transformed into a learnable module, denoted as . Recorded as Simultaneously, learnable parameters are introduced for the update step of sparse features. That is, the parameters are updated independently in each unfolded layer, but with the same initialization. The parameters of the entire iterative unfolded network are... .

[0053] In the embodiments of the present invention, the proposed image segmentation network It includes a front-end iterative unfolding neural network and a back-end lightweight segmentation module, miniUNet. The iterative unfolding neural network can be divided into dense feature... and sparsity features Two parts: The first part is the feature learning module. The input is the input image. and sparsity features Output dense features The second module is the soft thresholding shrinkage operator module, which takes the input image as input. and dense features Output sparse features Dense characteristics Learning Module for Subproblems It is an M+2 layer convolutional neural network. The input layer contains one 2D convolutional layer and one activation layer. The 2D convolutional layer has 6 input channels and 32 output channels, with a total of 192 convolutional kernels. The M intermediate layers each contain one 2D convolutional layer, one batch normalization (BN) layer, and one activation layer. The 2D convolutional layer has 32 input channels and 32 output channels, with a total of 1024 convolutional kernels, and the batch normalization layer has 32 channels. The third layer contains only one 2D convolutional layer with 32 input channels and 3 output channels, with a total of 96 convolutional kernels. All the 2D convolutional kernels used above are 3x3 in size with a stride of 1. The activation function used in all activation layers is ReLU. The backend miniUNet processes the extracted sparse features. Further processing results in a lightweight UNet network, incorporating two downsampling operations, with 3 input channels and 16 initial feature channels. This step presents the proposed image segmentation network. The total input is an image with sparse structural features, and the total output is a sparse feature mask image after segmentation.

[0054] (c) Design the loss function: First, give the mean squared error (MSE), the l1 norm of the sparse features, etc. Definition of )

[0055]

[0056]

[0057] SpaErr represents sparse error. Then, the weight hyperparameters in the loss function are appropriately selected. The loss function is obtained.

[0058]

[0059] in and These are the outputs of the lightweight segmentation module, namely the sparse feature mask image after segmentation, and the sparse features output by the iteratively unfolded network, respectively. The gold standard image with labels. For network parameters, 1 represents the weight.

[0060] This step utilizes information from the sparse features of the image to characterize the sparsity; it minimizes the loss function. The goal is to minimize the error between the segmented sparse feature mask and the gold standard, as well as the l1 norm of the sparse features.

[0061] (d) Model Training: The image segmentation network constructed in step (b) is optimized on the training set using a fully supervised learning method to obtain the optimal network parameters. Specifically, image patches with a certain type of sparse structure are used as network input, and the corresponding gold standard of sparse feature structure is used as the label for network training. After initializing the network, the ADAM algorithm is used to optimize the network parameters. The optimal network parameters are obtained after training.

[0062]

[0063] In an embodiment of the present invention, the Kaiming method is used to initialize network parameters. The network input consists of image patches from the dataset, and the output consists of sparse feature image patches and their mask images. The ADAM algorithm is used to optimize the network parameters. The hyperparameters during training are shown in Table 1, where bs is the batch size and ps is the image patch size. , The learning rate for the front-end iterative expansion network and the back-end lightweight segmentation module, where N is the number of training iterations.

[0064] Table 1 Network training hyperparameters

[0065]

[0066] (e) Testing the model: Feed the data from the test set into the network trained in (d) one by one. The image to be segmented As input to the network, a segmented sparse feature mask image is obtained.

[0067]

[0068] The sparse feature mask image is compared with the gold standard, and the evaluation method includes AUC.

[0069] To more clearly illustrate the feasibility and superiority of the present invention, the following shows the performance of the method of the present invention on two datasets: retinal vessel segmentation and road crack detection, and compares it numerically with the methods of miniUNet (2 downsampling layers, 16 initial channels) and conventional-sized UNet segmentation network (3 downsampling layers, 32 initial channels).

[0070] Example 1: Retinal vessel segmentation

[0071] Figure 3 The convergence of the optimization algorithm is illustrated by the loss function variation curve during the training of the proposed network model on the retinal vessel dataset. Table 2 quantitatively compares the parameter count and AUC of the proposed model with the traditional UNet model on the Drive dataset.

[0072] Table 2 Comparison of Network Model Parameter Quantity and AUC Numerical Results

[0073]

[0074] The effect of the reconstruction result of this invention is as follows: Figure 4 As shown in the figure. Columns 1 to 5 represent the original image, the gold standard (label), the segmentation result of the model of this invention, and the segmentation results of miniUNet and conventional-sized UNet networks, respectively. It can be seen that the segmentation result of the model of this invention is better, with no pseudo-vessels and close to real blood vessels.

[0075] Example 2: Road Crack Detection

[0076] Figure 5 The convergence of the optimization algorithm is illustrated by the loss function variation curves during the training of the network model proposed in this invention on the road crack detection dataset. Table 3 quantitatively compares the parameter count and AUC of the proposed model with the traditional Unet model on the public CrackTree dataset.

[0077] Table 3 Comparison of Network Model Parameter Quantity and AUC Numerical Results

[0078]

[0079] The effect of the reconstruction result of this invention is as follows: Figure 6 As shown. Columns 1 to 5 are the original image, the gold standard (label), the segmentation result of the model of this invention, and the segmentation results of miniUNet and conventional-sized UNet networks, respectively. It can be seen that the segmentation result of the model of this invention is better, and the cracks in the road are more coherent.

[0080] The core technical point of this invention lies in its network architecture, which cleverly integrates the advantages of mathematical modeling and data-driven methods. It incorporates sparse prior mathematical images into the segmentation network structure for image feature extraction, and by using alternating iterations, it avoids the step of solving large linear equation systems, significantly improving the efficiency and accuracy of image segmentation. Through the verification in Examples 1 and 2 on retinal vessel segmentation in medical image processing and road crack detection in industrial anomaly recognition, the proposed network architecture demonstrates its lightweight characteristics. Compared to the standard U-Net, it has fewer parameters yet achieves comparable or even superior performance.

Claims

1. A neural network method for sparse feature segmentation in the image domain, based on alternating soft thresholding and CNN modules, is characterized by: The steps include the following: (a) Collect data and create datasets: Collect images and gold standard images with sparse structure; randomly divide the collected datasets into training and test sets according to the proportions and perform preprocessing; (b) Establishing the decomposition model and image segmentation network: Based on the variational image decomposition model and the alternating minimization method, an iterative unfolding network is constructed, and a lightweight segmentation module is subsequently connected to form an image segmentation network. The input of the image segmentation network is the image to be segmented with sparse structural features, and the output is the segmented sparse feature mask image; (c) Design the loss function: Introduce mean squared error and l1 sparsity metric function into the loss function; (d) Training the network model: The image segmentation network model constructed in step (b) is optimized on the training set using a fully supervised learning method to obtain the optimal network parameters; (e) Testing the network model: After training, the iteratively unfolded network obtained in step (d) is tested on the test set to obtain a sparse feature segmentation mask image and compare it with the gold standard.

2. The method for constructing a neural network for sparse feature segmentation in the image domain based on alternating soft thresholding and CNN modules according to claim 1, characterized in that: The network model established in steps (b)-(d) is trained using the data collected in step (a), and sparse feature segmentation in the image domain is performed using step (e). The preprocessing methods for the collected data in step (a) include normalization, random horizontal flipping, random vertical flipping, and image patch cutting.

3. The method for constructing a neural network for sparse feature segmentation in the image domain based on alternating soft thresholding and CNN modules according to claim 1, characterized in that: The network proposed in step (b) comprises a front-end iterative unfolding neural network and a back-end lightweight U-Net network. The iterative unfolding neural network is divided into a feature learning module and a soft thresholding shrinkage operator module; wherein the input of the feature learning module is the input image. and sparsity features Output dense features The soft threshold shrinkage operator module takes the input image as its input. and dense features Output sparse features The total input of the proposed network is the image to be segmented, and the total output is a sparse feature segmentation map.

4. The method for constructing a neural network for sparse feature segmentation in the image domain based on alternating soft thresholding and CNN modules according to claim 3, characterized in that, The feature learning module is an M+2 layer convolutional neural network. The input layer contains one 2D convolutional layer and one activation layer. The 2D convolutional layer has 6 input channels and 32 output channels, for a total of 192 convolutional kernels. There are M intermediate layers, each containing one 2D convolutional layer, one batch normalization layer, and one activation layer. The 2D convolutional layers have 32 input channels and 32 output channels, for a total of 1024 convolutional kernels, and the batch normalization layer has 32 channels. The output layer contains only one 2D convolutional layer with 32 input channels and 3 output channels, for a total of 96 convolutional kernels. All 2D convolutional kernels used are 3x3 in size with a stride of 1. The activation function for all activation layers is ReLU. The network backend processes the extracted sparse features. Further processing was performed using a lightweight UNet network, which included two downsampling steps, with 3 input channels and 16 initial feature channels.

5. The method for constructing a neural network for sparse feature segmentation in the image domain based on alternating soft thresholding and CNN modules according to claim 1, characterized in that: The loss function in step (c) is in and These are the outputs of the lightweight segmentation module, namely the sparse feature image after segmentation, and the sparse features output by the iteratively unfolded network, respectively. The gold standard image with labels. For network parameters, 1 represents the weight; MSE stands for Mean Squared Error, defined as: SpaErr represents the sparse error, defined as: The sparsity of an image is characterized by information about its sparse features.

6. The method for constructing a neural network for sparse feature segmentation in the image domain based on alternating soft thresholding and CNN modules according to claim 1, characterized in that: Step (d) employs a fully supervised training method, using image patches with a certain type of sparse structure as network input and the corresponding gold standard for sparse structure segmentation as labels to train the network. After initializing the network, the ADAM algorithm is used to optimize the network parameters, and the optimal network parameters are obtained after training.

Images