Deep learning network regularization method based on image local feature enhancement

Abstract

The invention relates to a deep learning network regularization method based on image local feature enhancement, and belongs to the field of deep learning computer image processing. Compared with an existing image data augmentation method technology, such as pure noise adding, rotation, amplification, reduction and other technologies, the image data augmentation method is provided. According to the technology, an image local region interchange strategy is used, different region interchange strategies are used for different images, the regularization effect of region loss is achieved, meanwhile, compared with a traditional region discarding method, training pixel information is more effectively utilized, and therefore effective expansion of a data sample set is achieved; and the generalization ability and robustness of the deep learning algorithm are further improved.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and computer image processing. More specifically, embodiments of the present invention relate to a deep learning network regularization method based on image local feature enhancement. Background Art

[0002] Deep convolutional neural networks (CNNs) have demonstrated promising performance on computer vision problems such as image classification, object detection, semantic segmentation, and video analysis. Several training strategies, including data augmentation and regularization techniques, have been proposed to further improve training efficiency and performance. To prevent CNNs from focusing too heavily on a small region of the input image, random feature removal regularization has been proposed. For example, strategies such as region dropout are used to erase random regions of the input. Feature removal strategies improve generalization performance by allowing the model to focus not only on the most discriminative parts of an object, but also on the entire object region.

[0003] However, while region dropout strategies improve classification and localization performance to a certain extent, the dropped regions are often zeroed out or filled with random noise, significantly reducing the proportion of informative pixels in the training image. Therefore, it is crucial to develop deep learning data augmentation strategies that maximize the use of dropped regions while making deep learning training more efficient and generalizing more effectively. Summary of the Invention

[0004] A brief overview of the present invention is provided below to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important aspects of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is simply to present certain concepts in a simplified form as a prelude to the more detailed description discussed later.

[0005] In view of this, in order to overcome the above technical problems, the present invention provides a deep learning network regularization method based on image local feature enhancement, which effectively utilizes training pixels and improves the regularization effect of regional loss by cutting and pasting blocks between training images, where the real labels are also mixed proportionally according to the area of the blocks.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] Solution 1: The present invention provides a deep learning network regularization method based on image local feature enhancement. Specifically, it includes the following steps:

[0008] Step 1: Get the sample image information in the dataset, let x∈Rw×H×C and y represent the training image and its label respectively.

[0009] Step 2: By combining two training samples (x A ,y A ) and (x B ,y B ) to generate new training samples (~x, ~y).

[0010]

[0011] M∈{0,1} W×H represents the binary mask indicating the removal and padding from the two images, 1 is the binary mask filled with 1, and e is the element-wise multiplication. The combination ratio λ between the two data points is sampled from a Beta distribution (α, α). In all our experiments, we set α to 1, i.e., sample λ from a uniform distribution (0, 1).

[0012] Step 3: Sample the binary mask M, first of all, the representation x A and x B The bounding box coordinates of the cropping area on B = (r x , r y , r w , r h ) for sampling. x A The region B in the B The B cropped block is filled.

[0013] Step 4: Sample a rectangular mask M whose aspect ratio is proportional to the original image. The box coordinates are uniformly sampled as follows

[0014] r x ~Unif(0,W)

[0015] r y ~Unif(0,H)

[0016]

[0017] Make the crop area ratio For the cropped region, a binary mask M∈{0,1} is determined by filling 0 inside the bounding box B. W×H , otherwise fill in 1.

[0018] Step 5: In each training iteration, two randomly selected training samples are combined into small batches to generate cut-mixed samples (~x, ~y), where the true labels are also mixed proportionally by the area of the blocks. The augmented data is used to train the model with its original loss function.

[0019] Beneficial effects:

[0020] Compared to existing image data augmentation methods, such as simple noise addition, rotation, magnification, and reduction, the technology of the present invention uses an image region swapping strategy to achieve the regularization effect of region loss while effectively utilizing training pixels, further improving the generalization and robustness of deep learning algorithms. Different region swapping strategies are used for different images. For example, for multi-frame time series images, image sequences of different time frames can be swapped and added as samples to the dataset, thereby further expanding the data sample set. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a workflow diagram of a deep learning network regularization method based on image local feature enhancement of the present invention.

[0022] Figure 2 A structural diagram is selected for the data augmentation mode of the present invention. DETAILED DESCRIPTION

[0023] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings. For the sake of clarity and conciseness, not all features of an actual implementation are described in this specification. However, it should be understood that in the process of developing any such actual implementation, many implementation-specific decisions must be made in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and these constraints may vary from implementation to implementation. In addition, it should be understood that although the development work may be very complex and time-consuming, it is a routine task for those skilled in the art who benefit from the disclosure of the present invention.

[0024] It is also necessary to explain here that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show the device structure and / or processing steps closely related to the solution according to the present invention, while other details that are not closely related to the present invention are omitted.

[0025] step:

[0026] Step 1: Get the sample image information in the dataset, let x∈Rw×H×C and y represent the training image and its label respectively.

[0027] Step 2: Select the data augmentation mode to determine the two training samples (x A ,y A ) and (x B ,y B ).

[0028] Specific: Mode 1: (x A ,y A )=(x B ,yB ), that is, swapping two image blocks of the same image as augmented data. Mode 2: Randomly select other images from the same dataset as (x B ,y B ). Mode 3: When the data set consists of multiple time series images, the image sequences of different time frames are swapped and added to the data set as samples. That is, x1=(x A ,y A , t), randomly select other frames of the same s time series as (x B ,y B ), (x B ,y B )=x1(:,:,t).

[0029] Step 3: By combining two training samples (x A ,y A ) and (x B ,y B ) to generate new training samples (~x, ~y).

[0030] x%=Me x A +(1-M)ex B

[0031] y%=λy A +(1-λ)y B

[0032] M∈{0,1} W×H represents the binary mask indicating the removal and padding from the two images, 1 is the binary mask filled with 1, and e is the element-wise multiplication. The combination ratio λ between the two data points is sampled from a Beta distribution (α, α). In all our experiments, we set α to 1, i.e., sample λ from a uniform distribution (0, 1).

[0033] Step 3: Sample the binary mask M, first of all, the representation x A and x B The bounding box coordinates of the cropping area on B = (r x , r y , r w , r h ) for sampling. x A The region B in the B The B cropped block is filled.

[0034] Step 4: Sample a rectangular mask M whose aspect ratio is proportional to the original image. The box coordinates are uniformly sampled as follows

[0035] r x ~Unif(0,W)

[0036] r y ~Unif(0,H)

[0037]

[0038] Make the crop area ratio For the cropped region, a binary mask M∈{0,1} is determined by filling 0 inside the bounding box B. W×H , otherwise fill in 1.

[0039] Step 5: In each training iteration, two randomly selected training samples are combined into mini-batches to generate cut-mixed samples (~x, ~y), where the true labels are also mixed in proportion to the area of the blocks.

[0040] Step 6: Build a neural network model for training and prediction.

[0041] Step 6.1: Build a suitable, end-to-end fully convolutional neural network model. Its macro-architecture refers to the U-Net design. Specific hyperparameters, such as the number of convolutional network layers, convolution kernel size, step size, number of training rounds, optimizer, learning rate, etc., are obtained through debugging.

[0042] Step 6.2: Take the augmented dataset as input to the constructed convolutional neural network for training.

[0043] Step 6.3: After the network training is completed, the images to be classified are substituted into the neural network to obtain the final prediction results.

Claims

1. A deep learning network regularization method based on image local feature enhancement, characterized by Using the image local region exchange strategy, different region exchange strategies are used for different images to achieve the regularization effect of region loss while more effectively utilizing the training pixel information than the traditional region discarding method, thereby achieving effective expansion of the data sample set. The following steps are included: Step 1: Get the sample image information in the dataset, let x∈Rw×H×C and y represent the training image and its label respectively; Step 2: Select the data augmentation mode to determine the two training samples (x A ,y A ) and (x B ,y B ); Specific: Mode 1: (x A ,y A )=(x B ,y B ), that is, swapping two image blocks of the same image as augmented data, Mode 2: Randomly select other images of the same dataset as (x B ,y B ), Mode 3: When the data set consists of multiple time series images, the image sequences of different time frames are swapped and added to the data set as samples, that is, x1 = (x A ,y A , t), randomly select other frames of the same s time series as (x B ,y B ), (x B ,y B )=x1(:,:,t); Step 3: By combining two training samples (x A ,y A ) and (x B ,y B ) to generate new training samples ( ～ x, ～ y). M∈{0,1} W×H represents the binary mask indicating the removal and padding from the two images, 1 is the binary mask filled with 1, and e is the element-wise multiplication. The combination ratio λ between the two data points is sampled from the Beta distribution (α, α). In all our experiments, we set α to 1, i.e., sample λ from the uniform distribution (0, 1); Step 3: Sample the binary mask M, first of all, the representation x A and x B The bounding box coordinates of the cropping area on B = (r x , r y , r w , r h ) for sampling, x A The region B in the B The B cropped block is filled; Step 4: Sample the rectangular mask M whose aspect ratio is proportional to the original image. The box coordinates are uniformly sampled as follows r x ~Unif(0,W) r y ~Unif(0,H) Make the crop area ratio For the cropped region, a binary mask M∈{0,1} is determined by filling 0 inside the bounding box B. W×H , otherwise fill 1; Step 5: In each training iteration, two randomly selected training samples are combined into small batches to generate cut mixed samples (~x, ~y), where the true labels are also mixed proportionally to the area of the blocks; Step 6: Build a neural network model for training and prediction; Step 6.1: Build a suitable, end-to-end fully convolutional neural network model. Its macro-architecture refers to the U-Net design. Specific hyperparameters, such as the number of convolutional network layers, convolution kernel size, step size, number of training rounds, optimizer, and learning rate, are obtained through debugging. Step 6.2: Take the augmented dataset as input to the constructed convolutional neural network for training; Step 6.3: After the network training is completed, the images to be classified are substituted into the neural network to obtain the final prediction results.

Images