Robust high-level semantic region selection method and system based on learnable mask

Abstract

The application belongs to the field of image recognition and computer vision, and discloses a robust high-level semantic region selection method and system based on a learnable mask. The method is as follows: a plurality of sample images are collected and labeled to obtain a plurality of training samples; a convolutional neural network is used to extract features of the training samples to obtain a semantic feature map; a two-dimensional learnable mask matrix corresponding to the semantic feature map is constructed; in a forward process of the convolutional neural network, the semantic feature map is filtered by using the two-dimensional learnable mask matrix, and a prediction result is obtained according to a filtering output result; a total loss value is calculated according to a prediction loss and a mask sparsity regularization value, and parameters of the convolutional neural network and the mask matrix are updated and optimized; the input image is filtered by using the optimized mask matrix to obtain a robust high-level semantic region in the input image. The application can significantly improve the robustness and recognition accuracy of scene recognition under the condition of low-quality images such as blur, occlusion and overexposure.

Description

Technical Field

[0001] This invention relates to the fields of image processing and deep learning technology, and in particular to a robust region selection method and system for scene recognition implemented through neural networks, applicable to data processing and classification tasks of easily disturbed images such as underwater images and remote sensing images. Background Technology

[0002] In recent years, scene recognition, as a core task in computer vision, has been widely applied in fields such as environmental monitoring, marine exploration, urban planning, intelligent transportation, and military reconnaissance. The goal of this task is to semantically classify input images to identify the environment or scene category in which the image exists. Traditional scene recognition methods mainly rely on a combination of manually designed image features (such as SIFT and HOG) and traditional classifiers (such as SVM and Random Forest). However, these methods are often sensitive to complex backgrounds and interference factors, making it difficult to generalize to real-world application environments with complex noise or demanding imaging conditions, such as underwater and remote sensing images. With the development of convolutional neural networks (CNNs), especially the introduction of network architectures such as ResNet, DenseNet, and MobileNet, deep models have become the mainstream method for image recognition tasks. CNNs can automatically extract multi-level feature representations from raw images, where higher-level features closer to the output layer often contain rich semantic information and are widely used for classification tasks.

[0003] However, these methods still have the following problems.

[0004] Full-image processing introduces redundant information. Most current CNN models, by default, use the entire feature map for global average pooling or fully connected mapping when processing images. However, since real images contain a large number of non-discriminatory regions or background information, the features of these regions may be processed equally by the model, thereby introducing noise, redundant information, or even misleading information.

[0005] Unable to automatically identify "critical areas" In natural scenes or underwater images, the discriminative information is often concentrated in local areas, such as rock edges and sediment textures. Standard CNN models do not incorporate region selection mechanisms and often fail to focus on these "key areas," making them susceptible to factors such as blurring, occlusion, and changes in lighting.

[0006] Susceptible to interference from "category sharing areas" Some image regions, although appearing in multiple categories (e.g., a rocky area on the seabed may exist in multiple categories), lack discriminative power and may even be the source of misclassification. Traditional models indiscriminately utilize these shared regions, thereby obfuscating category boundaries.

[0007] Lack of robustness to low-quality images Images captured, especially underwater, in extreme weather, or at night, often suffer from low contrast, high noise, and blurred details. Current mainstream models perform inconsistently on these types of images and lack targeted enhancement or denoising mechanisms. Summary of the Invention

[0008] To address the technical problems of existing scene recognition technologies, such as poor robustness under poor image quality conditions (e.g., blurry, overexposed, occluded), insufficient attention to key regions, and improper handling of shared regions across multiple categories, the present invention aims to provide a robust semantic region selection method and system based on learnable masks. This method enables the model to automatically identify a small number of key regions for learning, and significantly improves recognition accuracy and anti-interference capabilities.

[0009] This invention proposes a feature selection mechanism based on a learnable mask matrix. Combined with a deep neural network structure, the network focuses on important semantic regions through adaptive adjustment of feature weights, effectively suppressing misclassification that may be caused by shared regions among multiple categories. Furthermore, this invention introduces an L1-norm-based regularization term to enhance sparsity, guiding the model to make decisions only using the regions with the highest discriminative power for classification, thereby achieving "few but precise" feature learning.

[0010] This invention constructs a UGS underwater geological scene dataset and combines it with a traditional remote sensing dataset. Extensive experiments under different image quality conditions demonstrate that this method can significantly improve the recognition accuracy and stability of mainstream CNN models.

[0011] The technical solution of this invention is as follows: A robust high-level semantic region selection method based on learnable masks includes the following steps: Multiple sample images are collected and labeled to obtain multiple training samples; Convolutional neural networks are used to extract features from training samples to obtain semantic feature maps; A two-dimensional learnable mask matrix corresponding to the semantic feature map is constructed; during the forward pass of the convolutional neural network, the semantic feature map is filtered using the two-dimensional learnable mask matrix, and a prediction result is obtained based on the filtering output; the prediction loss and the mask sparsity regularization value are calculated based on the prediction result to calculate the total loss value, and the parameters of the convolutional neural network and the mask matrix are optimized and updated. The optimized mask matrix is ​​used to perform feature filtering on the input image to obtain robust high-level semantic regions in the input image.

[0012] Preferably, each layer of the convolutional neural network extracts a feature map from the training samples to form the semantic feature map; the two-dimensional learnable mask matrix includes multiple mask matrices, each mask matrix corresponding to a feature map, used to perform point-by-point weighting processing on the corresponding feature map; each element of the mask matrix corresponds to a region of the feature map, and the larger the element value, the higher the weight assigned to the corresponding region in the classification.

[0013] Preferably, during the forward pass of the convolutional neural network, each layer of the mask matrix in the two-dimensional learnable mask matrix is ​​multiplied point by point with the feature map of the corresponding layer in the convolutional neural network to obtain the filtered feature map; then, global average pooling is performed on the filtered feature map of the last layer of the convolutional neural network and input into the classifier to obtain the prediction result.

[0014] Preferably, the total loss function is used. Calculate the total loss value and optimize and update the parameters of the convolutional neural network and the mask matrix so that the convolutional neural network gradually focuses on the most robust key region. To predict losses; As a mask sparsity regularization term, based on the L1 norm, it causes more elements in the mask matrix to approach zero in order to suppress redundant regions; This is a balancing coefficient used to control the contribution of the regularization term to the total loss.

[0015] Preferably, the convolutional neural network is ResNet18 or ResNet50; the classifier includes a fully connected layer and a Softmax layer.

[0016] Preferably, the sample image is a photograph of the seabed topography, a remote sensing image, or a low-quality image containing interference information.

[0017] Preferably, the interference information is blurry, overexposed, unevenly lit, or partially obscured.

[0018] A robust high-level semantic region selection system based on learnable masks, characterized in that it includes a training sample generation module, a feature extraction module, a training module, and a region selection module; The training sample generation module is used to acquire multiple sample images and label them to obtain multiple training samples; The feature extraction module is used to extract features from training samples using a convolutional neural network to obtain a semantic feature map; The training module is used to construct a two-dimensional learnable mask matrix corresponding to the semantic feature map; during the forward pass of the convolutional neural network, the two-dimensional learnable mask matrix is ​​used to filter the semantic feature map, and the prediction result is obtained based on the filtering output result; the prediction loss and the mask sparsity regularization value are calculated based on the prediction result to calculate the total loss value, and the parameters of the convolutional neural network and the mask matrix are optimized and updated. The region selection module is used to perform feature filtering on the input image using the optimized mask matrix to obtain robust high-level semantic regions in the input image.

[0019] A server is characterized by comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the methods described above.

[0020] A computer-readable storage medium having a computer program stored thereon, characterized in that the computer program implements the above-described method when executed by a processor.

[0021] The beneficial effects of the above-described technical solution of the present invention are as follows: This invention explicitly guides the model to focus on a small number of key regions in an image, thereby improving the discriminative power of classification. By introducing a regularization term to sparsify the mask matrix, it effectively suppresses the negative impact of redundant and interfering regions on the recognition results. Furthermore, this invention significantly improves the model's robustness in complex scenes such as low-quality images, heavily occluded images, and uneven lighting. It also supports an end-to-end training process without requiring additional key region annotation, making it highly practical and easy to use. Due to its good versatility and scalability, this invention can be widely applied to various scene recognition tasks, including marine remote sensing, underwater exploration, and aerial imagery. Attached Figure Description

[0022] Figure 1 This is a flowchart of the method of the present invention.

[0023] Figure 2 This is a framework diagram of the robust semantic region selection method based on learnable masks of the present invention. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to embodiments and accompanying drawings.

[0025] The method flow of this invention is as follows: Figure 1 As shown, the steps include: Multiple sample images are collected and labeled to obtain multiple training samples; Convolutional neural networks are used to extract features from training samples to obtain semantic feature maps; A two-dimensional learnable mask matrix corresponding to the semantic feature map is constructed; during the forward pass of the convolutional neural network, the semantic feature map is filtered using the two-dimensional learnable mask matrix, and a prediction result is obtained based on the filtering output; the prediction loss and the mask sparsity regularization value are calculated based on the prediction result to calculate the total loss value, and the parameters of the convolutional neural network and the mask matrix are optimized and updated. The optimized mask matrix is ​​used to perform feature filtering on the input image to obtain robust high-level semantic regions in the input image.

[0026] An optional embodiment of the present invention provides a robust scene recognition method based on learnable masks, the steps of which include: 1. Acquire image samples: Collect images of the target scene, which can be underwater images, remote sensing images, or other images with noise or occlusion; 2. Feature extraction: Use CNN networks (such as ResNet18 / 50 / 101) to extract features from the input image to obtain high-level semantic feature maps; 3. Mask matrix initialization: Construct and initialize a learnable mask matrix for the feature map of each layer of the CNN network, which is used to perform point-by-point weighting processing on the feature map of each layer; 4. Mask-based feature region selection: Obtain selected features using a mask matrix; 5. Construct the loss function and train: Based on the cross-entropy loss, a sparsity regularization term is introduced to constrain the learnable mask matrix, making it more sparse, thereby suppressing the interference of redundant features on prediction during training; by optimizing the total loss function (cross-entropy loss + sparsity regularization term), the model parameters and mask matrix are jointly trained. 6. Inference phase: After the model converges, efficient and robust scene recognition and prediction are performed using only the optimized mask matrix.

[0027] See Figure 2 An optional embodiment of the present invention provides an image acquisition device, an image feature extraction module, a feature mask generation module, and a scene label prediction module.

[0028] The image acquisition device is used to acquire the target image to be identified, and can be an underwater high-definition camera, a remote sensing satellite imaging device, or other image acquisition device; the image feature extraction module is used to process the input image using a convolutional neural network to obtain a high-level semantic feature map; the feature mask generation module is used to introduce a learnable mask matrix on the feature map, weight different regions, filter out region features with strong discriminative power, and combine sparse regularization to suppress redundant regions; the scene label prediction module performs global average pooling and classification operations on the features after mask filtering, and finally outputs the corresponding scene category label.

[0029] In the above embodiments, the image acquisition device can be a camera or other imaging device; the image feature extraction module can adopt a convolutional neural network structure such as ResNet or MobileNet; the feature mask generation module can be implemented using learnable matrix parameters obtained through training; and the scene label prediction module can be implemented using a fully connected layer and a Softmax classifier.

[0030] The specific workflow of an optional embodiment of the present invention is as follows: 1. Image Acquisition Use image acquisition equipment (such as underwater high-definition cameras, remote sensing satellite sensors, or drone cameras) to acquire images of the target scene. The image is then labeled. Here, w and h represent the width and height of the image. This image can be a photograph of the seabed topography, a remote sensing image, or other low-quality image containing complex interference (such as blur, overexposure, uneven lighting, or local occlusion). In practical applications, the image can be preprocessed to a uniform size (e.g., 224). (224 pixels) so that it can be used as input for deep neural networks.

[0031] 2. Feature Extraction Input image Inputting the encoder into a convolutional neural network (CNN), such as the mainstream architecture ResNet18 or ResNet50, and performing operations such as convolution, pooling, and non-linear activation, yields the semantic feature maps of each layer. : in, For the encoder of a convolutional neural network, This represents the number of channels in the output feature of the i-th layer. Represents the feature map of the i-th layer The width and height are given by L, where L is the number of layers in the CNN encoder. It contains abstract semantic information and can be used as a basis for classification.

[0032] 3. Mask matrix initialization In feature map In the spatial dimension, construct a corresponding two-dimensional learnable mask matrix. Each element of the mask matrix corresponds to a region of the feature map. If the element is close to 1, it means that the region is preserved and given a high weight in classification; if the element is close to 0, it means that the region contributes little to classification and will be suppressed. Mask matrix During training, it is optimized along with the network weights as a parameter, gradually learning the distribution of the most discriminative feature regions.

[0033] 4. Mask selection of feature regions During the forward pass of the CNN, the mask matrix of each layer is multiplied point-by-point with the feature map of each layer of the convolutional model (Hadamard product) to obtain the filtered feature map. : The filtered feature map retains only the response values ​​of key regions. Then, the last layer... Perform global average pooling (GAP) and input it into a classifier (such as a fully connected layer + Softmax) to obtain the prediction result: Where y is the classification probability distribution and f is the classification function.

[0034] 5. Construct the loss function and train. To simultaneously ensure classification accuracy and mask sparsity, this invention designs a total loss function: in, To predict the loss, the cross-entropy function is used to calculate the predicted loss based on the predicted result y of the training samples and the corresponding label y'. This is used to ensure the accuracy of the classification results; the more consistent the prediction result is with the corresponding label, the lower the prediction loss. The lower; = As a mask sparsity regularization term, based on the L1 norm, it causes more elements in the mask to approach zero in order to suppress redundant regions; This is a balancing coefficient used to control the contribution of the regularization term to the total loss. During training, it is adjusted based on the total loss value. By updating the network parameters and mask matrix parameters simultaneously, the model can use as few features as possible to make the most accurate predictions, gradually focusing the model on the most robust key regions.

[0035] 6. Reasoning Stage After the model training is complete, the finally learned mask matrix is ​​used. The features obtained from the input image are filtered. Note that each mask matrix corresponds to a layer of features in the model, and the model is responsible for filtering these features. During inference, the model only relies on the features filtered by the masks for classification, thereby ensuring: 1) fewer and more precise regions of interest; 2) avoiding the influence of interfering regions; and 3) improving robustness under conditions of blur, occlusion, and noise.

[0036] This invention proposes a robust scene recognition method based on learnable masks. Addressing the problems of attention dispersion, excessive redundant features, and misclassification caused by shared category regions in existing convolutional neural networks when processing low-quality images, this invention innovatively introduces a learnable mask matrix and a sparse regularization mechanism. This method can automatically filter out a small number of highly discriminative regions without relying on additional annotations, enabling the model to maintain high robustness and generalization ability even in complex environments. Validation on underwater geological scene datasets and remote sensing datasets demonstrates that this invention effectively improves the classification accuracy and stability of mainstream deep learning models, and possesses good versatility and scalability, making it widely applicable to underwater exploration, environmental monitoring, remote sensing analysis, and other image recognition tasks with high robustness requirements.

[0037] An optional embodiment of the present invention provides a robust high-level semantic region selection system based on learnable masks, characterized in that it includes a training sample generation module, a feature extraction module, a training module, and a region selection module; The training sample generation module is used to acquire multiple sample images and label them to obtain multiple training samples; The feature extraction module is used to extract features from training samples using a convolutional neural network to obtain a semantic feature map; The training module is used to construct a two-dimensional learnable mask matrix corresponding to the semantic feature map; during the forward pass of the convolutional neural network, the two-dimensional learnable mask matrix is ​​used to filter the semantic feature map, and the prediction result is obtained based on the filtering output result; the prediction loss and the mask sparsity regularization value are calculated based on the prediction result to calculate the total loss value, and the parameters of the convolutional neural network and the mask matrix are optimized and updated. The region selection module is used to perform feature filtering on the input image using the optimized mask matrix to obtain robust high-level semantic regions in the input image.

[0038] An optional embodiment of the present invention provides a server, characterized in that it includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the above-described method.

[0039] An optional embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program implements the above-described method when executed by a processor.

[0040] To illustrate the purpose of this invention, the above embodiments are merely illustrative and not intended to limit the scope of the invention. The scope of the invention is defined by the claims, and any equivalent substitutions and modifications that do not depart from the principles and core features of the embodiments of the invention should be included within the scope of the invention.

[0041] It will be apparent to those skilled in the art that the specific forms of the embodiments of the present invention are not limited to the details of the exemplary embodiments described above. Therefore, these embodiments should be considered exemplary and non-limiting, and the scope of the invention is defined by the claims rather than the foregoing description. Any reference numerals should not be construed as limiting the claims. Furthermore, the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules, or devices may be implemented by the same unit, module, or device through software or hardware.

[0042] Finally, although the preferred embodiments have been described in detail above, they are merely for illustrating the technical solutions of the embodiments of the present invention and not for limiting them. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions, but should not depart from the principles and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robust high-level semantic region selection method based on learnable masks, comprising the following steps: Multiple sample images were collected and labeled to obtain multiple training samples; Convolutional neural networks are used to extract features from training samples to obtain semantic feature maps; A two-dimensional learnable mask matrix corresponding to the semantic feature map is constructed; during the forward pass of the convolutional neural network, the semantic feature map is filtered using the two-dimensional learnable mask matrix, and a prediction result is obtained based on the filtering output; the prediction loss and the mask sparsity regularization value are calculated based on the prediction result to calculate the total loss value, and the parameters of the convolutional neural network and the mask matrix are optimized and updated. The optimized mask matrix is ​​used to perform feature filtering on the input image to obtain robust high-level semantic regions in the input image.

2. The method according to claim 1, characterized in that, Each layer of the convolutional neural network extracts a feature map from the training samples to form the semantic feature map; the two-dimensional learnable mask matrix includes multiple mask matrices, each mask matrix corresponding to a feature map, used to perform point-by-point weighting processing on the corresponding feature map; each element of the mask matrix corresponds to a region of the feature map, and the larger the element value, the higher the weight assigned to the corresponding region in the classification.

3. The method according to claim 2, characterized in that, During the forward pass of the convolutional neural network, each layer of the mask matrix in the two-dimensional learnable mask matrix is ​​multiplied point by point with the feature map of the corresponding layer in the convolutional neural network to obtain the filtered feature map; then, global average pooling is performed on the filtered feature map of the last layer of the convolutional neural network and input into the classifier to obtain the prediction result.

4. The method according to claim 2 or 3, characterized in that, Using the total loss function Calculate the total loss value and optimize and update the parameters of the convolutional neural network and the mask matrix so that the convolutional neural network gradually focuses on the most robust key region. To predict losses; As a mask sparsity regularization term, based on the L1 norm, it causes more elements in the mask matrix to approach zero in order to suppress redundant regions; This is a balancing coefficient used to control the contribution of the regularization term to the total loss.

5. The method according to claim 3, characterized in that, The convolutional neural network is ResNet18 or ResNet50; the classifier includes a fully connected layer and a Softmax layer.

6. The method according to claim 1, characterized in that, The sample images are seabed topographic photographs, remote sensing images, or low-quality images containing interference information.

7. The method according to claim 6, characterized in that, The interference information includes blurriness, overexposure, uneven lighting, and partial occlusion.

8. A robust high-level semantic region selection system based on learnable masks, characterized in that, It includes a training sample generation module, a feature extraction module, a training module, and a region selection module; The training sample generation module is used to acquire multiple sample images and label them to obtain multiple training samples; The feature extraction module is used to extract features from training samples using a convolutional neural network to obtain a semantic feature map; The training module is used to construct a two-dimensional learnable mask matrix corresponding to the semantic feature map; during the forward pass of the convolutional neural network, the two-dimensional learnable mask matrix is ​​used to filter the semantic feature map, and the prediction result is obtained based on the filtering output result; the prediction loss and the mask sparsity regularization value are calculated based on the prediction result to calculate the total loss value, and the parameters of the convolutional neural network and the mask matrix are optimized and updated. The region selection module is used to perform feature filtering on the input image using the optimized mask matrix to obtain robust high-level semantic regions in the input image.

9. A server, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images