Maritime target recognition method and system based on target background causal correction learning

Abstract

The present invention discloses a method and system for identifying marine targets based on target-background causal correction learning, belonging to the field of remote sensing detection technology. The present invention detects a marine target area to obtain a marine image dataset; utilizes two deep neural networks to extract image features and activate image semantics from the marine image dataset; uses a feature calibration method based on causal intervention theory to fuse the extracted image features and activated image semantics, eliminate background noise in the image, and obtain target feature semantics; and identifies and classifies the target feature semantics based on deep learning, outputting the classification results of the target categories. The present invention identifies marine targets based on target-background causal correction learning, thereby improving the accuracy of marine target recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing detection, and in particular relates to a method and system for identifying marine targets based on target background causal correction learning. Background Art

[0002] Maritime target recognition technology is of great significance in areas such as maritime safety, maritime regulation, and marine environmental protection. By identifying maritime targets, we can effectively monitor maritime activities, prevent and address maritime accidents, and safeguard maritime rights and interests. While maritime target recognition technology has made significant progress in recent years, the complex marine environment, characterized by waves, weather changes, and other disturbances, can lead to high instability in recognition results.

[0003] Among existing maritime target recognition technologies, deep learning-based methods have become mainstream and widely used. These methods extract features at different levels from the input image through a series of functionally distinct network layers. Downsampling reduces the size of the feature maps, improving the model's computational efficiency and robustness. Ultimately, the extracted features are integrated to output the target classification results. The entire process is highly efficient and accurate.

[0004] Although existing deep learning-based maritime target recognition technologies have made significant progress, such methods do not distinguish between foreground (i.e., target) and background (i.e., components other than the target in the image) during feature extraction. Therefore, the background in maritime target images, including waves, weather changes, buildings, mountains, etc., will interfere with the feature extraction process that should focus on the target itself, resulting in deviations in the learning process of existing methods. Therefore, there are deficiencies in the effectiveness and robustness of the extracted features, which limits the further development of maritime target recognition performance. Summary of the Invention

[0005] The purpose of the present invention is to provide a maritime target recognition solution, which can identify maritime targets based on target background causal correction learning and improve the accuracy of maritime target recognition.

[0006] The present invention achieves the above-mentioned purpose by adopting the following technical solutions.

[0007] A method for identifying maritime targets based on target background causal correction learning includes the following steps:

[0008] Detect target areas at sea and obtain marine image datasets;

[0009] Two deep neural networks are used to extract image features and activate image semantics on a maritime image dataset.

[0010] The feature calibration method based on causal intervention theory fuses the extracted image features and activated image semantics, eliminates background noise in the image, and obtains the target feature semantics;

[0011] Based on deep learning, the target feature semantics are identified and classified, and the classification results of the target category are output.

[0012] Preferably, optical or radar detection is performed on the target area at sea to obtain optical images or radar images at sea.

[0013] Preferably, the marine image dataset is denoted as in Represents the acquired marine image, y i represents the category label of the marine image, represents a set of real numbers, w, h, c represent the length, width and number of channels of the input image respectively, and N represents the number of marine images.

[0014] Preferably, feature extraction and semantic activation are performed by two pre-trained ResNet50 models, and the feature extraction is represented as f sem (x), semantic activation is represented as f sam (x), where x represents the acquired marine image; Respectively represent the feature mapping functions of the pre-trained ResNet50 model, represents the set of real numbers, n,n c They represent the spatial dimension and channel dimension of the output respectively, and w, h, and c represent the length, width, and number of channels of the input image respectively.

[0015] Preferably, the formula for fusing the extracted image features and the activated image semantics is expressed as:

[0016]

[0017] Among them, f sem (x i ) represents the extracted image features, f sam (x i ) represents the activated image semantics.

[0018] Preferably, a fully connected network is used as the classifier.

[0019] Preferably, the formula for feature recognition and classification based on deep learning is expressed as:

[0020]

[0021] Among them, c represents the probability function that maps the received feature map to the target category, represents the set of real numbers, n,nc They represent the spatial dimension and channel dimension of the output respectively, and K represents the probability distribution.

[0022] Preferably, the deep learning process is to optimize the following loss function:

[0023]

[0024] in, is the indicator function; c i (·),c j (·) are the i-th and j-th dimensions of the fully connected network output, N is the total number of samples, and the log base is e.

[0025] A marine target recognition system based on target-background causal correction learning includes a marine image acquisition device and a data processor, the data processor including an image feature extraction module, an image semantic activation module, a causal feature intervention module, and a target category decision module, the marine image acquisition device is connected to the image feature extraction module and the image semantic activation module of the data processor, the image feature extraction module and the image semantic activation module are connected to the causal feature intervention module, and the causal feature intervention module is connected to the target category decision module;

[0026] Among them, the marine image acquisition equipment is responsible for detecting the target area at sea and obtaining the marine image data set;

[0027] The image feature extraction module is responsible for extracting image features from the marine image dataset;

[0028] The image semantic activation module is responsible for image semantic activation of the marine image dataset;

[0029] The causal feature intervention module is responsible for fusing the extracted image features and activated image semantics based on the feature calibration method of causal intervention, eliminating background noise in the image and obtaining the target feature semantics;

[0030] The target category decision module is responsible for identifying and classifying the target feature semantics based on deep learning and outputting the classification results of the target category.

[0031] Preferably, the marine image acquisition device is an optical network camera or a radar sensor.

[0032] Preferably, the data processor is a computer or an embedded motherboard.

[0033] Preferably, the image feature extraction module and the image semantic activation module are both ResNet50 pre-trained modules based on deep learning methods.

[0034] The beneficial effects achieved by the technical solution of the present invention include: the feature calibration method of the present invention, based on the causal intervention theory, can accurately filter out the background semantics in the feature extraction results. Compared with conventional marine target recognition methods, causal intervention is introduced to model and eliminate the negative impact of the background in the image on the feature extraction of the deep learning model, so that the model focuses on the feature information of the extracted target itself, improves the discriminability of the extracted features, and can obtain more superior target recognition accuracy and robustness. At the same time, the data processor can be an embedded motherboard that can be built on a variety of intelligent monitoring equipment, making the system easy to implement, low complexity, wide adaptability, plug and play, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a framework diagram of a maritime target recognition system based on target background causal correction learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below with reference to the accompanying drawings for detailed description.

[0037] Embodiments of the present invention provide a method for maritime target recognition based on target-background causal correction learning. This method employs a target-background causal correction learning approach to model the background in maritime images, identifying it as a confounding factor in feature extraction using a deep learning model. Based on the causal relationship between the target and its background in the maritime image, causal intervention is used to eliminate background interference in target semantic information extraction, focusing on characterizing target features in the maritime image. This method can ignore complex background interference in maritime image data, thereby improving the accuracy, effectiveness, and robustness of target recognition.

[0038] The embodiment of the present invention also provides a marine target recognition system based on target background causal correction learning, such as Figure 1As shown, the system comprises: a marine image acquisition device and a data processor. The data processor includes: an image feature extraction module, an image semantic activation module, a causal feature intervention module, and a target category decision module. The causal feature intervention module is connected to the target category decision module, and the image feature extraction module and the image semantic activation module are connected to the causal feature intervention module. The marine image acquisition device is connected to the image feature extraction module and the image semantic activation module in the data processor. The marine image acquisition device can be an optical network camera, a radar sensor, or other image acquisition device. The data processor can be a computer or an embedded motherboard. The image feature extraction module can use a deep learning-based marine image feature extraction method to extract features from the input image. The image semantic activation module can use a deep learning-based marine image feature extraction method to perform semantic activation on the input image. The causal feature intervention module can use a feature calibration method based on causal intervention theory to filter out background semantics from the feature extraction results. The target category decision module can use a deep learning-based feature recognition and decision method to identify targets in marine images.

[0039] The embodiment of the present invention implements the above method through the above system to perform marine surface identification. The following is a specific processing flow.

[0040] First, the marine image acquisition equipment performs optical or radar detection on the marine target area, obtains marine optical or radar images, and constructs a marine image dataset, which is recorded as in, w, h, and c represent the length, width, and number of channels of the input image, respectively, and are input to the data processor.

[0041] Then, the image feature extraction module and image semantic activation module in the data processor perform feature extraction and semantic activation on the input marine image dataset through two different sets of deep neural networks. These two processing processes can be expressed as f sem (x) and f sam (x), where and They represent the pre-trained feature mapping function (such as the mapping function performed by the ResNet50 model), n, n c They represent the spatial dimension and channel dimension of the output respectively, so as to obtain the output of the same image under two different deep neural network mappings.

[0042] Then, the causal feature intervention module uses a feature calibration method based on causal intervention theory to eliminate the negative impact of the background in the image on the target feature extraction. The specific calculation process can be expressed as:

[0043]

[0044] The output result is the target feature semantics, which activates the feature semantics belonging to the foreground target and suppresses the feature semantics belonging to the background, thereby mining the true causal relationship.

[0045] Finally, the target category decision module uses a deep learning-based feature recognition and decision-making method and a fully connected network to map the output results of the input causal feature intervention module into a probability function of the target category. This process can be expressed as:

[0046]

[0047] The learning process is defined as optimizing the loss function as shown below:

[0048]

[0049] in, is the indicator function; c i (·),c j (·) are the i-th and j-th dimensions of the fully connected network output, N is the total number of samples, and the log base is e.

[0050] For the purpose of illustrating the present invention, the above embodiments are merely illustrative and not limiting of the scope of the present invention. For those skilled in the art, it will be apparent that the specific forms of the embodiments of the present invention are not limited to the details in the above exemplary embodiments. Therefore, these embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the claims rather than the above description. Any reference numerals should not be construed as limiting the claims. In addition, the term "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules or devices can be implemented by the same unit, module or device through software or hardware.

[0051] Although the preferred embodiment has been described in detail above, this is merely for the purpose of describing the technical solutions of the embodiments of the present invention and is not intended to limit the scope of the present invention. Those skilled in the art should understand that modifications or equivalent substitutions may be made to the technical solutions without departing from the principles and scope of the technical solutions of the embodiments of the present invention. Any equivalent substitutions and modifications that do not depart from the principles and core features of the embodiments of the present invention are intended to be included within the scope of the present invention, which is defined by the claims.

Claims

1. A method for identifying maritime targets based on target-background causal correction learning, characterized in that: The following steps are involved: Detect target areas at sea and obtain marine image datasets; Two deep neural networks are used to extract image features and activate image semantics on the marine image dataset. Specifically, two pre-trained ResNet50 models are used for feature extraction and semantic activation. The feature extraction is represented by f sem (x), semantic activation is represented as f sam (x), where x represents the acquired marine image; Respectively represent the feature mapping functions of the pre-trained ResNet50 model, represents the set of real numbers, n,n c Represent the spatial dimension and channel dimension of the output respectively, w, h, c represent the length, width and number of channels of the input image respectively; The feature calibration method based on causal intervention theory fuses the extracted image features and the activated image semantics, eliminates background noise in the image, and obtains the target feature semantics. The formula for fusing the extracted image features and the activated image semantics is expressed as: Among them, f sem (x i ) represents the extracted image features, f sam (x i ) represents the activated image semantics; A fully connected network is used as a classifier to identify and classify the target feature semantics based on deep learning, and the classification results of the target category are output.

2. The method according to claim 1, wherein Conduct optical or radar detection of target areas at sea and obtain optical or radar images at sea.

3. The method according to claim 1, wherein The marine image dataset is denoted as in Represents the acquired marine image, y i represents the category label of the marine image, represents a set of real numbers, w, h, c represent the length, width and number of channels of the input image respectively, and N represents the number of marine images.

4. The method according to claim 1, wherein The formula for feature recognition and classification based on deep learning is expressed as: Among them, c represents the probability function of mapping the received features to the target category, represents the set of real numbers, n,n c They represent the spatial dimension and channel dimension of the output respectively, and K represents the probability distribution.

5. The method according to claim 4, wherein The deep learning process is to optimize the following loss function: in, is the indicator function; c k (·),c j (·) are the kth and jth dimensions of the fully connected network output, N is the total number of samples, and the log base is e.

6. A marine target recognition system based on target-background causal correction learning, used to implement the method according to any one of claims 1 to 5, characterized in that: It includes an offshore image acquisition device and a data processor, the data processor includes an image feature extraction module, an image semantic activation module, a causal feature intervention module and a target category decision module, the offshore image acquisition device is connected to the image feature extraction module and the image semantic activation module of the data processor, the image feature extraction module and the image semantic activation module are connected to the causal feature intervention module, and the causal feature intervention module is connected to the target category decision module; Among them, the marine image acquisition equipment is responsible for detecting the target area at sea and obtaining the marine image data set; The image feature extraction module is responsible for extracting image features from the marine image dataset; The image semantic activation module is responsible for image semantic activation of the marine image dataset; The causal feature intervention module is responsible for fusing the extracted image features and activated image semantics based on the feature calibration method of causal intervention, eliminating background noise in the image and obtaining the target feature semantics; The target category decision module is responsible for identifying and classifying the target feature semantics based on deep learning and outputting the classification results of the target category.

7. The system according to claim 6, wherein: The marine image acquisition device is an optical network camera or a radar sensor; the data processor is a computer or an embedded motherboard; the image feature extraction module and the image semantic activation module are both ResNet50 pre-trained models.

Images