Marine target identification method and system based on target background causal correction learning

Abstract

The invention discloses a marine target identification method and system based on target background causal correction learning, and belongs to the technical field of remote sensing detection. The method comprises the following steps: detecting a marine target area to obtain a marine image data set; performing image feature extraction and image semantic activation on the marine image data set by using two deep neural networks; the extracted image features and the activated image semantics are fused through a feature calibration method based on the causal intervention theory, background noise in the image is eliminated, and target feature semantics are obtained; and identifying and classifying the target feature semantics based on deep learning, and outputting a classification result of the target category. According to the invention, the sea target is identified based on target background causal correction learning, and the accuracy of sea target identification is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing detection, and in particular relates to a method and system for identifying marine targets based on target background causal correction learning. Background Art

[0002] Maritime target recognition technology is of great significance in the fields of marine safety, maritime supervision, and marine environmental protection. By identifying maritime targets, we can effectively monitor maritime activities, prevent and deal with maritime accidents, and safeguard maritime rights and interests. In recent years, maritime target recognition technology has made significant progress, but the complex marine environment, such as waves, weather changes, and other interference, leads to high instability in the recognition results.

[0003] Among the existing maritime target recognition technologies, methods based on deep learning have become mainstream and are widely used in maritime target recognition. This type of method extracts features at different levels in the input image through a series of network layers with different functions, and reduces the size of the feature map by downsampling to improve the computational efficiency and robustness of the model. Finally, the extracted features are integrated to output the classification results of the target. The whole process has a considerable degree of efficiency and accuracy.

[0004] Although the existing deep learning-based maritime target recognition technology has made significant progress, this type of method does not distinguish between the foreground (i.e., the target) and the background (i.e., the components other than the target in the image) during the feature extraction process. Therefore, the background in the maritime target image, including waves, weather changes, buildings, mountains, etc., will interfere with the feature extraction process that should focus on the target itself, resulting in deviations in the learning process of the existing methods. Therefore, there are deficiencies in the effectiveness and robustness of the extracted features, which limits the further development of maritime target recognition performance. Summary of the invention

[0005] The purpose of the present invention is to provide a marine target recognition scheme, which can identify marine targets based on target background causal correction learning and improve the accuracy of marine target recognition.

[0006] The present invention achieves the above-mentioned purpose by adopting the following technical scheme.

[0007] A method for identifying marine targets based on target background causal correction learning includes the following steps:

[0008] Detect the target area at sea and obtain the marine image data set;

[0009] Two deep neural networks are used to extract image features and activate image semantics on marine image datasets;

[0010] The feature calibration method based on causal intervention theory fuses the extracted image features and activated image semantics, eliminates background noise in the image, and obtains the target feature semantics;

[0011] Based on deep learning, the target feature semantics are identified and classified, and the classification results of the target category are output.

[0012] Preferably, optical or radar detection is performed on the target area at sea to obtain optical images or radar images at sea.

[0013] Preferably, the marine image dataset is denoted as in Represents the acquired marine image, y i represents the category label of the marine image, represents a set of real numbers, w, h, c represent the length, width and number of channels of the input image respectively, and N represents the number of images at sea.

[0014] Preferably, feature extraction and semantic activation are performed by two pre-trained ResNet50 models, and the feature extraction is represented as f sem (x), the semantic activation is represented by f sam (x), where x represents the acquired marine image; They represent the feature mapping functions of the pre-trained ResNet50 model, represents the set of real numbers, n,n c They represent the spatial dimension and channel dimension of the output respectively, and w, h, and c represent the length, width, and number of channels of the input image respectively.

[0015] Preferably, the formula for fusing the extracted image features and the activated image semantics is expressed as:

[0016]

[0017] Among them, f sem (x i ) represents the extracted image features, f sam (x i ) represents the activated image semantics.

[0018] Preferably, a fully connected network is used as the classifier.

[0019] Preferably, the formula for feature recognition and classification based on deep learning is expressed as:

[0020]

[0021] Among them, c represents the probability function that maps the received feature map to the target category, represents the set of real numbers, n,nc They represent the spatial dimension and channel dimension of the output respectively, and K represents the probability distribution.

[0022] Preferably, the deep learning process is to optimize the following loss function:

[0023]

[0024] in, is the indicator function; c i (·),c j (·) are the i-th and j-th dimensions of the fully connected network output, respectively. N represents the total number of samples, and the log base is e.

[0025] A marine target recognition system based on target background causal correction learning includes a marine image acquisition device and a data processor, the data processor includes an image feature extraction module, an image semantic activation module, a causal feature intervention module and a target category decision module, the marine image acquisition device is connected to the image feature extraction module and the image semantic activation module of the data processor, the image feature extraction module and the image semantic activation module are connected to the causal feature intervention module, and the causal feature intervention module is connected to the target category decision module;

[0026] Among them, the marine image acquisition equipment is responsible for detecting the marine target area and obtaining the marine image data set;

[0027] The image feature extraction module is responsible for extracting image features from the marine image dataset;

[0028] The image semantic activation module is responsible for image semantic activation of the marine image dataset;

[0029] The causal feature intervention module is responsible for fusing the extracted image features and activated image semantics based on the feature calibration method of causal intervention, eliminating background noise in the image and obtaining the target feature semantics;

[0030] The target category decision module is responsible for identifying and classifying the target feature semantics based on deep learning and outputting the classification results of the target category.

[0031] Preferably, the marine image acquisition device is an optical network camera or a radar sensor.

[0032] Preferably, the data processor is a computer or an embedded motherboard.

[0033] Preferably, the image feature extraction module and the image semantic activation module are both ResNet50 pre-trained modules based on deep learning methods.

[0034] The beneficial effects achieved by the technical solution of the present invention include: the feature calibration method based on the causal intervention theory of the present invention can accurately filter out the background semantics in the feature extraction results. Compared with the conventional marine target recognition method, causal intervention is introduced to model and eliminate the negative impact of the background in the image on the feature extraction of the deep learning model, so that the model focuses on the feature information of the extracted target itself, improves the discriminability of the extracted features, and can obtain more superior target recognition accuracy and robustness. At the same time, the data processor can be an embedded motherboard that can be built on a variety of intelligent monitoring equipment, so that the system has the characteristics of easy implementation, low complexity, wide adaptability, plug and play, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a framework diagram of a marine target recognition system based on target background causal correction learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the above features and advantages of the present invention more obvious and easy to understand, embodiments are given below and described in detail with reference to the accompanying drawings.

[0037] The embodiment of the present invention provides a method for identifying marine targets based on target-background causal correction learning. The method adopts a marine target recognition method based on target-background causal correction learning, and models the background in the marine image so that it is recognized as a confounding factor for feature extraction of the deep learning model. According to the causal relationship between the target in the marine image and its background, causal intervention is used to eliminate the interference of the background on the extraction of target semantic information, and the target features in the marine image are characterized, so that the complex background interference can be ignored from the marine image data to improve the accuracy, effectiveness and robustness of its target recognition.

[0038] The embodiment of the present invention also provides a marine target recognition system based on target background causal correction learning, such as Figure 1As shown, it includes: a marine image acquisition device and a data processor. The data processor includes: an image feature extraction module, an image semantic activation module, a causal feature intervention module and a target category decision module. The causal feature intervention module is connected to the target category decision module, and the image feature extraction module and the image semantic activation module are connected to the causal feature intervention module; the marine image acquisition device is connected to the image feature extraction module and the image semantic activation module in the data processor. Among them, the marine image acquisition device can be an optical network camera, a radar sensor or other image acquisition device. The data processor can be a computer or an embedded motherboard. The image feature extraction module can use a marine image feature extraction method based on deep learning to extract features from the input image. The image semantic activation module can use a marine image feature extraction method based on deep learning to perform semantic activation on the input image. The causal feature intervention module can use a feature calibration method based on causal intervention theory to filter out background semantics in the feature extraction result. The target category decision module can use a feature recognition and decision method based on deep learning to identify targets in marine images.

[0039] The embodiment of the present invention implements the above method to perform marine surface recognition through the above system. The following is a specific processing flow.

[0040] First, the marine image acquisition equipment performs optical or radar detection on the marine target area, obtains marine optical or radar images, and constructs a marine image dataset, which is recorded as in, w, h, c represent the length, width and number of channels of the input image, respectively, and are input to the data processor.

[0041] Then, the image feature extraction module and image semantic activation module in the data processor perform feature extraction and semantic activation on the input marine image dataset through two different sets of deep neural networks. These two processing processes can be expressed as f sem (x) and f sam (x), where and They represent the pre-trained feature mapping functions (such as the mapping function performed by the ResNet50 model), n, n c They represent the spatial dimension and channel dimension of the output respectively, so as to obtain the output of the same image under two different deep neural network mappings.

[0042] Then, the causal feature intervention module uses a feature calibration method based on the causal intervention theory to eliminate the negative impact of the background in the image on the target feature extraction. The specific calculation process can be expressed as:

[0043]

[0044] The output result is the target feature semantics, which activates the feature semantics belonging to the foreground target and suppresses the feature semantics belonging to the background, thereby mining the true causal relationship.

[0045] Finally, the target category decision module uses a feature recognition and decision method based on deep learning, and uses a fully connected network to map the output results of the input causal feature intervention module to a probability function of the target category. The process can be expressed as:

[0046]

[0047] The learning process is defined as optimizing the loss function as shown below:

[0048]

[0049] in, is the indicator function; c i (·),c j (·) are the i-th and j-th dimensions of the fully connected network output, respectively. N represents the total number of samples, and the log base is e.

[0050] For the purpose of illustrating the present invention, the above-mentioned embodiments are only for description and not for limiting the scope of the present invention. For those skilled in the art, it is obvious that the specific form of the embodiments of the present invention is not limited to the details in the above-mentioned exemplary embodiments. Therefore, these embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the claims rather than the above description. Any reference numerals should not be regarded as limiting the claims. In addition, the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules or devices can be implemented by the same unit, module or device through software or hardware.

[0051] Although the above preferred embodiment is described in detail, it is only for describing the technical solution of the embodiment of the present invention and not for limiting. Those skilled in the art should understand that the technical solution can be modified or replaced by equivalents, but should not deviate from the principle and scope of the technical solution of the embodiment of the present invention. Any equivalent replacement and modification that does not deviate from the principle and core features of the embodiment of the present invention should be included in the scope of the present invention, and the scope of the present invention is determined by the claims.

Claims

1. A method for identifying marine targets based on target background causal correction learning, characterized in that: The following steps are involved: Detect the target area at sea and obtain the marine image data set; Two deep neural networks are used to extract image features and activate image semantics on marine image datasets; The feature calibration method based on causal intervention theory fuses the extracted image features and activated image semantics, eliminates background noise in the image, and obtains the target feature semantics; Based on deep learning, the target feature semantics are identified and classified, and the classification results of the target category are output.

2. The method according to claim 1, characterized in that Conduct optical or radar detection of target areas at sea and obtain marine optical images or marine radar images.

3. The method according to claim 1, characterized in that The marine image dataset is denoted as in represents the acquired marine image, y i represents the category label of the marine image, represents a set of real numbers, w, h, c represent the length, width and number of channels of the input image respectively, and N represents the number of images at sea.

4. The method according to claim 1, characterized in that Feature extraction and semantic activation are performed through two pre-trained ResNet50 models. The feature extraction is represented as f sem (x), the semantic activation is represented by f sam (x), where x represents the acquired marine image; They represent the feature mapping functions of the pre-trained ResNet50 model, represents the set of real numbers, n,n c They represent the spatial dimension and channel dimension of the output respectively, and w, h, and c represent the length, width, and number of channels of the input image respectively.

5. The method according to claim 4, characterized in that The formula for fusing the extracted image features and activated image semantics is expressed as: Among them, f sem (x i ) represents the extracted image features, f sam (x i ) represents the activated image semantics.

6. The method according to claim 1, characterized in that A fully connected network is used as the classifier.

7. The method according to claim 5, characterized in that The formula for feature recognition and classification based on deep learning is expressed as: Among them, c represents the probability function that maps the received feature map to the target category, represents the set of real numbers, n,n c They represent the spatial dimension and channel dimension of the output respectively, and K represents the probability distribution.

8. The method according to claim 7, characterized in that The deep learning process is to optimize the following loss function: in, is the indicator function; c i (·),c j (·) are the i-th and j-th dimensions of the fully connected network output, N is the total number of samples, and the log base is e.

9. A marine target recognition system based on target background causal correction learning, used to implement the method described in any one of claims 1 to 8, characterized in that: It includes an offshore image acquisition device and a data processor, the data processor includes an image feature extraction module, an image semantic activation module, a causal feature intervention module and a target category decision module, the offshore image acquisition device is connected to the image feature extraction module and the image semantic activation module of the data processor, the image feature extraction module and the image semantic activation module are connected to the causal feature intervention module, and the causal feature intervention module is connected to the target category decision module; Among them, the marine image acquisition equipment is responsible for detecting the marine target area and obtaining the marine image data set; The image feature extraction module is responsible for extracting image features from the marine image dataset; The image semantic activation module is responsible for image semantic activation of the marine image dataset; The causal feature intervention module is responsible for fusing the extracted image features and activated image semantics based on the feature calibration method of causal intervention, eliminating background noise in the image and obtaining the target feature semantics; The target category decision module is responsible for identifying and classifying the target feature semantics based on deep learning and outputting the classification results of the target category.

10. The system according to claim 9, characterized in that The marine image acquisition device is an optical network camera or a radar sensor; the data processor is a computer or an embedded motherboard; the image feature extraction module and the image semantic activation module are both ResNet50 pre-trained modules based on deep learning methods.

Images