Multi-modal remote sensing image sea target identification method based on AI

Abstract

The invention discloses an AI-based multi-modal remote sensing image sea target identification method, and particularly relates to the technical field of sea target identification. The method comprises the following steps: performing preprocessing and feature extraction on a remote sensing image of an intensive ship overlapping scene, evaluating the consistency of target segmentation based on an instance segmentation model, and comprehensively analyzing the segmentation performance and anti-occlusion capability of a deep learning model by measuring the deviation between a detection frame and a real target area in an occlusion test data set; under the condition of inaccurate target identification, the target position and shape of a shielded part are speculated by adopting a time sequence compensation method in combination with motion information of front and rear frames, so that the segmentation and identification accuracy of the target is improved in a dense ship overlapping environment, and finally effective distinguishing and accurate identification of the overlapping target are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of marine target recognition, and in particular to a marine target recognition method based on multimodal remote sensing images based on AI. Background Art

[0002] The AI-based multimodal remote sensing image maritime target recognition method uses remote sensing technology to collect multi-source image data of the ocean area, and combines multimodal image processing with AI analysis technology to accurately identify and detect maritime targets such as ships, tankers, icebergs, etc. This recognition method relies on multispectral, radar or high-resolution images obtained by satellites, drones or other aerial sensors, and integrates information from different modes through AI models to automatically or semi-automatically detect, classify and identify targets. This technology is widely used in marine monitoring, shipping management, maritime safety and environmental protection, and improves the efficiency and accuracy of maritime target recognition.

[0003] In the prior art, when identifying marine targets based on AI-based multimodal remote sensing images, first, the remote sensing images are preprocessed, including denoising, contrast enhancement, and geometric correction. Next, the features related to the target, such as shape, texture, spectral characteristics, etc., are extracted through feature extraction algorithms. Then, these features are analyzed and classified using a deep learning model to identify the target in the image. However, in a dense ship navigation environment, multiple targets may appear in the same area of ​​the remote sensing image and overlap with each other. This situation makes it difficult for the model to segment and identify the target separately, especially when the target is small and relatively close, the overlapping phenomenon will significantly increase the difficulty of classification. Long-distance shooting makes the target details blurred, and overlapping targets are difficult to effectively separate by conventional methods. And when the model cannot accurately identify the target, it may mistakenly identify multiple targets as a single target, or completely fail to identify some of the targets. Summary of the invention

[0004] The purpose of the present invention is to provide an AI-based multimodal remote sensing image marine target recognition method to address the shortcomings of the background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solution: a method for identifying marine targets in multimodal remote sensing images based on AI, comprising the following steps:

[0006] S1: acquiring an image of a plurality of ships sailing densely and with overlapping targets from a remote sensing image, wherein the image of overlapping targets includes ships of different densities and different types of overlapping situations;

[0007] S2: Extract features from the target overlapping images, and construct a dense target test dataset and an occlusion test dataset based on the extracted target overlapping image feature data;

[0008] S3: Use the instance segmentation model and run the model on the dense target test dataset to generate a pixel-level segmentation mask for each target. For overlapping targets, calculate the average boundary overlap rate between the segmentation mask of each target and its true boundary, analyze the change of the average boundary overlap rate, and evaluate the consistency of the target segmentation.

[0009] S4: Run the model in the occlusion test dataset to generate the detection box and segmentation mask of each target; measure the deviation between the occluded target detection box and the real target area, and analyze the abnormal changes of the deviation within a fixed time period to evaluate the anti-occlusion ability of the deep learning model;

[0010] S5: After comprehensively analyzing the consistency of target segmentation and the anti-occlusion ability of the deep learning model, the accuracy of the deep learning model in target recognition in a dense ship overlapping environment is evaluated;

[0011] S6: When the deep learning model cannot accurately identify targets in a dense and overlapping ship environment, the time series compensation method is used to infer the position and shape of the occluded part in the current frame through the motion compensation technology of the previous and next frames. By combining the time information of consecutive frames, the dense and overlapping targets can be further distinguished and identified.

[0012] Preferably, in S3, the average boundary overlap rate fluctuation index is generated after analyzing the change of the average boundary overlap rate of the target overlapping area, and the method for obtaining the average boundary overlap rate fluctuation index is:

[0013] Get the average boundary overlap rate IoU of N target overlapping areas and construct the corresponding time series {IoU1, IoU2, …, IoU N}; and set the sliding window size to w, that is, the window contains w consecutive IoU values; for each position i in the sequence, from IoU i The initial sliding window contains the next w IoU values ​​IoU i ,IoU i+1 ,…,IoU i+w-1 ; The IoU value in each window is calculated by weighted average; the weight of each window is set to w1, w2, ..., w w , the weighted average IoU in the sliding window is: Among them, μ i is the weighted average in the sliding window starting from the i-th IoU, IoU i+k-1 represents the kth IoU value in the sliding window, w k is the weight of the kth IoU value, and calculates the global IoU mean μ global , the expression is: For each sliding window, the weighted average IoU i , calculate its mean IoU with the global IoU value μ global Deviation Δ i , the expression is: Δ i =μ i -μ global ; Calculate the average boundary overlap rate fluctuation index, the expression is: Where N-w+1 is the total number of sliding window calculations, and WD is the average boundary overlap rate fluctuation index.

[0014] Preferably, in S4, after analyzing the abnormal change of the deviation within a fixed time period, a mask deviation abnormality index is generated, and the method for obtaining the mask deviation abnormality index is:

[0015] Collect the mask deviation data obtained in the W time period and establish the corresponding time series set {MD1, MD2, …, MD n}; where MD n is the mask deviation value at the nth time point; for the mask deviation sequence, its mean is calculated. For a single-dimensional mask deviation, the covariance matrix is ​​a scalar; for a multi-dimensional mask deviation, the covariance matrix is ​​a d×d matrix, where d is the dimension of the deviation vector. The calculation formula of the covariance matrix Σ is: T is the matrix transpose, for each time point MD i The mask deviation is calculated to calculate the distance D between each time point and the mean μ M MD i , the expression is: MD i -μ is the difference between the ith deviation and the mean, Σ -1 is the inverse matrix of the covariance matrix, MD i -μ T is the vector MD i -Transpose of μ; the distance D between each time point obtained and the mean μ M MD i The standard distance under normal conditions preset in the historical data is compared, and data points greater than or equal to the standard distance are regarded as abnormal points. The proportion of abnormal points to all data points is calculated, that is, the mask deviation abnormality index is calculated.

[0016] Preferably, in S5, after comprehensively analyzing the consistency of target segmentation and the anti-occlusion capability of the deep learning model, the accuracy of target recognition of the deep learning model in a dense ship overlapping environment is evaluated, specifically:

[0017] The average boundary overlap rate fluctuation index and the mask deviation anomaly index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model uses each group of first eigenvectors to predict the accuracy value label of target recognition in a dense ship overlapping environment as the prediction target, and takes minimizing the sum of prediction errors of the accuracy value labels of target recognition in all dense ship overlapping environments as the training target. The machine learning model is trained until the sum of prediction errors converges, and the model training is stopped. The accuracy value of target recognition in a dense ship overlapping environment is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0018] Preferably, the acquired accuracy value of target recognition in the dense ship overlapping environment is compared with a gradient standard threshold, the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the accuracy value of target recognition in the dense ship overlapping environment is compared with the first standard threshold and the second standard threshold respectively;

[0019] If the accuracy value of target recognition in the dense ship overlapping environment is greater than the second standard threshold, it means that the deep learning model has high accuracy in target recognition in the dense ship overlapping environment, and a high-accuracy recognition signal is generated and marked as accurate recognition;

[0020] If the accuracy value of target recognition in the dense ship overlapping environment is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it means that the accuracy of target recognition of the deep learning model in the dense ship overlapping environment is average, and a medium accuracy recognition signal is generated and marked as medium accuracy recognition;

[0021] If the accuracy value of target recognition in a dense ship overlapping environment is less than the first standard threshold, it means that the accuracy of target recognition of the deep learning model in a dense ship overlapping environment is low. At this time, a low-accuracy recognition signal is generated and marked as an inaccurate recognition.

[0022] Preferably, in S6, when the deep learning model cannot accurately identify the target in a dense ship overlapping environment, the position and shape of the blocked part in the current frame are inferred by the motion compensation technology of the previous and next frames based on the time series compensation method, specifically:

[0023] When the deep learning model cannot accurately identify the target in the dense ship overlapping environment, that is, the accuracy value of the target recognition in the dense ship overlapping environment is less than the first standard threshold, for the subsequent fixed time period, the image sequence of continuous frames {I t-1 , I t , I t+1}; where I t is the current frame, I t-1 and I t+1They are the images of the previous frame and the next frame respectively. There are multiple overlapping and dense objects in each frame.

[0024] The position of the target in the previous frame: Let the i-th target be in the previous frame I t-1 The position in is: p i，t-1 =(x i，t-1 ,y i，t-1 ); The position of the target in the current frame: The position of the target in the current frame I t The position in is: p i，t =(x i，t ,y i，t ); The position of the target in the next frame: The position of the target in the next frame I t+1 The position in is: p i，t+1 =(x i，t+1 ,y i，t+1 ); calculate the displacement through the positions of the previous and next frames. The displacement formula is: the displacement vector v of the target i Expressed as: v i =p i，t -p i，t-1 =(x i，t -x i，t-1 ,y i，t -y i，t-1 ); Speed ​​formula: target speed u i is the displacement per unit time: Where Δt is the time interval; based on the displacement information of the previous and next frames, the position of the occluded part of the target in the current frame is predicted The expression is: The formula is based on the displacement information of the previous two frames to infer the position of the occluded part of the target in the current frame.

[0025] Preferably, when the target is in the current frame I t When it is partially blocked in I t-1 and I t+1 The motion trajectory in is used to perform motion compensation, thereby inferring the position and shape of the blocked part, specifically:

[0026] In the time interval Δt, the motion trajectory is approximated by the optical flow equation, which is expressed as: x u+I y v+I t =0;I x ,I y is the brightness gradient of the image in the x and y directions, u and V are the horizontal and vertical components of the optical flow, representing the velocity component of the target, and I t is the change of image brightness in the time direction; based on the optical flow method, the motion compensation formula of the target is: Where Δv i =(u i , vi ) is the motion component obtained by optical flow calculation, which is used to compensate for the position of the occluded part of the target; the shape S of the occluded part in the current frame i The shape of the previous and next frames is estimated, and the expression is: S i，t-1 and S i，t+1 It is the shape of the target in the previous and next frames, which means that the shape of the occluded part in the current frame is inferred by the average of the shapes of the previous and next frames.

[0027] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0028] 1. This invention constructs a dense target test data set and an occlusion test data set by extracting features from dense navigation and target overlapping images, and uses an instance segmentation model to generate segmentation masks and evaluate segmentation consistency and anti-occlusion capabilities, thereby comprehensively analyzing the performance of the deep learning model. In addition, the average boundary coincidence rate fluctuation index and mask deviation anomaly index are introduced, and the target recognition accuracy of the model is evaluated through a polynomial regression model. According to the accuracy of the set threshold classification model, the corresponding recognition signal is generated, so that the deep learning model can more accurately deal with complex overlapping and occlusion scenes.

[0029] 2. When the model recognition is inaccurate, the present invention uses the time series compensation method and the motion compensation technology of the previous and next frames to infer the position and shape of the occluded target in the current frame, thereby further improving the accuracy of target recognition. The optical flow method is used for motion trajectory analysis and shape compensation to ensure that the target can be correctly identified even in the case of dense targets and partial occlusion. The dynamic time series information effectively enhances the robustness of the deep learning model and improves the accuracy of marine target recognition, especially in an environment with dense ships and severe overlap. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0031] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] For examples, see Figure 1 As shown, the AI-based multimodal remote sensing image marine target recognition method described in this embodiment includes the following steps:

[0034] S1: acquiring an image of a plurality of ships sailing densely and with overlapping targets from a remote sensing image, wherein the image of overlapping targets includes ships of different densities and different types of overlapping situations;

[0035] S2: Extract features from the target overlapping images, and construct a dense target test dataset and an occlusion test dataset based on the extracted target overlapping image feature data;

[0036] S3: Use the instance segmentation model and run the model on the dense target test dataset to generate a pixel-level segmentation mask for each target. For overlapping targets, calculate the average boundary overlap rate between the segmentation mask of each target and its true boundary, analyze the change of the average boundary overlap rate, and evaluate the consistency of the target segmentation.

[0037] S4: Run the model in the occlusion test dataset to generate the detection box and segmentation mask of each target; measure the deviation between the occluded target detection box and the real target area, and analyze the abnormal changes of the deviation within a fixed time period to evaluate the anti-occlusion ability of the deep learning model;

[0038] S5: After comprehensively analyzing the consistency of target segmentation and the anti-occlusion ability of the deep learning model, the accuracy of the deep learning model in target recognition in a dense ship overlapping environment is evaluated;

[0039] S6: When the deep learning model cannot accurately identify targets in a dense and overlapping ship environment, the time series compensation method is used to infer the position and shape of the occluded part in the current frame through the motion compensation technology of the previous and next frames. By combining the time information of consecutive frames, the dense and overlapping targets can be further distinguished and identified.

[0040] In S1, an image of a plurality of ships sailing densely and with overlapping targets is obtained from a remote sensing image, wherein the image of overlapping targets includes ships of different densities and different types of overlapping situations.

[0041] In a dense navigation scenario, the density of ships can present different distribution patterns according to the navigation environment and the type of ships. According to the distribution density of ships, it can be divided into the following typical situations:

[0042] Low-density navigation: The distance between ships is large, and each ship usually occupies a relatively independent area in the image, and there is little overlap between targets. It usually appears in open sea areas, and the sailing routes between ships are scattered, which is suitable for long-distance ocean transportation scenarios. Although there is less target overlap, due to the small size of the ships and the long distance between them, the model may miss small targets due to resolution issues.

[0043] Medium-density navigation: The distance between ships is small, and some targets overlap. Especially when the ships are sailing in the same direction, the outlines of the front and rear ships may overlap. This is common in sea areas with many ships or at the entrances of large ports. Although the ships are densely distributed, there is still a certain amount of space for some targets to be independently visible. The overlap of some targets increases the difficulty of segmenting the overlapping area, especially when the hull structures of multiple ships overlap, which may lead to incorrect boundary division.

[0044] High-density navigation: The distance between ships is very close, even close to each other or staggered, and multiple ship targets in the image overlap seriously, forming a dense target group. It occurs in large ports, berthing areas, or scenes where fleets work together. This type of dense navigation also occurs in scenes of emergency rescue or collective fishing operations. In such dense distribution situations, the model needs to be able to segment and distinguish a large number of overlapping targets, which is extremely difficult. The model is prone to merging multiple targets into one large target or ignoring the existence of small targets.

[0045] In remote sensing images, the overlapping of ships will present different types of overlapping due to different ocean environments, shooting angles and ship navigation status. The following are several common target overlapping situations:

[0046] Overlap in the front and rear directions (longitudinal overlap): When the ships are sailing in the same direction, the projections of the front and rear ships may partially overlap in the image. The front ship may block part of the area of ​​the rear ship, especially the overlap between the bow and the stern. Usually occurs in ports or waterways, when multiple ships enter or leave the same route one after another, typical longitudinal overlap occurs in the image. Front-to-back overlap affects the boundary extraction of the target, especially when the ships are of similar size, the model may regard these ships as one target, resulting in misidentification or missed detection.

[0047] Overlap in the left and right directions (lateral overlap): When ships travel side by side, the side profiles of the ships may overlap, making it difficult to accurately distinguish the boundaries of the target. In this case, the lateral overlap of the ships makes it difficult for the model to distinguish the specific boundaries and shapes of each ship. Left-right overlaps usually occur when large fleets sail in coordination or travel side by side. In port areas, such lateral overlaps are also prone to occur when ships are berthed. Lateral overlap not only affects the extraction of boundaries, but also interferes with the classification of ships (for example, different types of ships may be misclassified as the same category because of their similar appearance).

[0048] Partial overlap (local overlap): The ships are not completely overlapping, but some areas (such as the bow or stern) are blocked by other ships. This is usually because the ships are sailing at different depths or speeds, resulting in partial overlap of some parts of the hull. This situation usually occurs in an environment where ships are relatively freely distributed and sailing at different speeds, especially in narrow waterways, where partial overlap is prone to occur when ships shuttle. Partial overlap can cause the model to ignore the blocked parts during feature extraction, or separate some targets into multiple independent small targets, affecting the accuracy of overall recognition.

[0049] Full overlap (heavy occlusion): Some ships are completely obscured by other targets, only partially visible or barely visible. This is especially common in fleets that are moored or sailing in close proximity. Full occlusion is more common in mooring areas, docks, or near large fleets where there are a large number of ships. Large ships may completely obscure small and medium-sized ships behind them. Fully overlapping ships can cause the model to miss targets or mistakenly treat hidden targets as background. The model must infer the existence of occluded targets through other means, such as context awareness or motion trajectory inference.

[0050] Complex mixed overlap: including vertical, horizontal and partial overlap, multiple targets occlude each other or partially occlude each other, resulting in extremely complex recognition. This situation often occurs in emergency rescue scenarios, such as in severe weather conditions, when multiple rescue ships, fishing boats or other ships are mixed sailing, mooring, and supporting, and the order and angle of the ships vary. Complex mixed overlap makes target segmentation and classification extremely difficult. The model needs to be able to handle multiple different overlapping methods at the same time and ensure that each target is correctly identified and segmented.

[0051] S2: Extract features from the target overlapping images, and construct a dense target test dataset and an occlusion test dataset based on the extracted target overlapping image feature data.

[0052] Before extracting features from images, remote sensing images need to be preprocessed to ensure data quality and consistency. Use Gaussian filtering, median filtering and other methods to eliminate noise in remote sensing images to ensure that images are not disturbed by noise when extracting features. Use histogram equalization or adaptive contrast enhancement (CLAHE) technology to improve the contrast of the image, making the boundaries and shapes of the target clearer. If there are multiple data sources (such as optical, radar images, etc.), align and fuse different types of images to increase the richness of image information and enhance the expression of target features.

[0053] The remote sensing image is geometrically corrected to eliminate the geometric distortion caused by the sensor's viewing angle and terrain changes, so that the image is aligned with the actual geographic coordinates. Based on the target distribution, the part of the image containing the target overlap is cropped out to form a region of interest (ROI) so as to focus on feature extraction in dense target areas.

[0054] Feature extraction is to obtain useful information from the image for target recognition and segmentation. When dealing with images with overlapping targets, it is necessary to extract multiple features to ensure that the model can accurately distinguish the targets. Use edge detection algorithms such as the Sobel operator and the Canny operator to extract the edges of the target objects. This edge information is very important for segmentation and distinguishing overlapping targets. Calculate the contours of the ship targets and obtain geometric features such as the perimeter, area, and curvature of the contours of the targets. These features can help the model distinguish targets of similar size but different shapes. Analyze the texture information in the image through local binary patterns, especially in the target overlapping area, where texture features can help the model distinguish the boundaries between different targets. Calculate the gray level co-occurrence matrix of the target area to obtain the texture features of the target in the image, including contrast, correlation, entropy, etc. This helps to distinguish the texture differences of overlapping targets, especially when the appearance of the ships is similar.

[0055] Calculate the color distribution of the target area, especially in optical images, the color features of the ship can be used to distinguish targets of similar shape but different colors. If the image is a multispectral remote sensing image, the spectral features of different targets can be obtained by analyzing data in different bands (such as infrared, visible light), which is very effective for distinguishing overlapping targets with different optical properties. In order to deal with overlapping targets of different sizes and scales, a multi-scale feature extraction method can be used, combined with architectures such as feature pyramid networks (FPN) to obtain multi-scale information of the target. This ensures that even when there are large and small targets overlapping in the image, the model can still capture their respective features.

[0056] Based on the extracted feature data, we first construct a dense target test dataset to test the performance of the model in dense targets and complex overlapping scenes. Use annotation tools to annotate each target at the pixel level. Especially in the target overlapping area, ensure that each target has accurate boundary labels, which can be used to train and test the model. Define the criteria for dense targets, such as when the distance between targets is less than a certain threshold (such as the target boundaries are less than 20 pixels apart), they are considered dense targets. Group the targets in the image according to their density to ensure that the dataset contains enough dense target instances. According to different densities, extract ship navigation images with different densities from the processed remote sensing images. Ensure that the dataset contains scenes from low density to high density so as to comprehensively evaluate the performance of the model. Save the feature extraction results (such as geometric features, texture features, depth features, etc.) of each dense target into the feature library for subsequent testing and evaluation. In order to enhance the robustness of the dataset, data augmentation operations such as rotation, scaling, and translation can be performed on the dense target images to generate more dense scene samples.

[0057] Construct an occlusion test dataset: Extract vessels in the image that are partially occluded by other objects, such as the bow or stern of a ship being blocked by other ships. This type of data is used to evaluate the model's ability to handle partially occluded objects. Select vessels that are completely occluded by other objects or only a small part is exposed. This type of data places higher demands on the model's reasoning ability. According to the degree of occlusion, classify the objects in the image into partially occluded and severely occluded. Ensure that the dataset contains objects with different degrees of occlusion so that the performance of the model can be fully evaluated under various occlusion conditions. When sampling, consider multiple types of occlusion scenarios, such as objects being occluded by other ships, waves, or floating objects, to ensure the diversity of the occlusion test dataset. Manually annotate the boundaries of occluded objects, especially partially occluded objects. Although some areas are occluded, it is still necessary to generate complete segmentation labels for these objects in order to evaluate whether the model can accurately infer and recover the occluded objects. For severely occluded objects, generate special markers for the occluded areas and determine whether the model can handle these difficult scenarios in subsequent evaluations.

[0058] In order to allow the model to better learn the characteristics of occluded targets, the dataset can be expanded by simulating different degrees of occlusion to enhance the model's adaptability to occlusion phenomena. For example, new training samples can be generated by randomly adding occluders to the image. Ensure that the feature set of the occluded target (such as geometry, texture, and depth features) is consistent with the features in the dense target test dataset so that the performance of the model in different scenarios can be compared during evaluation.

[0059] S3: Use the instance segmentation model and run the model on a dense target test dataset to generate a pixel-level segmentation mask for each target. For overlapping targets, calculate the average boundary overlap ratio between the segmentation mask of each target and its true boundary, analyze the changes in the average boundary overlap ratio, and evaluate the consistency of the target segmentation.

[0060] Use commonly used instance segmentation models such as MaskR-CNN, DeepLabV3, etc., load pre-trained models or models specially trained for dense target scenes at sea. Input the constructed dense target test dataset into the model. The images in this dataset contain densely sailing ships and overlapping targets. The model performs pixel-level segmentation on the input image and generates a segmentation mask for each target. The mask is a binary image in which each pixel value indicates whether the pixel is part of the target. When dealing with overlapping target areas, the model tries to distinguish the boundaries of different targets, even if they overlap in some areas. Each target will have a separate segmentation mask.

[0061] Each image in the dense target test dataset has been manually annotated with accurate real segmentation boundaries. This annotation information can be used as a criterion for evaluating the segmentation effect of the model. The segmentation mask of each target generated by the instance segmentation model represents the model's predicted segmentation result for the target.

[0062] Calculate the average boundary overlap rate IoU to measure the overlap between the predicted mask and the true boundary. The calculation formula is: Where Aintersection is the intersection area between the predicted mask and the true boundary, A union is their union area. For each object, the intersection area and union area between the predicted mask and the true boundary are calculated respectively. The intersection area is divided by the union area to get the IoU of each object. The IoU is calculated for all objects in each image, and all IoUs in the image are averaged to get the average boundary overlap rate of the image.

[0063] For target overlapping areas, the model will generate overlapping masks of multiple targets during the prediction process. It is necessary to calculate the IoU of these overlapping areas separately and focus on analyzing the boundary overlap rate of these areas. In overlapping areas, the IoU value may decrease due to the blurred target boundaries, so the mask generation can be optimized by adjusting the model threshold or through post-processing (such as Soft-NMS).

[0064] After analyzing the change of the average boundary overlap rate of the target overlapping area, the average boundary overlap rate fluctuation index is generated. The method for obtaining the average boundary overlap rate fluctuation index is:

[0065] Get the average boundary overlap rate IoU of N target overlapping areas and construct the corresponding time series {IoU1, IoU2, ..., IoU N}; and set the sliding window size to w (usually 3, 5 or 7), that is, the window contains w consecutive IoU values; for each position i in the sequence, from IoU i The initial sliding window contains the next w IoU values ​​(IoU i , IoU i+1 ,...,IoU i+w-1 ); The IoU value in each window is calculated by weighted average; the weight of each window is set to w1, w2, ..., w w (Generally satisfied ). The weighted average IoU within the sliding window is: Among them, μi is the weighted average in the sliding window starting from the i-th IoU, IoU i+k-1 represents the kth IoU value in the sliding window, w k is the weight of the kth IoU value. In order to measure the difference between the weighted average IoU in each window and the overall IoU mean, the global IoU mean μ is first calculated. global , the expression is: For each sliding window, the weighted average IoU i , calculate its mean IoU with the global IoU value μ global Deviation Δ i , the expression is: Δ i =|μ i -μ global |; Calculate the average boundary overlap rate fluctuation index, the expression is: Where N-w+1 is the total number of sliding window calculations, because the weighted average IoU of N-w+1 windows can be calculated from i=1 to i=N-w+1 respectively, and WD is the average boundary overlap rate fluctuation index.

[0066] When the average boundary overlap rate fluctuation index is larger, it means that the segmentation performance of the model fluctuates greatly in different target overlapping areas. Specifically, the model is more accurate in segmentation in some areas, but poor in segmentation in other areas. This inconsistency reflects the lack of stability of the model when dealing with complex scenes. It may be disturbed when dealing with dense and overlapping targets, resulting in unstable boundaries of the segmentation results or difficulty in accurately distinguishing target overlapping areas. Therefore, the larger the fluctuation index, the poorer the target segmentation consistency of the model, and it is easy to have large segmentation errors when dealing with complex scenes.

[0067] The smaller the average boundary overlap rate fluctuation index, the more consistent the segmentation performance of the model in different target overlapping areas. Specifically, whether it is a dense target or an overlapping area, the segmentation effect of the model is relatively stable, and the average boundary overlap rate fluctuation index is small, indicating that the model can handle different scenes and target types well. A lower fluctuation index means that the model's segmentation results for the target boundary are more accurate and consistent, and the segmentation effect will not be significantly affected by target overlap or environmental complexity. Therefore, the smaller the fluctuation index, the better the consistency of the target segmentation, and the more reliable and robust the model's performance in various scenes.

[0068] S4: Run the model in the occlusion test dataset to generate a detection box and segmentation mask for each target; measure the deviation between the occluded target detection box and the true target area, and analyze the abnormal changes in the deviation within a fixed time period to evaluate the anti-occlusion ability of the deep learning model.

[0069] Use the constructed occlusion test dataset and input the pre-trained deep learning model (such as Mask R-CNN, YOLO, etc.). These images contain different degrees of target occlusion, such as partial occlusion and full occlusion.

[0070] The model first generates a detection box for each target in the image. These boxes are the model's preliminary positioning results for each target. The output of the detection box: the rectangular area of ​​each target selected by the model, which contains the main part of the target.

[0071] Based on the generated detection box, the model generates a pixel-level segmentation mask for each target. The mask represents the specific shape and boundary of the target. For occluded targets, the model may only generate a partial mask. The unoccluded part will be clearer, while the mask of the occluded part may be incomplete or have errors.

[0072] Each object in the occlusion test dataset has been manually annotated with an accurate ground truth bounding box and a true segmentation mask. These annotated data are used as a benchmark for evaluating model performance.

[0073] The detection box deviation BBD is the deviation between the detection box generated by the measurement model and the real detection box. The calculation expression of the detection box deviation is: In the formula, A pred is the area of ​​the detection box generated by the model, A true is the area of ​​the real target detection box. The larger the BBD value, the greater the difference between the detection box generated by the model and the real box. The IoU between the predicted detection box and the real detection box is calculated as: In the formula, A intersection is the overlapping area between the predicted box and the real box, A union is the area of ​​their union. A lower IoU indicates that the model has a large error in generating the detection box for the occluded area. By comparing the difference between the generated segmentation mask and the true mask, the accuracy of the model's segmentation under occlusion is measured. The mask deviation MD can be calculated by the pixel-level error, expressed as: P pred is the number of pixels in the predicted segmentation mask, P true is the number of pixels in the true mask. A higher MD indicates that the mask generated by the model deviates more from the true target mask.

[0074] After analyzing the abnormal changes of the deviation within a fixed time period, the mask deviation anomaly index is generated. The method for obtaining the mask deviation anomaly index is as follows:

[0075] The mask deviation data acquired in the W time period are collected and the corresponding time series set {MD1, MD2, ..., MD n}; where MD n is the mask deviation value at the nth time point; for the mask deviation sequence, its mean is calculated. The covariance matrix is ​​used to measure the linear relationship between the mask deviations of each dimension. For a single-dimensional mask deviation, the covariance matrix is ​​a scalar; for a multi-dimensional mask deviation, the covariance matrix is ​​a d×d matrix (where d is the dimension of the deviation vector). The calculation formula of the covariance matrix ∑ is: T is the matrix transpose, for each time point MD i The mask deviation is calculated to calculate the distance D between each time point and the mean μ M (MD i ), the expression is: MD i -μ is the difference between the ith deviation and the mean, Σ -1 is the inverse matrix of the covariance matrix, (MD i -μ) T is the vector MD i -μ transpose; the larger the distance, the farther the mask deviation at that time point is from the center of the overall distribution, which means that the deviation point may be an outlier. The distance D between each time point and the mean μ is obtained. M (MD i ) is compared with the standard distance under normal conditions preset in the historical data, and the data points greater than or equal to the standard distance are regarded as abnormal points. The proportion of abnormal points to all data points is calculated, that is, the mask deviation abnormality index is calculated.

[0076] When the mask deviation anomaly index is larger, it means that the segmentation performance of the deep learning model when dealing with occluded targets has large fluctuations and more anomalies occur. This means that the model's prediction of the segmentation boundary or shape of the target in the occluded scene is significantly different from the true value, which is prone to false detection or missed detection. When the deviation anomaly index is large, the model may not be able to effectively cope with complex occlusion situations and show poor anti-occlusion capabilities, especially when the target is partially or completely occluded, the segmentation mask deviation of the model increases significantly, affecting the robustness and reliability of the model.

[0077] The smaller the mask deviation anomaly index is, the more stable the segmentation performance of the model is under different occlusion levels, the smaller the deviation fluctuation is, and no obvious anomalies occur. A smaller anomaly index indicates that the model can maintain a high accuracy when processing occluded scenes, and the shape and boundary segmentation of the occluded target is closer to the true value, showing a strong anti-occlusion ability. Even when the target is partially occluded or interfered by a complex background, the model can still effectively identify the complete outline of the target and maintain a low segmentation error, indicating that it has good robustness in complex scenes.

[0078] S5: After comprehensively analyzing the consistency of target segmentation and the anti-occlusion ability of the deep learning model, the accuracy of target recognition of the deep learning model in a dense ship overlapping environment is evaluated.

[0079] The average boundary overlap rate fluctuation index and the mask deviation anomaly index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model uses each group of first eigenvectors to predict the accuracy value label of target recognition in a dense ship overlapping environment as the prediction target, and takes minimizing the sum of prediction errors of the accuracy value labels of target recognition in all dense ship overlapping environments as the training target. The machine learning model is trained until the sum of prediction errors converges, and the model training is stopped. The accuracy value of target recognition in a dense ship overlapping environment is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0080] The method for obtaining the accuracy value of target recognition in a dense ship overlapping environment is as follows: from the first eigenvector training data of the trained machine learning model, the corresponding function expression is obtained: WR=FWD,EQ; where F is the output function of the model, WD is the average boundary overlap rate fluctuation index, EQ is the mask deviation anomaly index, and WR is the accuracy value of target recognition in a dense ship overlapping environment.

[0081] The acquired accuracy value of target recognition in the dense ship overlapping environment is compared with the gradient standard threshold, the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the accuracy value of target recognition in the dense ship overlapping environment is compared with the first standard threshold and the second standard threshold respectively;

[0082] If the accuracy value of target recognition in the dense ship overlapping environment is greater than the second standard threshold, it means that the deep learning model has high accuracy in target recognition in the dense ship overlapping environment, and a high-accuracy recognition signal is generated and marked as accurate recognition;

[0083] If the accuracy value of target recognition in the dense ship overlapping environment is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it means that the accuracy of target recognition of the deep learning model in the dense ship overlapping environment is average, and a medium accuracy recognition signal is generated and marked as medium accuracy recognition;

[0084] If the accuracy value of target recognition in a dense ship overlapping environment is less than the first standard threshold, it means that the accuracy of target recognition of the deep learning model in a dense ship overlapping environment is low. At this time, a low-accuracy recognition signal is generated and marked as an inaccurate recognition.

[0085] S6: When the deep learning model cannot accurately identify targets in a dense and overlapping ship environment, the time series compensation method is used to infer the position and shape of the occluded part in the current frame through the motion compensation technology of the previous and next frames. By combining the time information of consecutive frames, the dense and overlapping targets can be further distinguished and identified.

[0086] When the deep learning model cannot accurately identify the target in the dense ship overlapping environment, that is, the accuracy value of the target recognition in the dense ship overlapping environment is less than the first standard threshold, for the subsequent fixed time period, the image sequence of continuous frames {I t-1 , I t , I t+1}; where I t is the current frame, I t-1 and I t+1 They are the images of the previous frame and the next frame respectively. There are multiple overlapping and dense targets (such as ships) in each frame, and the movement of these targets can be modeled by the displacement between frames.

[0087] The position of the target in the previous frame: Let the i-th target be in the previous frame I t-1 The position in is: p i，t-1 =(x i，t-1 ,y i，t-1 ); The position of the target in the current frame: The position of the target in the current frame I t The position in is: p i，t=(x i，t ,y i，t ); The position of the target in the next frame: The position of the target in the next frame I t+1 The position in is: p i，t+1 =(x i，t+1 ,y i，t+1 ); The movement of the target can be represented by speed or displacement. Assuming the target's speed is constant, its displacement can be calculated by the position of the previous and next frames.

[0088] Displacement formula: The displacement vector v of the target i Expressed as: v i =p i，t -p i，t-1 =(x i，t -x i，t-1 ,y i，t -y i，t-1 ); Speed ​​formula: target speed u i is the displacement per unit time: Where Δt is the time interval; based on the displacement information of the previous and next frames, the position of the occluded part of the target in the current frame is predicted The expression is: The formula is based on the displacement information of the previous two frames to infer the position of the occluded part of the target in the current frame.

[0089] When the target is in the current frame I t When it is partially blocked in I t-1 and I t+1 The motion trajectory in is used to perform motion compensation, thereby inferring the position and shape of the blocked part, specifically:

[0090] In a small time interval Δt, assuming that the brightness of the target remains unchanged, the motion trajectory is approximated by the optical flow equation, which is expressed as: x u+I y v+I t =0;I x ,I y is the brightness gradient of the image in the x and y directions, u and v are the horizontal and vertical components of the optical flow, representing the velocity component of the target, and I t is the change of image brightness in the time direction. Based on the optical flow method, the motion compensation formula of the target is: Where Δv i =(u i , v i ) is the motion component obtained by optical flow calculation, which is used to compensate for the position of the occluded part of the target; the shape of the target can be inferred through the continuous changes between frames. Assuming that the shape of the target remains consistent in a short period of time, the shape S of the occluded part in the current framei It can be estimated by the shape of the previous and next frames, and the expression is: S i,t-1 and S i,t+1 It is the shape of the target in the previous and next frames, which means that the shape of the occluded part in the current frame is inferred by the average of the shapes of the previous and next frames.

[0091] Through the time information of consecutive frames, the motion trajectory of each target can be tracked and its position in each frame can be determined. The position and shape of the occluded target can be predicted using trajectory tracking methods (such as Kalman filter), specifically:

[0092] The trajectory tracking formula (Kalman filter prediction model) is: K is the Kalman gain matrix, z i,t is the measured target position, and H is the measurement matrix.

[0093] Based on the motion compensation information of the previous and next frames, the time series analysis method is used to distinguish dense and overlapping targets: a time series prediction model (such as the ARIMA model) is used to predict the future position of the target and distinguish overlapping targets. The next position of the target is predicted by the time series model, which further helps to distinguish different overlapping targets.

[0094] In this application, through the time series compensation method, the information of the previous and next frames can be used to infer the position and shape of the occluded target in the current frame. The specific steps include compensating through the position and motion trajectory of the previous and next frames, calculating the motion component of the target through the optical flow method, inferring the motion of the occluded target by combining the trajectory tracking method such as the Kalman filter, and using the time series analysis model to distinguish and predict dense and overlapping targets. Through these methods, the target recognition accuracy of the deep learning model in a dense ship overlapping environment can be effectively improved.

[0095] In this embodiment, after obtaining images of densely sailing ships and overlapping targets in remote sensing images, firstly, feature extraction is performed on these images, and a dense target test data set and an occlusion test data set are constructed based on the feature data. Then, the instance segmentation model is used to run on the dense target test data set to generate a pixel-level segmentation mask of the target, and the average boundary overlap rate between the segmentation mask and the real boundary is calculated to evaluate the consistency of the target segmentation. Then, in the occlusion test data set, the detection frame and segmentation mask of the target are generated, the deviation between the detection frame and the real area is measured, and the abnormal changes in the deviation are analyzed to evaluate the anti-occlusion ability of the model. On this basis, the segmentation consistency and anti-occlusion ability are comprehensively analyzed to evaluate the accuracy of target recognition of the deep learning model in a dense ship overlapping environment. If the model recognition is inaccurate, based on the time series compensation method, the motion compensation technology of the previous and next frames is used to infer the position and shape of the occluded part in the current frame, and the time information of the continuous frames is combined to further distinguish and identify the dense and overlapping targets.

[0096] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0097] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0098] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0099] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for identifying marine targets in multimodal remote sensing images based on AI, characterized by: The steps include: S1: acquiring an image of a plurality of ships sailing densely and with overlapping targets from a remote sensing image, wherein the image of overlapping targets includes ships of different densities and different types of overlapping situations; S2: Extract features from the target overlapping images, and construct a dense target test dataset and an occlusion test dataset based on the extracted target overlapping image feature data; S3: Using the instance segmentation model, run the model on the dense object test dataset to generate pixel-level segmentation masks for each object; For overlapping targets, the average boundary overlap ratio between the segmentation mask of each target and its true boundary is calculated. After analyzing the change of the average boundary overlap ratio, the consistency of the target segmentation is evaluated. S4: Run the model on the occlusion test dataset to generate the detection box and segmentation mask of each object, and generate the detection box and segmentation mask of each object; The deviation between the occluded target detection frame and the real target area is measured, and the abnormal changes of the deviation within a fixed time period are analyzed to evaluate the anti-occlusion ability of the deep learning model; S5: After comprehensively analyzing the consistency of target segmentation and the anti-occlusion ability of the deep learning model, the accuracy of the deep learning model in target recognition in a dense ship overlapping environment is evaluated; S6: When the deep learning model cannot accurately identify targets in a dense and overlapping ship environment, the time series compensation method is used to infer the position and shape of the occluded part in the current frame through the motion compensation technology of the previous and next frames. By combining the time information of consecutive frames, the dense and overlapping targets can be further distinguished and identified.

2. The method for identifying marine targets in multimodal remote sensing images based on AI according to claim 1, characterized in that: In S3, the average boundary overlap rate fluctuation index is generated after analyzing the change of the average boundary overlap rate of the target overlapping area. The method for obtaining the average boundary overlap rate fluctuation index is: Get the average boundary overlap rate IoU of N target overlapping areas and construct the corresponding time series {IoU1, IoU2, …, IoU N }; and set the sliding window size to w, that is, the window contains w consecutive IoU values; for each position i in the sequence, from IoU i The initial sliding window contains the next w IoU values ​​(IoU i ,IoU i+1 ,…,IoU i+w-1 ); The IoU value in each window is calculated by weighted average; the weight of each window is set to w1, w2, …, w w , the weighted average IoU in the sliding window is: Among them, μ i is the weighted average in the sliding window starting from the i-th IoU, IoU i+k-1 represents the kth IoU value in the sliding window, w k is the weight of the kth IoU value, and calculates the global IoU mean μ global , the expression is: For each sliding window, the weighted average IoU i , calculate its mean IoU with the global IoU value μ global Deviation Δ i , the expression is: Δ i =|μ i -μ global |; Calculate the average boundary overlap rate fluctuation index, the expression is: Where N-w+1 is the total number of sliding window calculations, and WD is the average boundary overlap rate fluctuation index.

3. The method for identifying marine targets using multimodal remote sensing images based on AI according to claim 2, characterized in that: In S4, the abnormal change of the deviation within a fixed time period is analyzed to generate a mask deviation abnormality index. The method for obtaining the mask deviation abnormality index is: Collect the mask deviation data obtained in the W time period and establish the corresponding time series set {MD1, MD2, …, MD n }; where MD n is the mask deviation value at the nth time point; for the mask deviation sequence, its mean is calculated. For a single-dimensional mask deviation, the covariance matrix is ​​a scalar; for a multi-dimensional mask deviation, the covariance matrix is ​​a d×d matrix, where d is the dimension of the deviation vector. The calculation formula of the covariance matrix Σ is: T is the matrix transpose, for each time point MD i The mask deviation is calculated to calculate the distance D between each time point and the mean μ M (MD i ), the expression is: MD i -μ is the difference between the ith deviation and the mean, Σ -1 is the inverse matrix of the covariance matrix, MD i -μ) T is the vector MD i -Transpose of μ; the distance D between each time point obtained and the mean μ M (MD i ) is compared with the standard distance under normal conditions preset in the historical data, and the data points greater than or equal to the standard distance are regarded as abnormal points. The proportion of abnormal points to all data points is calculated, that is, the mask deviation abnormality index is calculated.

4. The method for identifying marine targets using multimodal remote sensing images based on AI according to claim 3 is characterized by: In S5, after comprehensively analyzing the consistency of target segmentation and the anti-occlusion ability of the deep learning model, the accuracy of the deep learning model in target recognition in a dense ship overlapping environment is evaluated. Specifically: The average boundary overlap rate fluctuation index and the mask deviation anomaly index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model uses each group of first eigenvectors to predict the accuracy value label of target recognition in a dense ship overlapping environment as the prediction target, and takes minimizing the sum of prediction errors of the accuracy value labels of target recognition in all dense ship overlapping environments as the training target. The machine learning model is trained until the sum of prediction errors converges, and the model training is stopped. The accuracy value of target recognition in a dense ship overlapping environment is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

5. The method for identifying marine targets using multimodal remote sensing images based on AI according to claim 4, characterized in that: The acquired accuracy value of target recognition in the dense ship overlapping environment is compared with the gradient standard threshold, the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the accuracy value of target recognition in the dense ship overlapping environment is compared with the first standard threshold and the second standard threshold respectively; If the accuracy value of target recognition in the dense ship overlapping environment is greater than the second standard threshold, it means that the deep learning model has high accuracy in target recognition in the dense ship overlapping environment, and a high-accuracy recognition signal is generated and marked as accurate recognition; If the accuracy value of target recognition in the dense ship overlapping environment is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it means that the accuracy of target recognition of the deep learning model in the dense ship overlapping environment is average, and a medium accuracy recognition signal is generated and marked as medium accuracy recognition; If the accuracy value of target recognition in a dense ship overlapping environment is less than the first standard threshold, it means that the accuracy of target recognition of the deep learning model in a dense ship overlapping environment is low. At this time, a low-accuracy recognition signal is generated and marked as an inaccurate recognition.

6. The method for identifying marine targets in multimodal remote sensing images based on AI according to claim 1, characterized in that: In S6, when the deep learning model cannot accurately identify the target in a dense ship overlapping environment, the position and shape of the blocked part in the current frame are inferred by the motion compensation technology of the previous and next frames based on the time series compensation method, specifically: When the deep learning model cannot accurately identify the target in the dense ship overlapping environment, that is, the accuracy value of the target recognition in the dense ship overlapping environment is less than the first standard threshold, for the subsequent fixed time period, the image sequence of continuous frames {I t-1 ,I t ,I t+1 }; where I t is the current frame, I t-1 and I t+1 They are the images of the previous frame and the next frame respectively. There are multiple overlapping and dense objects in each frame. The position of the target in the previous frame: Let the i-th target be in the previous frame I t-1 The position in is: p i,t-1 =x i,t-1 ,y i,t-1 ); The position of the target in the current frame: The position of the target in the current frame I t The position in is: p i,t =x i,t ,y i,t ); The position of the target in the next frame: The position of the target in the next frame I t+1 The position in is: p i,t+1 =x i,t+1 ,y i,t+1 ); calculate the displacement through the positions of the previous and next frames. The displacement formula is: the displacement vector v of the target i Expressed as: v i =p i,t -p i,t-1 =(x i,t -x i,t-1 ,y i,t -y i,t-1 ); Speed ​​formula: The speed of the target u i is the displacement per unit time: Where Δt is the time interval; based on the displacement information of the previous and next frames, the position of the occluded part of the target in the current frame is predicted The expression is: The formula is based on the displacement information of the previous two frames to infer the position of the occluded part of the target in the current frame.

7. The method for identifying marine targets in multimodal remote sensing images based on AI according to claim 6, characterized in that: When the target is in the current frame I t When it is partially blocked in I t-1 and I t+1 The motion trajectory in is used to perform motion compensation, thereby inferring the position and shape of the blocked part, specifically: In the time interval Δt, the motion trajectory is approximated by the optical flow equation, which is expressed as: x u+I y v+I t =0;I x ,I y is the brightness gradient of the image in the x and y directions, u and v are the horizontal and vertical components of the optical flow, representing the velocity component of the target, and I t is the change of image brightness in the time direction; based on the optical flow method, the motion compensation formula of the target is: Where Δv i =(u i ,v i ) is the motion component obtained by optical flow calculation, which is used to compensate for the position of the occluded part of the target; the shape S of the occluded part in the current frame i The shape of the previous and next frames is estimated, and the expression is: S i,t-1 and S i,t+1 It is the shape of the target in the previous and next frames, which means that the shape of the occluded part in the current frame is inferred by the average of the shapes of the previous and next frames.

Images