SAR image inclined frame detection method based on rotation matrix estimation

Abstract

The invention discloses an SAR image inclined frame detection method based on rotation matrix estimation. The SAR image inclined frame detection method comprises the following steps: constructing an SAR image data set; according to the method, an SAR image inclined frame detection model is constructed, the SAR image inclined frame detection model adopts a neural network model, the neural network model comprises a backbone part, a neck part and a head part, the backbone part is used for realizing feature extraction of an input image, the neck part is used for realizing fusion of multi-scale features, and the head part is used for realizing detection of a target; using the constructed SAR image data set to train the constructed neural network model; and utilizing the trained neural network to realize inclined frame detection of the SAR image target. According to the invention, the cascade detection head is improved, so that the edge of the target bounding box is more stable.

Description

Technical Field

[0001] The invention belongs to SAR image detection technology, in particular to a SAR image oblique frame detection method based on rotation matrix estimation. Background Art

[0002] After decades of development, synthetic aperture radar (SAR) image target detection methods have produced a large number of methods. Compared with optical images, SAR images have poor visual readability, low resolution, coherent speckle noise, geometric distortion and other problems, which bring difficulties and challenges to target detection. The current mainstream methods include target detection methods based on structural features, grayscale features, texture features and deep learning, among which the target detection algorithm based on deep learning is the most widely used. Conventional deep learning algorithms usually use horizontal boxes to detect ship targets. When using horizontal boxes to detect targets, the box contains more background areas and there are a large number of redundant pixels, which will cause classification difficulties. At the same time, densely arranged ships will also cause the non-maximum suppression algorithm to suppress effective targets, resulting in missed targets. The oblique box target detection algorithm can effectively solve the above drawbacks, but it will also cause new problems.

[0003] The existing SAR image oblique frame target detection method has a series of problems such as angle boundary discontinuity, target detection is greatly affected by angle, and boundary instability. At present, there is still a lack of a method to improve the above problems and improve the accuracy of ship target detection. Summary of the invention

[0004] The purpose of the present invention is to provide a SAR image oblique frame detection method based on rotation matrix estimation.

[0005] The technical solution to achieve the purpose of the present invention is: a SAR image oblique frame detection method based on rotation matrix estimation, comprising:

[0006] Construct a SAR image dataset, wherein the images in the dataset are SAR images of the target to be detected, and the targets in the SAR images are annotated using tilted boxes;

[0007] Constructing a SAR image oblique frame detection model, wherein the SAR image oblique frame detection model adopts a neural network model, wherein the neural network model includes: a backbone part, a neck part, and a head part, wherein the backbone part is used to realize feature extraction of an input image, the neck part is used to realize fusion of multi-scale features, and the head part realizes detection of a target;

[0008] Using the constructed SAR image dataset, the constructed neural network model is trained;

[0009] The trained neural network is used to realize the oblique frame detection of SAR image targets.

[0010] Preferably, the head part includes a convolution module, a distance weight mask generation module, an angle weight mask generation module and a loss function calculation module based on rotation matrix estimation; the neural network model processes the image as follows:

[0011] Step S2-1, the resolution is The SAR image is extracted from the backbone part and the neck part, and multi-scale feature fusion is performed in turn to obtain The channel resolution is Feature map, where is a constant, and the feature map is sent to the convolution module; The convolution module decodes the feature map to obtain the center point coordinate information of the target bounding box. The center point coordinate information contains the center point positions of N possible target bounding boxes. Step S2-2, the distance weight mask generation module generates a distance weight mask using the center point coordinate information output by the convolution module; Step S2-3, the distance weighted convolution module uses the feature map output by the convolution module in step S2-1 and the distance weighted mask output by the distance weighted mask generation module to implement distance weighting of the feature map output by the convolution module, that is, each channel of the feature map output by the convolution module is multiplied by the pixel value of the corresponding position of the distance weighted mask one by one to obtain a distance-weighted feature map, and the distance-weighted feature map is decoded to obtain the angle information of the target bounding box; Step S2-4, the angle weight mask generation module generates an angle weight mask using the angle information output by the distance weighted convolution module; Step S2-5, the angle-weighted convolution module uses the distance-weighted feature map output by the distance-weighted convolution module in step S2-3 and the angle-weighted mask output by the angle-weighted mask generation module to achieve angle weighting of the distance-weighted feature map, that is, each channel of the distance-weighted feature map is multiplied by the pixel value of the corresponding position of the angle-weighted mask one by one to obtain a new feature map, and the distance-weighted and angle-weighted feature maps are decoded to obtain the length and width information of the target bounding box; Step S2-6, the loss function calculation module based on the rotation matrix estimation uses the center point coordinate information output by the convolution module, the angle information output by the distance weighted convolution module, and the length and width information output by the angle weighted convolution module to calculate the loss value.

[0012] Preferably, in step S2-2, the distance weight mask generation module uses the center point coordinate information output by the convolution module to generate a distance weight mask in the following specific method: Assume The center point of the bounding box of possible targets is , then The pixel coordinates of the feature map where the possible target is located are ,in , , ; for the first possible targets creates a blank single-channel image with the same resolution as the feature map , The pixel value of each pixel Satisfy the formula , in, , For a single channel image No. The horizontal and vertical coordinates of the pixel points, is a constant; Will Single channel image corresponding to the target The corresponding pixels are added together, and the result is passed through the sigmoid function to obtain a unique single-channel image, which is the distance weighted mask.

[0013] Preferably, the angle weight mask generation module uses the angle information output by the distance weighted convolution module to generate the angle weight mask in the following specific method: Assume The bounding box angles of possible targets are ,in ; For the possible targets creates a blank single-channel image with the same resolution as the distance-weighted feature map of the distance-weighted convolution module , The pixel value of each pixel Satisfy the formula , in, , In the formula, ( , ) is a single channel image No. The horizontal and vertical coordinates of the pixel points, , For the The horizontal and vertical coordinates of the pixel points in the feature map channel where the target is located, is a constant; Will Single channel image of a target The corresponding pixels are summed and the result is passed through the sigmoid function to obtain a unique single-channel image, which is the angle weight mask.

[0014] Preferably, the loss function calculation module based on rotation matrix estimation uses the center point coordinate information output by the convolution module, the angle information output by the distance weighted convolution module, and the length and width information output by the angle weighted convolution module. The specific method for calculating the loss value is: The target oblique frame information includes ( , , , , , ),in Flag bit, used to calculate Value, where , , Wherein they are the horizontal coordinate and vertical coordinate of the target frame, , are the width and length of the target box respectively. is the angle information of the target frame, and the loss function is , in , is the center distance, is the aspect ratio loss, is the weight factor, is the square of the diagonal length, They are the minimum and maximum horizontal and vertical coordinates of the four vertices of the target frame, respectively. is the intersection-over-union ratio between the predicted box and the true box; ; is the category loss; , , , , They are the horizontal coordinate, vertical coordinate, width, length and angle of the center point of the prediction box respectively.

[0015] Compared with the prior art, the present invention has the following significant advantages: the present invention makes the edge of the target bounding box more stable by improving the cascade detection head; the present invention adopts an improved loss function, which effectively improves the problem of angle boundary discontinuity and significantly reduces the influence of angle on target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a training and detection flowchart of a SAR image oblique frame detection model based on rotation matrix estimation.

[0017] Figure 2 Schematic diagram of a method for generating distance weighted masks for the distance weighted mask generation module.

[0018] Figure 3Schematic diagram of a method for generating an angle weight mask for an angle weight mask generation module.

[0019] Figure 4 is the SAR image used in the embodiment.

[0020] Figure 5 Schematic diagram of the generated single-channel distance weighted mask.

[0021] Figure 6 Schematic diagram of the generated single-channel angle weight mask.

[0022] Figure 7 It is the labeled SAR image to be detected and the detection result diagram. DETAILED DESCRIPTION

[0023] In order to facilitate the understanding of the present invention, the present invention will be described more comprehensively with reference to the accompanying drawings. The present invention can be implemented in different forms and is not limited to the embodiments described in the text. On the contrary, the embodiments are provided to make the content disclosed in the present invention more thorough and comprehensive. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0024] In this embodiment, it is assumed that a RSDD-SAR image dataset is selected, in which an image such as Figure 4 (Left), the resolution is 512*512, and the feature map generated by the SAR image is as follows Figure 4 (Right), the resolution is 32*32, there are 64 channels, and it contains an object. From the object's annotation information, we can know that the position of the object on the feature map is (16, 16), and the angle of the object is 45°. After prior calculation, they are 8 and 12 respectively.

[0025] Step S1, selecting training samples to construct a SAR image data set, wherein the images in the data set are images of targets to be detected, and the targets in the SAR images are annotated using tilted boxes.

[0026] Step S2, constructing a SAR image oblique frame detection model, the SAR image oblique frame detection model adopts a neural network model, the neural network model is divided into three parts: backbone, neck, head, wherein the backbone part realizes feature extraction of the input image, the neck part realizes fusion of multi-scale features, and the head part realizes target detection; the head part includes a convolution module, a distance weight mask generation module, an angle weight mask generation module, and a loss function calculation module based on rotation matrix estimation;

[0027] The neural network model processes the image as follows:

[0028] Step S2-1, input resolution is The SAR image is obtained by extracting the backbone and neck features of the neural network and fusion of multi-scale features. The channel resolution is Feature map, where is a constant, and the feature map is sent to the head part.

[0029] The feature map is sent to the convolution module, which decodes the feature map to obtain the center point coordinate information of the target bounding box. The center point coordinate information includes the center point positions of the bounding boxes of multiple possible targets. The center point coordinate information is sent to the distance weighted mask generation module, the angle weighted mask generation module, and the loss function calculation module based on rotation matrix estimation. The feature map decoded by the convolution module is sent to the distance weighted convolution module.

[0030] Step S2-2, the distance weight mask generation module generates a distance weight mask using the center point coordinate information output by the convolution module, and outputs it to the distance weighted convolution module; wherein the method for generating the distance weight mask is as follows: Figure 2 As shown, the center point coordinate information includes The center point of the bounding box of possible targets, assuming that The center point of the bounding box of possible targets is , we can get the pixel coordinates of the feature map where the target is located as ,in , , . possible targets creates a blank single-channel image with the same resolution as the feature map , The pixel value of each pixel Satisfy the formula

[0031] ,

[0032] in, , for No. The horizontal and vertical coordinates of the pixel points, is a constant.

[0033] The feature map in step S2-1 is decoded by the convolution module Single channel image corresponding to the target The corresponding pixels are summed and the result is passed through the sigmoid function to obtain a unique single-channel image, which is the distance weight mask of the feature map.

[0034] Step S2-3, the distance weighted convolution module uses the feature map output by the convolution module in step S2-1 and the distance weighted mask output by the distance weighted mask generation module to achieve distance weighting of the feature map output by the convolution module, that is, each channel of the feature map output by the convolution module is multiplied by the pixel value of the corresponding position of the distance weighted mask one by one to obtain a distance-weighted feature map, and the distance-weighted feature map is decoded to obtain the angle information of the target bounding box, the angle information contains the angle values ​​of multiple possible targets, and is sent to the angle weighted mask generation module and the loss function calculation module based on rotation matrix estimation respectively, and the distance-weighted feature map is sent to the angle weighted convolution module;

[0035] Step S2-4, the angle weight mask generation module generates an angle weight mask using the angle information output by the distance weighted convolution module, and outputs the angle weight mask to the angle weighted convolution module; wherein the method for generating the angle weight mask is as follows: Figure 3 , as follows, the angle information of the input target frame contains possible target angle values, assuming the The bounding box angles of possible targets are ,in .

[0036] For the possible targets to create a blank single-channel image with the same resolution as the distance-weighted feature map output by the distance-weighted convolution module in step S2-3 , The pixel value of each pixel Satisfy the formula

[0037] ,

[0038] in, , , for No. The horizontal and vertical coordinates of the pixel points, , For the The horizontal and vertical coordinates of the pixel points in the feature map channel where the target is located, is a constant.

[0039] The distance-weighted feature map obtained by decoding in step S2-3 Single channel image of a target The corresponding pixels are summed and the result is passed through the sigmoid function to obtain a unique single-channel image, which is the angle weight mask of the feature map.

[0040] Step S2-5, the angle-weighted convolution module uses the distance-weighted feature map output by the distance-weighted convolution module in step S2-3 and the angle-weighted mask output by the angle-weighted mask generation module to achieve angle weighting of the distance-weighted feature map, that is, each channel of the distance-weighted feature map is multiplied by the pixel value of the corresponding position of the angle-weighted mask one by one to obtain a new feature map, and the distance-weighted and angle-weighted feature maps are decoded to obtain the length and width information of the target bounding box, which is output to the loss function calculation module based on rotation matrix estimation.

[0041] Step S2-6, the loss function calculation module based on the rotation matrix estimation uses the center point coordinate information output by the convolution module, the angle information output by the distance weighted convolution module, and the length and width information output by the angle weighted convolution module to calculate the loss value and output it. The loss function calculation method is as follows: the rotation matrix is ​​used instead of the target angle information. The original target oblique frame information contains five items ( , , , , ), , Wherein they are the horizontal coordinate and vertical coordinate of the target frame, , are the width and length of the target box respectively. is the angle information of the target frame, which is now replaced by a new element ( , , , , , ) six items, of which Flag bit, used to calculate Value, where , the original loss function is

[0042] ,

[0043] Now add the sine-cosine conversion sign loss, the new loss function is

[0044] ,

[0045] in , is the center distance, is the aspect ratio loss, is the weight factor, is the square of the diagonal length, , , , They are the minimum and maximum horizontal and vertical coordinates of the four vertices of the target frame, respectively. is the intersection-over-union ratio between the predicted box and the true box; ; is the category loss; , , , , They are the horizontal coordinate, vertical coordinate, width, length and angle of the center point of the prediction box respectively.

[0046] Step S3, using the constructed SAR image data set to train the constructed neural network model;

[0047] The SAR images in the data set are divided into multiple batches and sent to the constructed neural network to obtain the loss value. The loss value is used to complete back propagation and update the parameters. The above steps are repeated until the model meets the accuracy requirements.

[0048] Step S4, using the trained neural network to detect the oblique frame of the SAR image target, the specific method is as follows:

[0049] Step S4-1, input the SAR image to be detected into the trained neural network, and obtain the feature extraction of the backbone and neck part of the neural network and the multi-scale feature fusion. The channel resolution is Feature map, where is a constant, and the feature map is sent to the head part, and all subsequent operations are implemented by the head part. The feature map is sent to the convolution module, which decodes the feature map and obtains the center point coordinate information of the target bounding box. The center point coordinate information contains the center point positions of the bounding boxes of multiple possible targets, which are sent to the distance weighted mask generation module and the angle weighted mask generation module respectively. The feature map decoded by the convolution module is sent to the distance weighted convolution module;

[0050] Step S4-2, the distance weight mask generation module generates a distance weight mask using the center point coordinate information output by the convolution module, and outputs it to the distance weighted convolution module;

[0051] Step S4-3, the distance weighted convolution module uses the feature map output by the convolution module in step S4-1 and the distance weighted mask output by the distance weighted mask generation module to implement distance weighting of the feature map, that is, each channel of the feature map is multiplied by the pixel value of the corresponding position of the distance weighted mask one by one to obtain a distance-weighted feature map, and the distance-weighted feature map is decoded to obtain the angle information of the target bounding box, the angle information contains the angle values ​​of multiple possible targets, and is sent to the angle weighted mask generation module respectively, and the distance-weighted feature map is sent to the angle weighted convolution module;

[0052] Step S4-4, the angle weighted mask generation module generates an angle weighted mask using the angle information output by the distance weighted convolution module, and outputs the mask to the angle weighted convolution module;

[0053] Step S4-5, the angle-weighted convolution module uses the distance-weighted feature map output by the distance-weighted convolution module in step S4-3 and the angle-weighted mask output by the angle-weighted mask generation module to achieve angle weighting of the distance-weighted feature map, that is, each channel of the feature map is multiplied by the pixel value of the corresponding position of the angle-weighted mask one by one to obtain the angle-weighted feature map, and the distance-weighted and angle-weighted feature maps are decoded to obtain the length and width information of the target bounding box.

[0054] Step S4-6, according to the coordinate information of the center point of the target bounding box output by the convolution module in S4-1, the angle information of the target bounding box output by the distance weighted convolution module in S4-3, and the length and width information of the target bounding box output by the angle weighted convolution module in S4-5, the position and shape of the target are determined to achieve target detection. Figure 7 (right). Figure 7 It can be observed that the object is detected correctly and the model performs well.

Claims

1. A SAR image oblique frame detection method based on rotation matrix estimation, characterized in that: include: Construct a SAR image dataset, wherein the images in the dataset are SAR images of the target to be detected, and the targets in the SAR images are annotated using tilted boxes; Constructing a SAR image oblique frame detection model, wherein the SAR image oblique frame detection model adopts a neural network model, wherein the neural network model includes: a backbone part, a neck part, and a head part, wherein the backbone part is used to realize feature extraction of an input image, the neck part is used to realize fusion of multi-scale features, and the head part realizes detection of a target; Using the constructed SAR image dataset, the constructed neural network model is trained; The trained neural network is used to realize the oblique frame detection of SAR image targets.

2. The SAR image oblique frame detection method based on rotation matrix estimation according to claim 1 is characterized in that: The head part includes a convolution module, a distance weight mask generation module, an angle weight mask generation module and a loss function calculation module based on rotation matrix estimation; the neural network model processes the image as follows: Step S2-1, the resolution is The SAR image is extracted from the backbone part and the neck part, and multi-scale feature fusion is performed in turn to obtain The channel resolution is Feature map, where is a constant, and the feature map is sent to the convolution module; The convolution module decodes the feature map to obtain the center point coordinate information of the target bounding box. The center point coordinate information contains the center point positions of N possible target bounding boxes. Step S2-2, the distance weight mask generation module generates a distance weight mask using the center point coordinate information output by the convolution module; Step S2-3, the distance weighted convolution module uses the feature map output by the convolution module in step S2-1 and the distance weighted mask output by the distance weighted mask generation module to implement distance weighting of the feature map output by the convolution module, that is, each channel of the feature map output by the convolution module is multiplied by the pixel value of the corresponding position of the distance weighted mask one by one to obtain a distance-weighted feature map, and the distance-weighted feature map is decoded to obtain the angle information of the target bounding box; Step S2-4, the angle weight mask generation module generates an angle weight mask using the angle information output by the distance weighted convolution module; Step S2-5, the angle-weighted convolution module uses the distance-weighted feature map output by the distance-weighted convolution module in step S2-3 and the angle-weighted mask output by the angle-weighted mask generation module to achieve angle weighting of the distance-weighted feature map, that is, each channel of the distance-weighted feature map is multiplied by the pixel value of the corresponding position of the angle-weighted mask one by one to obtain a new feature map, and the distance-weighted and angle-weighted feature maps are decoded to obtain the length and width information of the target bounding box; Step S2-6, the loss function calculation module based on the rotation matrix estimation uses the center point coordinate information output by the convolution module, the angle information output by the distance weighted convolution module, and the length and width information output by the angle weighted convolution module to calculate the loss value.

3. The SAR image oblique frame detection method based on rotation matrix estimation according to claim 2 is characterized in that: Step S2-2: The distance weight mask generation module uses the center point coordinate information output by the convolution module to generate a distance weight mask in the following specific method: Assume The center point of the bounding box of possible targets is , then The pixel coordinates of the feature map where the possible target is located are ,in , , ; for the first possible targets creates a blank single-channel image with the same resolution as the feature map , The pixel value of each pixel Satisfy the formula ， in, , For a single channel image No. The horizontal and vertical coordinates of the pixel points, is a constant; Will Single channel image corresponding to the target The corresponding pixels are added together, and the result is passed through the sigmoid function to obtain a unique single-channel image, which is the distance weighted mask.

4. The SAR image oblique frame detection method based on rotation matrix estimation according to claim 2 is characterized in that: The angle weight mask generation module uses the angle information output by the distance weighted convolution module to generate the angle weight mask in the following way: Assume The bounding box angles of possible targets are ,in ; For the possible targets creates a blank single-channel image with the same resolution as the distance-weighted feature map of the distance-weighted convolution module , The pixel value of each pixel Satisfy the formula , in, , In the formula, ( , ) is a single channel image No. The horizontal and vertical coordinates of the pixel points, , For the The horizontal and vertical coordinates of the pixel points in the feature map channel where the target is located, is a constant; Will Single channel image of a target The corresponding pixels are summed and the result is passed through the sigmoid function to obtain a unique single-channel image, which is the angle weight mask.

5. The SAR image oblique frame detection method based on rotation matrix estimation according to claim 2 is characterized in that: The loss function calculation module based on rotation matrix estimation uses the center point coordinate information output by the convolution module, the angle information output by the distance weighted convolution module, and the length and width information output by the angle weighted convolution module. The specific method for calculating the loss value is: The target oblique frame information includes ( , , , , , ),in Flag bit, used to calculate Value, where , , Wherein they are the horizontal coordinate and vertical coordinate of the target frame, , are the width and length of the target box respectively. is the angle information of the target frame, and the loss function is ， in , is the center distance, is the aspect ratio loss, is the weight factor, is the square of the diagonal length, They are the minimum and maximum horizontal and vertical coordinates of the four vertices of the target frame, respectively. is the intersection-over-union ratio between the predicted box and the true box; ; is the category loss; , , , , They are the horizontal coordinate, vertical coordinate, width, length and angle of the center point of the prediction box respectively.

Images