A rotation target detection method based on diffusion model and Gaussian modeling

Abstract

This invention proposes a rotating target detection method based on diffusion model and Gaussian modeling. The method includes: Step 1, acquiring a remote sensing image dataset for rotating target detection, and performing Gaussian representation on the ground truth bounding boxes of objects in the dataset to obtain the ground truth Gaussian bounding boxes of objects; Step 2, adding standard normal noise to the ground truth Gaussian bounding boxes of objects to obtain noisy bounding boxes, and extracting multi-scale image features using the noisy bounding boxes; Step 3, inputting the extracted multi-scale image features into a head network to detect the category, position expectation, and position variance of objects in the image; Step 4, denoising the position expectation of objects to obtain the position of the object's center point, and denoising and performing singular value decomposition on the position variance of objects to obtain the length, width, and angle of the object.

Description

Technical Field

[0001] This invention relates to the field of computer vision image processing technology, and in particular to an end-to-end target detection method based on a diffusion model. Background Technology

[0002] Rotational object detection is a fundamental task in computer vision, aiming to detect the category and location of target objects in an image. The main difference between rotational and horizontal object detection is that rotational object detection uses a rotated bounding box to represent the object's location, while horizontal object detection uses a horizontal bounding box. Current rotational object detection methods can generally be divided into three main categories: single-stage methods based on convolutional neural networks, two-stage methods, and end-to-end methods. Single-stage methods based on convolutional neural networks achieve faster inference speeds due to their simple network structure; two-stage methods progressively refine the detection results from coarse to fine; and end-to-end methods eliminate a large number of manually designed modules.

[0003] In recent years, diffusion models have made progress in various fields of computer vision, especially in image generation, depth estimation, anomaly detection, and horizontal object detection. Horizontal object detection based on diffusion models refines the predicted bounding box by adding noise to the ground truth label and then gradually removing the noise.

[0004] While diffusion models have achieved some success in horizontal object detection, they are difficult to directly apply to rotating object detection tasks. In rotating object detection, objects are arbitrarily rotated and are sensitive to the rotation angle. The challenges of using diffusion models for rotating object detection are as follows:

[0005] (1) Effective representation of the object's rotating frame: Common methods use the object's center point coordinates, width, height and rotation angle to explicitly represent the object's rotating frame, but the dimensions of the rotation angle are different from those of the center point coordinates, width and height.

[0006] (2) Organic combination of diffusion model and rotating target detection: The noise addition and denoising process of diffusion model is quite different from traditional target detection methods.

[0007] (3) Alignment of object rotation box with features: The multi-scale features extracted by the model are horizontal, while the detection box is arbitrarily rotated, so feature alignment is required based on the rotation box. Summary of the Invention

[0008] Objective of this invention: The objective of this invention is to overcome the shortcomings of existing technologies and propose a rotating target detection method based on diffusion models and Gaussian modeling. This invention offers solutions to the problems of effective representation of rotating bounding boxes, organic integration of diffusion models and rotating target detection, and alignment of rotating bounding boxes with features. Compared with existing target detection methods based on diffusion models, this method unifies the dimensions of the rotating bounding box, organically integrates diffusion models with rotating target detection, aligns features, and improves detection accuracy.

[0009] Technical Solution: To address the aforementioned technical problems, this invention proposes a rotating target detection method based on a diffusion model and Gaussian modeling. This method includes the following steps:

[0010] Step 1: Obtain the remote sensing image dataset for rotating target detection, and perform Gaussian representation on the ground truth bounding boxes of the objects in the dataset to obtain the ground truth Gaussian bounding boxes of the objects.

[0011] Step 2: Add standard normal noise to the real Gaussian bounding box of the object in Step 1 to obtain a noise bounding box, and use the noise bounding box to extract multi-scale image features;

[0012] Step 3: Input the multi-scale image features extracted in Step 2 into the head network to detect the category, position expectation, and position variance of objects in the image;

[0013] Step 4: Denoise the expected position of the object in Step 3 to obtain the position of the object's center point. Denoise the variance of the object's position in Step 3 and perform singular value decomposition to obtain the object's length, width, and angle.

[0014] Furthermore, in step 1, the method for obtaining the remote sensing image dataset for rotating target detection is as follows:

[0015] (2.1) Construct a remote sensing image dataset for rotating target detection. Merge the remote sensing image datasets DIOR and DOTA, crop the dataset images to a fixed size of 800*800, and move and modify the coordinate positions of objects in the images accordingly.

[0016] (2.2) Perform image flipping preprocessing on the dataset, including horizontal flipping, vertical flipping and diagonal flipping, and delete the annotation files with empty targets;

[0017] (2.3) Divide the processed dataset into training set, validation set and test set in a ratio of 7:1:2.

[0018] Furthermore, in step 1, the process of performing Gaussian representation on the ground truth bounding boxes of objects in the dataset to obtain the ground truth Gaussian bounding boxes of the objects is as follows:

[0019] (3.1) Extract the object's true bounding box (cx1,cy1,w1,h1,θ1), where (cx1,cy1) are the coordinates of the object's center point, (w1,h1) are the object's width and height, and θ1 is the object's rotation angle.

[0020] (3.2) Gaussian representation of the object's true bounding box, the expected value of the true Gaussian bounding box is calculated by the formula μ1=(cx1,cy1);

[0021] (3.3) The formula for calculating the variance of the true Gaussian bounding box is as follows:

[0022]

[0023] In formula (1), R1 represents the two-dimensional rotation matrix, Λ1 represents the eigenvalue diagonal matrix, cos represents the cosine function, sin represents the sine function, w1 represents the width of the rotation frame, h1 represents the height of the rotation frame, θ1 represents the rotation angle of the object, and T represents the matrix transpose.

[0024] Furthermore, in step 2, the process of adding standard normal noise to the actual Gaussian bounding box of the object in step 1 to obtain the noise bounding box is as follows:

[0025] (4.1) Generate standard normal noise ∈ 1 according to the standard normal equation;

[0026] (4.2) Add standard normal noise ∈1 to the expectation μ1 of the true Gaussian box to obtain the expectation μ2 of the noise box. The calculation formula is as follows:

[0027] μ2=μ1+∈1 (2)

[0028] (4.3) Add standard normal noise ∈1 to the variance ∑1 of the true Gaussian box to obtain the variance ∑2 of the noise box. The calculation formula is as follows:

[0029] ∑2=∑1+∈1 (3).

[0030] Furthermore, in step 2, the process of extracting multi-scale image features using noisy bounding boxes is as follows:

[0031] (5.1) The image obtained from the preprocessing in step 1 is first input into the backbone network ResNet50, and then into the feature pyramid FPN to obtain four layers of initial image features o1, o2, o3, o4, with sizes of 256*200*200, 256*100*100, 256*50*50, and 256*25*25, respectively, where 256 is the number of initial image feature channels;

[0032] (5.2) Scale the expected value μ2 and variance ∑2 of the noise box to the same range as the initial image feature size, extract the multi-scale image features f using bilinear interpolation, and stitch the features together to a size of 7*7*256, where 7 represents the side length of the multi-scale image feature f and 256 represents the number of channels of the multi-scale image feature f.

[0033] Furthermore, in step 3, the multi-scale image features extracted in step 2 are input into the head network, and the process of detecting object categories in the image is as follows:

[0034] The classification head network head1 is used to detect the category of objects in an image. The input of the classification head network head1 is the multi-scale image features f, and the output is the object category cls. The calculation formula is as follows:

[0035] cls=head1(f) (4)

[0036] In Formula 4, cls represents the object category, head1 represents the classification head network, and f represents multi-scale image features.

[0037] Furthermore, in step 3, the multi-scale image features extracted in step 2 are input into the head network, and the process of detecting the position expectation and position variance of objects in the image is as follows:

[0038] (7.1) The position expectation and position variance of objects in the image are detected using the localization head network head2. The input of the localization head network head2 is the multi-scale image features f, and the output is the position expectation μ3 and position variance ∑3 of the object. The calculation formula is as follows:

[0039] μ3,∑3=head2(f) (5)

[0040] In Formula 5, μ3 represents the expected position of the object, ∑3 represents the variance of the object's position, head2 represents the localization head network, and f represents the multi-scale image features.

[0041] Furthermore, in step 4, the process of denoising the position of the object obtained in step 3 to obtain the position of the object's center point is as follows:

[0042] (8.1) Generate random noise ∈2 using a random function;

[0043] (8.2) The position of the object is denoised by the expected value μ3, and the formula for obtaining the position of the object's center point is as follows:

[0044] (cx2,cy2)=∈2-μ3 (6)

[0045] In Formula 6, cx2 represents the x-coordinate of the object's center point, and cy2 represents the y-coordinate of the object's center point.

[0046] Furthermore, in step 4, the process of denoising the position variance of the object from step 3 and performing singular value decomposition to obtain the object's length, width, and angle is as follows:

[0047] (9.1) Generate random noise ∈3 using a random function;

[0048] (9.2) Noise is removed from the position variance ∑3 of the object. The formula for calculating the position variance ∑4 after noise removal is as follows:

[0049] ∑4=∈3-∑3 (7)

[0050] (9.3) The formula for calculating the singular value decomposition of the denoised location variance ∑4 is as follows:

[0051]

[0052] In Formula 8, R2 represents the two-dimensional rotation matrix, Λ2 represents the eigenvalue diagonal matrix, cos represents the cosine function, sin represents the sine function, w2 represents the width of the object, h2 represents the height of the object, and θ2 represents the rotation angle of the object.

[0053] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0054] First, this invention addresses the different dimensions of the object's rotation angle, center point coordinates, width, and height in rotating target detection by designing a Gaussian representation and constructing a Gaussian bounding box for the object, which can effectively represent the object's rotation bounding box.

[0055] Second, this invention adds standard normal noise to the real Gaussian bounding box of the object, which can denoise the predicted position expectation and variance, and realize the organic combination of rotating target detection task and diffusion model.

[0056] Third, the present invention further extracts features on multi-scale features based on Gaussian boxes with added noise, effectively achieving feature alignment. Attached Figure Description

[0057] Figure 1 This is an overall flowchart of the present invention;

[0058] Figure 2 This is a flowchart of the network training process of the present invention;

[0059] Figure 3 This is a flowchart of the network inference process of the present invention. Detailed Implementation

[0060] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0061] This invention proposes a rotating target detection method based on a diffusion model and Gaussian modeling, which includes the following steps:

[0062] Step 1: Obtain the remote sensing image dataset for rotating target detection, and perform Gaussian representation on the ground truth bounding boxes of the objects in the dataset to obtain the ground truth Gaussian bounding boxes of the objects.

[0063] Step 2: Add standard normal noise to the real Gaussian bounding box of the object in Step 1 to obtain a noise bounding box, and use the noise bounding box to extract multi-scale image features;

[0064] Step 3: Input the multi-scale image features extracted in Step 2 into the head network to detect the category, position expectation, and position variance of objects in the image;

[0065] Step 4: Denoise the expected position of the object in Step 3 to obtain the position of the object's center point. Denoise the variance of the object's position in Step 3 and perform singular value decomposition to obtain the object's length, width, and angle.

[0066] Furthermore, in step 1, the method for obtaining the remote sensing image dataset for rotating target detection is as follows:

[0067] (2.1) Construct a remote sensing image dataset for rotating target detection. Merge the remote sensing image datasets DIOR and DOTA, crop the dataset images to a fixed size of 800*800, and move and modify the coordinate positions of objects in the images accordingly.

[0068] (2.2) Perform image flipping preprocessing on the dataset, including horizontal flipping, vertical flipping and diagonal flipping, and delete the annotation files with empty targets;

[0069] (2.3) Divide the processed dataset into training set, validation set and test set in a ratio of 7:1:2.

[0070] Furthermore, in step 1, the process of performing Gaussian representation on the ground truth bounding boxes of objects in the dataset to obtain the ground truth Gaussian bounding boxes of the objects is as follows:

[0071] (3.1) Extract the object's true bounding box (cx1,cy1,w1,h1,θ1), where (cx1,cy1) are the coordinates of the object's center point, (w1,h1) are the object's width and height, and θ1 is the object's rotation angle.

[0072] (3.2) Gaussian representation of the object's true bounding box, the expected value of the true Gaussian bounding box is calculated by the formula μ1=(cx1,cy1);

[0073] (3.3) The formula for calculating the variance of the true Gaussian bounding box is as follows:

[0074]

[0075] In formula (1), R1 represents the two-dimensional rotation matrix, Λ1 represents the eigenvalue diagonal matrix, cos represents the cosine function, sin represents the sine function, w1 represents the width of the rotation frame, h1 represents the height of the rotation frame, θ1 represents the rotation angle of the object, and T represents the matrix transpose.

[0076] Furthermore, in step 2, the process of adding standard normal noise to the actual Gaussian bounding box of the object in step 1 to obtain the noise bounding box is as follows:

[0077] (4.1) Generate standard normal noise ∈ 1 according to the standard normal equation;

[0078] (4.2) Add standard normal noise ∈1 to the expectation μ1 of the true Gaussian box to obtain the expectation μ2 of the noise box. The calculation formula is as follows:

[0079] μ2=μ1+∈1 (2)

[0080] (4.3) Add standard normal noise ∈1 to the variance ∑1 of the true Gaussian box to obtain the variance ∑2 of the noise box. The calculation formula is as follows:

[0081] ∑2=∑1+∈1 (3).

[0082] Furthermore, in step 2, the process of extracting multi-scale image features using noisy bounding boxes is as follows:

[0083] (5.1) The image obtained from the preprocessing in step 1 is first input into the backbone network ResNet50, and then into the feature pyramid FPN to obtain four layers of initial image features o1, o2, o3, o4, with sizes of 256*200*200, 256*100*100, 256*50*50, and 256*25*25, respectively, where 256 is the number of initial image feature channels;

[0084] (5.2) Scale the expected value μ2 and variance ∑2 of the noise box to the same range as the initial image feature size, extract the multi-scale image features f using bilinear interpolation, and stitch the features together to a size of 7*7*256, where 7 represents the side length of the multi-scale image feature f and 256 represents the number of channels of the multi-scale image feature f.

[0085] Furthermore, in step 3, the multi-scale image features extracted in step 2 are input into the head network, and the process of detecting object categories in the image is as follows:

[0086] The classification head network head1 is used to detect the category of objects in an image. The input of the classification head network head1 is the multi-scale image features f, and the output is the object category cls. The calculation formula is as follows:

[0087] cls=head1(f) (4)

[0088] In Formula 4, cls represents the object category, head1 represents the classification head network, and f represents multi-scale image features.

[0089] Furthermore, in step 3, the multi-scale image features extracted in step 2 are input into the head network, and the process of detecting the position expectation and position variance of objects in the image is as follows:

[0090] (7.1) The position expectation and position variance of objects in the image are detected using the localization head network head2. The input of the localization head network head2 is the multi-scale image features f, and the output is the position expectation μ3 and position variance ∑3 of the object. The calculation formula is as follows:

[0091] μ3,∑3=head2(f) (5)

[0092] In Formula 5, μ3 represents the expected position of the object, ∑3 represents the variance of the object's position, head2 represents the localization head network, and f represents the multi-scale image features.

[0093] Furthermore, in step 4, the process of denoising the position of the object obtained in step 3 to obtain the position of the object's center point is as follows:

[0094] (8.1) Generate random noise ∈2 using a random function;

[0095] (8.2) The position of the object is denoised by the expected value μ3, and the formula for obtaining the position of the object's center point is as follows:

[0096] (cx2,cy2)=∈2-μ3 (6)

[0097] In Formula 6, cx2 represents the x-coordinate of the object's center point, and cy2 represents the y-coordinate of the object's center point.

[0098] Furthermore, in step 4, the process of denoising the position variance of the object from step 3 and performing singular value decomposition to obtain the object's length, width, and angle is as follows:

[0099] (9.1) Generate random noise ∈3 using a random function;

[0100] (9.2) Noise is removed from the position variance ∑3 of the object. The formula for calculating the position variance ∑4 after noise removal is as follows:

[0101] ∑4=∈3-∑3 (7)

[0102] (9.3) The formula for calculating the singular value decomposition of the denoised location variance ∑4 is as follows:

[0103]

[0104] In Formula 8, R2 represents the two-dimensional rotation matrix, Λ2 represents the eigenvalue diagonal matrix, cos represents the cosine function, sin represents the sine function, w2 represents the width of the object, h2 represents the height of the object, and θ2 represents the rotation angle of the object.

Claims

1. A rotating target detection method based on diffusion model and Gaussian modeling, characterized in that, The method includes the following steps: Step 1: Obtain the remote sensing image dataset for rotating target detection, and perform Gaussian representation on the ground truth bounding boxes of the objects in the dataset to obtain the ground truth Gaussian bounding boxes of the objects. Step 2: Add standard normal noise to the real Gaussian bounding box of the object in Step 1 to obtain a noise bounding box, and use the noise bounding box to extract multi-scale image features; Step 3: Input the multi-scale image features extracted in Step 2 into the head network to detect the category, position expectation, and position variance of objects in the image; Step 4: Denoise the expected position of the object in Step 3 to obtain the position of the object's center point; Denoise the variance of the object's position in Step 3 and perform singular value decomposition to obtain the object's length, width, and angle. In step 1, the process of performing Gaussian representation on the ground truth bounding boxes of objects in the dataset to obtain the ground truth Gaussian bounding boxes of the objects is as follows: (3.1) Extracting the true frame of the object ,in, The coordinates of the object's center point For the width and height of the object, The rotation angle of the object; (3.2) Gaussian representation of the object's true bounding box, and the formula for calculating the expectation of the true Gaussian bounding box. ; (3.3) The formula for calculating the variance of the true Gaussian bounding box is as follows: (1) In formula (1), This represents a two-dimensional rotation matrix. This represents the eigenvalue diagonal matrix. This represents the cosine function. This represents the sine function, and T represents the matrix transpose. In step 4, the process of denoising the position variance of the object from step 3 and performing singular value decomposition to obtain the object's length, width, and angle is as follows: (9.1) Generating random noise using random functions ; (9.2) Variance of the position of the object Denoising, and the variance of the denoised location The calculation formula is as follows: (7) (9.3) The position variance after denoising The formula for singular value decomposition is as follows: (8) In formula 8, This represents a two-dimensional rotation matrix. This represents the eigenvalue diagonal matrix. This represents the cosine function. It represents the sine function. This indicates the width of the object. This indicates the height of the object. This represents the rotation angle of the object after processing by formula (8).

2. The rotating target detection method based on diffusion model and Gaussian modeling according to claim 1, characterized in that, In step 1, the method for obtaining the remote sensing image dataset for rotating target detection is as follows: (2.1) Construct a remote sensing image dataset for rotating target detection. Merge the remote sensing image datasets DIOR and DOTA, crop the dataset images to a fixed size of 800*800, and move and modify the coordinate positions of objects in the images accordingly. (2.2) Perform image flipping preprocessing on the dataset, including horizontal flipping, vertical flipping and diagonal flipping, and delete the annotation files with empty targets; (2.3) Divide the processed dataset into training set, validation set and test set in a ratio of 7:1:

2.

3. The rotating target detection method based on diffusion model and Gaussian modeling according to claim 1, characterized in that, In step 2, the process of adding standard normal noise to the real Gaussian bounding box of the object in step 1 to obtain the noise bounding box is as follows: (4.1) Generate standard normal noise based on the standard normal equation ; (4.2) Expectation to the true Gaussian box Add standard normal noise The expected value of the noise box The calculation formula is as follows: (2) (4.3) Variance to the true Gaussian box Add standard normal noise The variance of the noise box is obtained. The calculation formula is as follows: (3)。 4. The rotating target detection method based on diffusion model and Gaussian modeling according to claim 3, characterized in that, In step 2, the process of extracting multi-scale image features using noisy bounding boxes is as follows: (5.1) The image obtained from the preprocessing in step 1 is first input into the backbone network ResNet50, and then into the feature pyramid FPN to obtain 4 layers of initial image features. The sizes are 256*200*200, 256*100*100, 256*50*50, and 256*25*25, respectively, where 256 is the initial number of image feature channels; (5.2) Desiring the noise box and variance The numerical values ​​are scaled to the same range as the initial image feature size, and multi-scale image features are extracted using bilinear interpolation. The features are then concatenated, resulting in a size of 7*7*256, where 7 represents the multi-scale image features. The side length, 256, represents the multi-scale image features. The number of channels.

5. The rotating target detection method based on diffusion model and Gaussian modeling according to claim 1, characterized in that, In step 3, the multi-scale image features extracted in step 2 are input into the head network, and the process of detecting object categories in the image is as follows: A classification head network is used to detect the category of objects in an image. Classification head network The input is multi-scale image features. The output is the category of the object. The calculation formula is as follows: (4) In formula 4, Indicates the category of an object. This represents the classification head network. Represents multi-scale image features.

6. The rotating target detection method based on diffusion model and Gaussian modeling according to claim 1, characterized in that, In step 3, the multi-scale image features extracted in step 2 are input into the head network. The process of detecting the position expectation and position variance of objects in the image is as follows: (7.1) The expected position and variance of the position of objects in the image are detected using a localization head network. Positioning the head network The input is multi-scale image features. The output is the expected position of the object. and location variance The calculation formula is as follows: (5) In formula 5, Represents the expected position of an object. Represents the variance of an object's position. This indicates the location of the head network. Represents multi-scale image features.

7. The rotating target detection method based on diffusion model and Gaussian modeling according to claim 6, characterized in that, In step 4, the process of denoising the position of the object obtained in step 3 to obtain the position of the object's center point is as follows: (8.1) Generating random noise using random functions ; (8.2) Expected position of the object The formula for obtaining the center point position of an object after noise reduction is as follows: (6) In formula 6, The x-coordinate represents the position of the center point of the object. The vertical coordinate represents the position of the center point of the object.

Images