Sea target identification scene generation method based on detection parameters

Abstract

The invention provides a sea target identification scene generation method based on detection parameters for a sea target identification task. The method comprises the following three steps: (1) generating a target mask image; (2) generating a target-free scene by using DeepFillv2 (DeepFillv2); and (3) target physical scene synthesis based on improved Poisson fusion.

Description

Technical field

[0001] This paper proposes a method for generating maritime target recognition scenarios based on detection parameters. This method aims to generate maritime target recognition scenarios that incorporate different detection parameters, such as target distance, sea conditions, and detection azimuth. This method can address the issue of limited data and provide technical support for improving maritime target recognition rates. This technology belongs to the field of artificial intelligence. Background Art

[0002] Maritime target recognition is the automatic detection and identification of maritime targets using sensors, data processing, and other technologies. It has a wide range of applications in areas such as marine safety and resource development. Traditional maritime recognition methods mainly rely on physical scene images, but when historical data is insufficient, it is difficult to generate high-quality physical scene images under limited data conditions. In addition, existing traditional maritime target recognition scene generation algorithms mostly focus on scene physical elements related to the environment or interference. For example, haze and rainy day generation algorithms use deep learning-based methods to simulate atmospheric scattering and raindrop reflection to generate corresponding scenes; noise generation algorithms such as Gaussian noise and salt and pepper noise are used to simulate sensor interference effects. These methods ignore the impact of detection parameters on target recognition scenes. To solve this problem, the present invention proposes a maritime target recognition scene generation method based on detection parameters. Summary of the Invention

[0003] This paper proposes a method for generating a marine target recognition scene based on detection parameters. The method first generates a target mask image, then generates a target-free image based on the image restoration model DeepFillv2, and finally uses the improved Poisson method to fuse and generate a marine target recognition scene. The process is as follows: Figure 1 shown.

[0004] The present invention mainly comprises the following steps:

[0005] Step 1: Generate target mask image

[0006] Step 1: Collect image data with different detection parameters, denoted as set D = {I1, I2, ..., I i ,…,I N}, where I i (i=1, 2...N) represents the i-th image, and N is the total number of images.

[0007] Step 2: For each image I in the dataset D i , calibrate the position and size of the target object in the i-th image, and represent the area where the target object is located by a normalized bounding box. Define the normalized bounding box parameter vector y center,i,j w is the horizontal and vertical coordinates of the center point of target object j in the i-th image.i and h i Represents the width and height of the bounding box of the i-th image. Normalize the above parameter values ​​to the interval [0,1], and let in and y′ center,i,j are the normalized horizontal and vertical coordinates of the center point of the target object. i and h′ i are the normalized bounding box width and height.

[0008] Step 3: Convert the normalized bounding box parameters to pixel-level bounding box coordinates:

[0009]

[0010] Among them, W i ,H i The actual width and height of the image respectively; x min,ij and y min,ij 、x max,ij and y max,ij The target object j in image I i The pixel coordinates of the lower left and lower right corners of the bounding box in .

[0011] Step 4: Generate a mask image of the jth object H i Indicates the number of vertical pixels of the i-th image, W i Represents the number of horizontal pixels of the i-th image, and C represents the number of channels of the image. The pixel area where the j-th target object is located is marked with a black mask with a mask value of 0, and the rest of the area is marked with a white mask with a mask value of 255. The mask construction process is expressed as:

[0012]

[0013] Among them, M i,j (p,q) represents the mask value of the jth object at position (p,q) in the i-th image.

[0014] Step 2: Generate an object-free scene using DeepFillv2

[0015] Step 1: The present invention uses the DeepFillv2 model to repair the target area to generate a target-free scene image. The input of the model includes the original image and the binary mask. The comprehensive mask of the i-th image is denoted as Where ∨ represents the pixel-by-pixel "or" operation. If at a certain position (p,q), any mask M i,j (p,q) is 1, then M i (p,q) is 1; otherwise, M i(p,q) is 0. Normalize the input image and mask:

[0016]

[0017] Concatenate the normalized image and mask into the input feature tensor:

[0018]

[0019] Step 2: Use a multi-stage reasoning strategy to gradually generate target-free scene images. At each stage, the generator takes the initial feature X i And the output of the previous stage as input, generating a progressively optimized image:

[0020]

[0021] in, is the image generated at stage k, is the function of the kth stage of the generator, and the initial state is Through this process, the texture and lighting consistency of the background area are gradually optimized, and finally a natural and realistic target-free background image is generated.

[0022] Step 3: Output the target-free scene image. After multi-stage optimization, select the second stage output O stage2,i The object-free scene image I generated as the i-th original image OFSI,i .

[0023] Step 3: Target physical scene synthesis based on improved Poisson fusion

[0024] Step 1: Extract the target area from the source image through the mask image. In the target area (the location where the mask value is 0), the pixel value of the source image is retained. In the non-target area (the location where the mask value is 255), the pixel value is set to 0. Finally, the target area image Ω is obtained. orig The target area Ω is generated by adjusting the target azimuth φ and the target distance D.

[0025] (1) Adjust the target area according to the target distance. The target distance is expressed in meters (m) as the actual physical distance between the detection device and the target object. The formula is as follows:

[0026]

[0027] Where D is the target distance (unit: meter), f is the focal length of the camera, which can be determined by estimation or calibration; H real is the actual physical height of the target (unit: meter), H image is the pixel height of the target in the image. The size of the target area is adjusted by the target distance D.

[0028] (2) Adjust the target area according to the target azimuth. The target azimuth φ is the horizontal angle of the target relative to the detection device, expressed in degrees (°). It is calculated by measuring the relative position of the target in the field of view of the detection device. The formula is:

[0029]

[0030] Where x and y represent the horizontal and vertical positions of the target in the detection device's coordinate system, respectively. The detection azimuth is calculated using the center of the image in the top view as the detection reference point, with 0° pointing north as the reference. The target area is rotated by the target azimuth angle φ to ensure that the target object's orientation in the image is consistent with its physical orientation.

[0031] Step 2: Use image contrast to characterize the sea conditions. The contrast calculation formula is:

[0032]

[0033] Energy represents the contrast of the image. The larger the contrast value, the greater the fluctuations in the sea surface and the more turbulent the sea conditions. The smaller the contrast value, the calmer the sea surface. a and b represent the grayscale of two adjacent pixels in the image, and each (a, b) corresponds to a grayscale value combination of a pair of pixels. P(a, b) represents the probability of pixels with grayscale a and grayscale b appearing at the same time in the image. The sea conditions are divided into four levels based on the image contrast, including calm, light waves, medium waves, and large waves. According to the sea condition level C sea Select a background image.

[0034] Step 3: The target-free image I OFSI,i As background image I t,i In image I t,i A random position is generated on the image and the selected target image I is placed s The target area Ω is integrated into the position, and the gradient of the target area Ω is optimized so that its boundary It naturally connects with the background while keeping the gradient information inside the region Ω consistent with the source image. The optimization objective is expressed as:

[0035]

[0036] Among them, f represents the optimized image pixel value, It is to select the part with larger gradient in the overlapping part of the target area and the background image. is the gradient operator, Indicates the boundaries of the target area.

[0037] Step 4: Perform local smoothing to eliminate edge textures in the target area and set the gradient to zero:

[0038]

[0039] Step 5: Perform weighted adjustment on the gradient of the target area to eliminate the brightness difference between the target area and the background image. The specific formula is:

[0040]

[0041] Where α is 0.2 times the average gradient in the region Ω; β = 0.2.

[0042] Step 6: Discretize the Poisson equation and use the finite difference method to transform the gradient field constraints into a system of linear equations for solution. Restore the pixel values ​​of the target region Ω and make it naturally connect with the background image to form a new scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A feature-driven maritime target recognition scene generation method process

[0044] Figure 2 Source image and corresponding mask image

[0045] Figure 3 Source image and corresponding target-free background image

[0046] Figure 4 Image Poisson fusion results based on target azimuth

[0047] Figure 5 Image Poisson fusion results based on target distance

[0048] Figure 6 Poisson fusion results of images in different sea conditions DETAILED DESCRIPTION

[0049] Step 1: Generate target mask image

[0050] Step 1: Collect image data with different detection parameters, denoted as set D = {I1, I2, ..., I i ,…,I N}, where I i (i=1, 2...N) represents the i-th image, and N is the total number of images.

[0051] Step 2: For each image I in the dataset D i , calibrate the position and size of the target object in the i-th image, and represent the area where the target object is located by a normalized bounding box. Define the normalized bounding box parameter vector y center,i,j w is the horizontal and vertical coordinates of the center point of target object j in the i-th image. iand h i Represents the width and height of the bounding box of the i-th image. Normalize the above parameter values ​​to the interval [0,1], and let in and y′ center,i,j are the normalized horizontal and vertical coordinates of the center point of the target object. i and h′ i are the normalized bounding box width and height.

[0052] Step 3: Convert the normalized bounding box parameters to pixel-level bounding box coordinates:

[0053]

[0054] Among them, W i ,H i The actual width and height of the image respectively; x min,ij and y min,ij 、x max,ij and y max,ij The target object j in image I i The pixel coordinates of the lower left and lower right corners of the bounding box in .

[0055] Step 4: Generate a mask image of the jth object H i Indicates the number of vertical pixels of the i-th image, W i Represents the number of horizontal pixels of the i-th image, and C represents the number of channels of the image. The pixel area where the j-th target object is located is marked with a black mask with a mask value of 0, and the rest of the area is marked with a white mask with a mask value of 255. The mask construction process is expressed as:

[0056]

[0057] Among them, M i,j (p,q) represents the mask value of the jth object at position (p,q) in the i-th image.

[0058] [Example] Generate a mask image based on a sample dataset of marine ship images. Figure 2 are example images in the dataset and their corresponding mask images. (a) is the original image, and (b) is the target mask image corresponding to the original image.

[0059] Step 2: Generate an object-free scene using DeepFillv2

[0060] Step 1: The present invention uses the DeepFillv2 model to repair the target area to generate a target-free scene image. The input of the model includes the original image and the binary mask. The comprehensive mask of the i-th image is denoted as Where ∨ represents the pixel-by-pixel "or" operation. If at a certain position (p,q), any mask M i,j (p,q) is 1, then M i (p,q) is 1; otherwise, M i (p,q) is 0. Normalize the input image and mask:

[0061]

[0062] Concatenate the normalized image and mask into the input feature tensor:

[0063]

[0064] Step 2: Use a multi-stage reasoning strategy to gradually generate target-free scene images. At each stage, the generator takes the initial feature X i And the output of the previous stage as input, generating a progressively optimized image:

[0065]

[0066] in, is the image generated at stage k, is the function of the kth stage of the generator, and the initial state is Through this process, the texture and lighting consistency of the background area are gradually optimized, and finally a natural and realistic target-free background image is generated.

[0067] Step 3: Output the target-free scene image. After multi-stage optimization, select the second stage output The object-free scene image generated as the i-th original image

[0068] [Example] Use the DeepFillv2 model to generate a background-free image and generate the result Figure 3 , are example images in the dataset and their corresponding background images without targets. (a) is the original image, and (b) is the background image without targets corresponding to the original image.

[0069] Step 3: Target physical scene synthesis based on improved Poisson fusion

[0070] Step 1: Extract the target area from the source image through the mask image. In the target area (the location where the mask value is 0), the pixel value of the source image is retained. In the non-target area (the location where the mask value is 255), the pixel value is set to 0. Finally, the target area image Ω is obtained. orig The target area Ω is generated by adjusting the target azimuth φ and the target distance D.

[0071] (1) Adjust the target area according to the target distance. The target distance is expressed in meters (m) as the actual physical distance between the detection device and the target object. The formula is as follows:

[0072]

[0073] Where D is the target distance (unit: meter), f is the focal length of the camera, which can be determined by estimation or calibration; H real is the actual physical height of the target (unit: meter), H image is the pixel height of the target in the image. The size of the target area is adjusted by the target distance D.

[0074] (2) Adjust the target area according to the target azimuth. The target azimuth φ is the horizontal angle of the target relative to the detection device, expressed in degrees (°). It is calculated by measuring the relative position of the target in the field of view of the detection device. The formula is:

[0075]

[0076] Where x and y represent the horizontal and vertical positions of the target in the detection device's coordinate system, respectively. The detection azimuth is calculated using the center of the image in the top view as the detection reference point, with 0° pointing north as the reference. The target area is rotated by the target azimuth angle φ to ensure that the target object's orientation in the image is consistent with its physical orientation.

[0077] Step 2: Use image contrast to characterize the sea conditions. The contrast calculation formula is:

[0078]

[0079] Energy represents the contrast of the image. The larger the contrast value, the greater the fluctuations in the sea surface and the more turbulent the sea conditions. The smaller the contrast value, the calmer the sea surface. a and b represent the grayscale of two adjacent pixels in the image, and each (A, B) corresponds to a grayscale value combination of a pair of pixels. P(a, b) represents the probability of pixels with grayscale a and grayscale b appearing at the same time in the image. The sea conditions are divided into four levels based on the image contrast, including calm, light waves, medium waves, and large waves. According to the sea condition level C sea Select a background image.

[0080] Step 3: Untargeted image As background image I t,i In image I t,i A random position is generated on the image and the selected target image I is placed s The target area Ω is integrated into the position, and the gradient of the target area Ω is optimized so that its boundary It naturally connects with the background while keeping the gradient information inside the region Ω consistent with the source image. The optimization objective is expressed as:

[0081]

[0082] Among them, f represents the optimized image pixel value, It is to select the part with larger gradient in the overlapping part of the target area and the background image. is the gradient operator, Indicates the boundaries of the target area.

[0083] Step 4: Perform local smoothing to eliminate edge textures in the target area and set the gradient to zero:

[0084]

[0085] Step 5: Perform weighted adjustment on the gradient of the target area to eliminate the brightness difference between the target area and the background image. The specific formula is:

[0086]

[0087] Where α is 0.2 times the average gradient in the region Ω; β = 0.2.

[0088] Step 6: Discretize the Poisson equation and use the finite difference method to transform the gradient field constraints into a system of linear equations for solution. Restore the pixel values ​​of the target region Ω and make it naturally connect with the background image to form a new scene.

[0089] [Example] Use the improved Poisson fusion method to generate scenes based on target azimuth parameters. Figure 4 The following is an example of the generated results, where (a) is the original image and (b) is the image with different target azimuth angles generated by the improved Poisson fusion method. The improved Poisson fusion method is used to generate scenes based on target distance parameters. Figure 5 The following is an example of the generated results, where (a) is the original image and (b) is an image of different target distances generated by the improved Poisson fusion method. Different background images are selected according to the sea conditions for Poisson fusion to generate sea target recognition scene images of different sea conditions. The generated results are as follows: Figure 6 .

Claims

1. A method for generating a marine target recognition scene based on detection parameters, characterized in that It consists of the following steps: Step 1: Generate target mask image Step 1: Collect image data with different detection parameters, denoted as set D = {I1, I2, ..., I i ,…,I N }, where I i (i=1,2...N) represents the i-th image, N is the total number of images; Step 2: For each image I in the dataset D i , calibrate the position and size of the target object in the i-th image, and represent the area where the target object is located by a normalized bounding box; Define the normalized bounding box parameter vector y center,i,j are the horizontal and vertical coordinates of the center point of target object j in the i-th image; w i and h i Represents the width and height of the bounding box of the i-th image; normalize the above parameter values ​​to the [0,1] interval, and let in and y′ center,i,j are the normalized horizontal and vertical coordinates of the center point of the target object; w′ i and h′ i are the normalized bounding box width and height; Step 3: Convert the normalized bounding box parameters to pixel-level bounding box coordinates: Among them, W i ,H i The actual width and height of the image respectively; x min,ij and y min,ij 、x max,ij and y max,ij The target object j in image I i The pixel coordinates of the lower left and lower right corners of the bounding box in ; Step 4: Generate a mask image of the jth object H i Indicates the number of vertical pixels of the i-th image, W i Represents the number of horizontal pixels of the i-th image, and C represents the number of channels of the image; the pixel area where the j-th target object is located is marked with a black mask with a mask value of 0, and the remaining areas are marked with a white mask with a mask value of 255; the mask construction process is expressed as: Among them, M i,j (p,q) represents the mask value of the jth object at position (p,q) in the i-th image; Step 2: Generate an object-free scene using DeepFillv2 Step 1: The present invention repairs the target area through the DeepFillv2 model to generate a target-free scene image; the input of the model includes the original image and the binary mask; the comprehensive mask of the i-th image is recorded as Where ∨ represents a pixel-by-pixel "or" operation; if at a certain position (p, q), any mask M i,j (p,q) is 1, then M i (p,q) is 1; otherwise, M i (p,q) is 0; normalize the input image and mask: Concatenate the normalized image and mask into the input feature tensor: Step 2: A multi-stage reasoning strategy is used to gradually generate target-free scene images. At each stage, the generator is initialized with the initial feature X i And the output of the previous stage as input, generating a progressively optimized image: in, is the image generated at stage k, is the function of the kth stage of the generator, and the initial state is Through this process, the texture and lighting consistency of the background area are gradually optimized, and finally a natural and realistic target-free background image is generated; Step 3: Output the target-free scene image; after multi-stage optimization, select the second stage output The object-free scene image generated as the i-th original image Step 3: Target physical scene synthesis based on improved Poisson fusion Step 1: Extract the target area from the source image through the mask image. In the target area (the position where the mask value is 0), the pixel value of the source image is retained; in the non-target area (the position where the mask value is 255), the pixel value is set to 0; finally, the target area image Ω is obtained. orig ; Adjust according to the target azimuth φ and target distance D to generate the target area Ω; (1) Adjust the target area according to the target distance. The target distance is expressed in meters (m) using the actual physical distance between the detection device and the target object. The formula is as follows: Where D is the target distance (unit: meter), f is the focal length of the camera, which can be determined by estimation or calibration; H real is the actual physical height of the target (unit: meter), H image is the pixel height of the target in the image; the size of the target area is adjusted by the target distance D; (2) Adjust the target area according to the target azimuth angle. The target azimuth angle φ is the horizontal angle of the target relative to the detection device, expressed in degrees (°). It is calculated by measuring the relative position of the target in the field of view of the detection device. The formula is: Where x and y represent the horizontal and vertical positions of the target in the detection device coordinate system, respectively. The detection azimuth is calculated using the center of the image in the top view as the detection reference point, with 0° pointing north. The target area is rotated by the target azimuth angle φ to ensure that the orientation of the target object in the image is consistent with the actual physical scene. Step 2: Use image contrast to characterize the sea conditions. The contrast calculation formula is: Energy represents the contrast of the image. The larger the contrast value, the greater the fluctuation of the sea surface and the more turbulent the sea conditions. The smaller the contrast value, the calmer the sea surface. a and b represent the grayscale of two adjacent pixels in the image, and each (a, b) corresponds to a grayscale value combination of a pair of pixels. P(a, b) represents the probability of the simultaneous appearance of pixels with grayscale a and grayscale b in the image. The sea conditions are divided into four levels according to the image contrast, including calm, light waves, medium waves, and large waves. According to the sea condition level C sea Select a background image; Step 3: Untargeted image As background image I t,i ; In image I t,i A random position is generated on the image and the selected target image I is placed s The target area Ω is integrated into the position, and the gradient of the target area Ω is optimized so that its boundary It naturally connects with the background while keeping the gradient information inside the region Ω consistent with the source image; the optimization objective is expressed as: Among them, f represents the optimized image pixel value, In the overlapping part of the target area and the background image, the part with larger gradient is selected; is the gradient operator, Indicates the boundary of the target area; Step 4: Perform local smoothing to eliminate edge textures in the target area and set the gradient to zero: Step 5: Perform weighted adjustment on the gradient of the target area to eliminate the brightness difference between the target area and the background image. The specific formula is: Where α is 0.2 times the average gradient in the region Ω; β = 0.2; Step 6: Discretize the Poisson equation and use the finite difference method to transform the constraints of the gradient field into a system of linear equations for solution; restore the pixel values ​​of the target area Ω and make it naturally connect with the background image to form a new scene.

Images