A method for generating a marine target recognition scene based on a detection parameter

Abstract

Aiming at the task of maritime target recognition, a method of maritime target recognition scene generation based on detection parameters is proposed. It contains three steps: (1) generating target mask image; (2) generating target-free scene using DeepFillv2; (3) target physical scene synthesis based on improved Poisson fusion.

Description

Technical Field

[0001] This invention proposes a method for generating maritime target recognition scenes based on detection parameters. The method aims to generate maritime target recognition scenes that include different detection parameters such as target distance, sea state, and detection azimuth, thus solving the problem of limited data and providing technical support for improving the maritime target recognition rate. This technology belongs to the field of artificial intelligence. Background Technology

[0002] Maritime target recognition utilizes sensors, data processing, and other technologies to automatically detect and identify maritime targets, finding wide application in areas such as maritime security and resource development. Traditional maritime target recognition methods primarily rely on physical scene images, but when historical data is insufficient, it's difficult to generate high-quality physical scene images under limited data conditions. Furthermore, existing conventional maritime target recognition scene generation algorithms mostly focus on scene physical elements related to the environment or interference. For example, fog and rain generation algorithms use deep learning-based methods to simulate atmospheric scattering and raindrop reflection to generate corresponding scenes; Gaussian noise and salt-and-pepper noise are used to simulate sensor interference effects. These methods neglect the influence of detection parameters on the target recognition scene. To address this issue, this invention proposes a maritime target recognition scene generation method based on detection parameters. Summary of the Invention

[0003] This invention proposes a method for generating maritime target recognition scenes based on detection parameters. The method first generates a target mask image, then generates a target-free image based on the DeepFillv2 image inpainting model, and finally uses an improved Poisson method to fuse the images to generate a maritime target recognition scene. The process is shown in Figure 1.

[0004] This invention mainly includes the following steps:

[0005] Step 1: Generate the target mask image

[0006] Step 1: Collect image data with different detection parameters, denoted as set. ,in Indicates the first Zhang Image It represents the total number of images.

[0007] Step 2: Process the dataset Each image in The calibrated target object is in the first The location and size of the target object in the image are determined, and the region containing the target object is represented by a normalized bounding box. A normalized bounding box parameter vector is defined. , , It is the first The horizontal and vertical coordinates of the center point of the target object j in the image. and Let represent the width and height of the bounding box of the i-th image. Normalize all these parameter values ​​to the interval [0,1], and let . , , , .in and These are the normalized x and y coordinates of the center point of the target object. and These are the normalized bounding box width and height.

[0008] Step 3: Convert the normalized bounding box parameters to pixel-level bounding box coordinates:

[0009]

[0010]

[0011] in, These are the actual width and height of the image, respectively; and , and Target object j in the image The pixel coordinates of the bottom left and bottom right corners of the bounding box.

[0012] Step 4: Generate the first Mask image of an object , Indicates the first The vertical pixel count of an image. Indicates the first The number of horizontal pixels in an image This represents the number of channels in the image. The first... The pixel region containing the target object is marked with a black mask with a mask value of 0, and the remaining regions are marked with a white mask with a mask value of 255. The mask construction process is represented as follows:

[0013]

[0014] in, Indicates the first The first image An object is in position The mask value at that location.

[0015] Step 2: Use DeepFillv2 to generate a targetless scene

[0016] Step 1: This invention uses the DeepFillv2 model to repair the target region, generating a target-free scene image. The model's input includes the original image and a binarized mask. The first step... The composite mask of the image is denoted as ,in This indicates a pixel-by-pixel OR operation. If at a certain location... Above, any mask If it is 1, then It is 1; otherwise, It's 0. Normalize the input image and mask:

[0017] ,

[0018] The normalized image and the mask are concatenated to form the input feature tensor:

[0019]

[0020] Step 2: Employ a multi-stage inference strategy to progressively generate targetless scene images. At each stage, the generator uses initial features... Using the output from the previous stage as input, a progressively optimized image is generated:

[0021]

[0022] in, For the first Images generated in stages, Let be the function of the generator in the k-th stage, with the initial state as follows: This process progressively optimizes the texture and lighting consistency of the background area, ultimately generating a natural and realistic backgroundless image.

[0023] Step 3: Output the image of the scene without a target. After multi-stage optimization, select the second stage output. As the first A targetless scene image generated from the original image. .

[0024] Step 3: Target physical scene synthesis based on improved Poisson fusion

[0025] Step 1: Extract the target region from the source image using a mask image. In the target region (where the mask value is 0), the pixel values ​​retain the values ​​from the source image. In non-target regions (where the mask value is 255), the pixel values ​​are set to 0. The final target region image is obtained. According to the target azimuth angle Distance to target Make adjustments to generate the target area. .

[0026] (1) Adjust the target area according to the target distance. The target distance is expressed as the actual physical distance between the detection device and the target object, in meters (m), as follows:

[0027]

[0028] in, This is the target distance (unit: meters). It is the camera's focal length, determined through estimation or calibration; It is the target's actual physical height (unit: meters). It is the pixel height of the target in the image. (Based on target distance) Adjust the size of the target area.

[0029] (2) Adjust the target area according to the target azimuth. Target azimuth It is the horizontal angle of the target relative to the detection equipment, expressed in degrees (°). It is calculated by measuring the target's relative position in the detection equipment's field of view, using the following formula:

[0030]

[0031] in and These represent the horizontal and vertical positions of the target in the coordinate system of the detection equipment, respectively. The detection azimuth angle is calculated using the image center in the top view as the detection reference point, with north as the 0° reference. The target azimuth angle is then used as the reference point. Rotate the target area to ensure that the orientation of the target object in the image is consistent with its orientation in the actual physical scene.

[0032] Step 2: Use image contrast to characterize sea conditions. The contrast calculation formula is:

[0033]

[0034] The contrast value indicates the image's contrast. The higher the contrast value, the greater the undulation of the sea surface and the more turbulent the sea conditions. The lower the contrast value, the calmer the sea surface. and These represent the gray levels of two adjacent pixels in the image, each... A combination of grayscale values ​​corresponding to a pair of pixels; This indicates that the gray level in the image is With gray level The probability of pixels appearing simultaneously. Sea states are classified into four levels based on image contrast: calm, light waves, moderate waves, and large waves. (Based on sea state levels...) Choose a background image.

[0035] Step 3: Transfer the targetless image As background image In the image A random location is generated above, and the selected target image is placed there. target area Integrate into this location and optimize the target area. The gradient of its boundary Seamlessly blends with the background while maintaining the area The internal gradient information is consistent with the source image. The optimization objective is expressed as:

[0036]

[0037] in, This represents the optimized image pixel values. It selects the portion with the larger gradient in the overlapping area between the target region and the background image. For gradient operators, Indicates the boundary of the target area.

[0038] Step 4: Perform local smoothing to eliminate edge texture in the target area, and set the gradient to zero.

[0039]

[0040] Step 5: Weight the gradient of the target region to eliminate the brightness difference between the target region and the background image. The specific formula is as follows:

[0041]

[0042] in It is a region 0.2 times the internal average gradient; .

[0043] Step 6: Discretize the Poisson equation and use the finite difference method to transform the gradient field constraints into a system of linear equations for solution. Recover the target region. The pixel values ​​are then used to seamlessly integrate the image with the background, creating a new scene. Attached Figure Description

[0044] Figure 1. Flowchart of a feature-driven method for generating maritime target recognition scenes.

[0045] Figure 2. Source image and corresponding mask image

[0046] Figure 3. Source image and corresponding background image without target.

[0047] Figure 4. Poisson fusion result of image based on target azimuth angle.

[0048] Figure 5. Image Poisson fusion results based on target distance.

[0049] Figure 6. Poisson fusion results of images under different sea conditions Detailed Implementation

[0050] Step 1: Generate the target mask image

[0051] Step 1: Collect image data with different detection parameters, denoted as set. ,in Indicates the first Zhang Image It represents the total number of images.

[0052] Step 2: Process the dataset Each image in The calibrated target object is in the first The location and size of the target object in the image are determined, and the region containing the target object is represented by a normalized bounding box. A normalized bounding box parameter vector is defined. , , It is the first The horizontal and vertical coordinates of the center point of the target object j in the image. and Let represent the width and height of the bounding box of the i-th image. Normalize all these parameter values ​​to the interval [0,1], and let . , , , .in and These are the normalized x and y coordinates of the center point of the target object. and These are the normalized bounding box width and height.

[0053] Step 3: Convert the normalized bounding box parameters to pixel-level bounding box coordinates:

[0054]

[0055]

[0056] in, These are the actual width and height of the image, respectively; and , and Target object j in the image The pixel coordinates of the bottom left and bottom right corners of the bounding box.

[0057] Step 4: Generate the first Mask image of an object , Indicates the first The vertical pixel count of an image. Indicates the first The number of horizontal pixels in an image This represents the number of channels in the image. The first... The pixel region containing the target object is marked with a black mask with a mask value of 0, and the remaining regions are marked with a white mask with a mask value of 255. The mask construction process is represented as follows:

[0058]

[0059] Among them, Indicates the first The first image An object is in position The mask value at that location.

[0060]

Example

[0061] Step 2: Use DeepFillv2 to generate a targetless scene

[0062] Step 1: This invention uses the DeepFillv2 model to repair the target region, generating a target-free scene image. The model's input includes the original image and a binarized mask. The first step... The composite mask of the image is denoted as , among which This indicates a pixel-by-pixel OR operation. If at a certain location... Above, any mask If it is 1, then It is 1; otherwise, It's 0. Normalize the input image and mask:

[0063] ,

[0064] The normalized image and the mask are concatenated to form the input feature tensor:

[0065]

[0066] Step 2: Employ a multi-stage inference strategy to progressively generate targetless scene images. At each stage, the generator uses initial features... Using the output from the previous stage as input, a progressively optimized image is generated:

[0067]

[0068] in, For the first Images generated in stages, Let be the function of the generator in the k-th stage, with the initial state as follows: This process progressively optimizes the texture and lighting consistency of the background area, ultimately generating a natural and realistic backgroundless image.

[0069] Step 3: Output the image of the scene without a target. After multi-stage optimization, select the second stage output. As the first A targetless scene image generated from the original image. .

[0070]

Example

[0071] Step 3: Target physical scene synthesis based on improved Poisson fusion

[0072] Step 1: Extract the target region from the source image using a mask image. In the target region (where the mask value is 0), the pixel values ​​retain the values ​​from the source image. In non-target regions (where the mask value is 255), the pixel values ​​are set to 0. The final target region image is obtained. According to the target azimuth angle Distance to target Make adjustments to generate the target area. .

[0073] (1) Adjust the target area according to the target distance. The target distance is expressed as the actual physical distance between the detection device and the target object, in meters (m), as follows:

[0074]

[0075] in, This is the target distance (unit: meters). It is the camera's focal length, determined through estimation or calibration; It is the target's actual physical height (unit: meters). It is the pixel height of the target in the image. (Based on target distance) Adjust the size of the target area.

[0076] (2) Adjust the target area according to the target azimuth. Target azimuth It is the horizontal angle of the target relative to the detection equipment, expressed in degrees (°). It is calculated by measuring the target's relative position in the detection equipment's field of view, using the following formula:

[0077]

[0078] in and These represent the horizontal and vertical positions of the target in the coordinate system of the detection equipment, respectively. The detection azimuth angle is calculated using the image center in the top view as the detection reference point, with north as the 0° reference. The target azimuth angle is then used as the reference point. Rotate the target area to ensure that the orientation of the target object in the image is consistent with its orientation in the actual physical scene.

[0079] Step 2: Use image contrast to characterize sea conditions. The contrast calculation formula is:

[0080]

[0081] The contrast value indicates the image's contrast. The higher the contrast value, the greater the undulation of the sea surface and the more turbulent the sea conditions. The lower the contrast value, the calmer the sea surface. and These represent the gray levels of two adjacent pixels in the image, each... A combination of grayscale values ​​corresponding to a pair of pixels; This indicates that the gray level in the image is With gray level The probability of pixels appearing simultaneously. Sea states are classified into four levels based on image contrast: calm, light waves, moderate waves, and large waves. (Based on sea state levels...) Choose a background image.

[0082] Step 3: Transfer the targetless image As background image In the image A random location is generated above, and the selected target image is placed there. target area Integrate into this location and optimize the target area. The gradient of its boundary Seamlessly blends with the background while maintaining the area The internal gradient information is consistent with the source image. The optimization objective is expressed as:

[0083]

[0084] in, This represents the optimized image pixel values. It selects the portion with the larger gradient in the overlapping area between the target region and the background image. For gradient operators, Indicates the boundary of the target area.

[0085] Step 4: Perform local smoothing to eliminate edge texture in the target area, and set the gradient to zero.

[0086]

[0087] Step 5: Weight the gradient of the target region to eliminate the brightness difference between the target region and the background image. The specific formula is as follows:

[0088]

[0089] in It is a region 0.2 times the internal average gradient; .

[0090] Step 6: Discretize the Poisson equation and use the finite difference method to transform the gradient field constraints into a system of linear equations for solution. Recover the target region. The pixel values ​​are then used to seamlessly integrate the image with the background, creating a new scene.

[0091] [Example] Scene generation based on target azimuth parameters is performed using an improved Poisson fusion method. Figure 4 shows an example of the generated results, where (a) is the original image and (b) are images generated by the improved Poisson fusion method at different target azimuth angles. Scene generation based on target distance parameters is also performed using an improved Poisson fusion method. Figure 5 shows an example of the generated results, where (a) is the original image and (b) are images generated by the improved Poisson fusion method at different target distances. Different background images are selected based on sea conditions for Poisson fusion to generate maritime target recognition scene images for different sea conditions. An example of the generated results is shown in Figure 6.

Claims

1. A method for generating a maritime target recognition scene based on detection parameters, characterized in that... It includes the following steps: Step 1: Generate the target mask image Step 1.1: Collect image data with different detection parameters, denoted as set. ,in Indicates the first Zhang Image , It represents the total number of images; Step 1.2: Process the dataset Each image in The calibrated target object is in the first The location and size of the target object in the image are determined, and the region containing the target object is represented by a normalized bounding box; a normalized bounding box parameter vector is defined. , , It is the first The x and y coordinates of the center point of the target object j in the image; and Let represent the width and height of the bounding box of the i-th image; normalize all the above parameter values ​​to the interval [0,1], and let , , , ;in and These are the normalized x and y coordinates of the center point of the target object; and These are the normalized bounding box width and height; Step 1.3: Convert the normalized bounding box parameters to pixel-level bounding box coordinates: in, These are the actual width and height of the image, respectively; and , and Target object j in the image The pixel coordinates of the bottom left and bottom right corners of the bounding box; Step 1.4: Generate the first Mask image of an object , Indicates the first The vertical pixel count of an image. Indicates the first The number of horizontal pixels in an image This represents the number of channels in the image; the first... The pixel region containing the target object is marked with a black mask with a mask value of 0, and the remaining regions are marked with a white mask with a mask value of 255; the mask construction process is expressed as follows: in, Indicates the first The first image An object is in position The mask value at that location; Step 2: Use DeepFillv2 to generate a targetless scene Step 2.1: Inpaint the target region using the DeepFillv2 model to generate a target-free scene image; the model's input includes the original image and a binarized mask; the first step... The composite mask of the image is denoted as ,in This indicates a pixel-by-pixel "OR" operation; if at a certain position... Above, any mask If it is 1, then =1; otherwise, It is 0; normalize the input image and mask: ， The normalized image and the mask are concatenated to form the input feature tensor: Step 2.2: Employ a multi-stage inference strategy to progressively generate targetless scene images. In each stage, the generator uses initial features... Using the output from the previous stage as input, a progressively optimized image is generated: in, For the first Images generated in stages, Let be the function of the generator in the k-th stage, with the initial state as follows: This process gradually optimizes the texture and lighting consistency of the background area, ultimately generating a natural and realistic backgroundless image. Step 2.3: Output the image of the scene without a target; after multi-stage optimization, select the second stage output. As the first A targetless scene image generated from the original image. ; Step 3: Target physical scene synthesis based on improved Poisson fusion Step 3.1: Extract the target region from the source image using the mask image. In the target region, i.e., where the mask value is 0, the pixels retain the values ​​from the source image; in non-target regions, i.e., where the mask value is 255, the pixel values ​​are set to 0; thus, the target region image is obtained. According to the target azimuth Distance to target Make adjustments to generate the target area. ; (1) Adjust the target area according to the target distance; the target distance is expressed as the actual physical distance between the detection device and the target object, in meters, and the formula is as follows: in, It is the target distance. It is the camera's focal length, determined through estimation or calibration; It is the actual physical height of the target. It is the pixel height of the target in the image; by target distance Adjust the size of the target area; (2) Adjust the target area according to the target azimuth; target azimuth It is the horizontal angle of the target relative to the detection equipment, expressed in degrees. It is calculated by measuring the target's relative position in the detection equipment's field of view, using the following formula: in and These represent the horizontal and vertical positions of the target in the coordinate system of the detection equipment, respectively; the detection azimuth angle is calculated using the image center in the top view as the detection reference point, with north as the 0° reference; the target azimuth angle is used as the reference point. Rotate the target area to ensure that the orientation of the target object in the image is consistent with its orientation in the actual physical scene; Step 3.2: Use image contrast to characterize sea conditions. The contrast calculation formula is: The contrast value indicates the image's contrast. The higher the contrast value, the greater the undulation of the sea surface and the more turbulent the sea conditions. The lower the contrast value, the calmer the sea surface. and These represent the gray levels of two adjacent pixels in the image, each... A combination of grayscale values ​​corresponding to a pair of pixels; This indicates that the gray level in the image is With gray level The probability of pixels appearing simultaneously; classifying sea conditions into four levels based on image contrast: calm, light waves, moderate waves, and large waves; according to sea condition levels... Select a background image; Step 3.3: Transfer the targetless image As background image ; in the image A random location is generated above, and the selected target image is placed there. target area Integrate into this location and optimize the target area. The gradient of its boundary Seamlessly blends with the background while maintaining the area The internal gradient information is consistent with the source image; the optimization objective is represented as: in, This represents the optimized image pixel values. It selects the portion with the larger gradient in the overlapping area between the target region and the background image; For gradient operators, Indicates the boundary of the target area; Step 3.4: Perform local smoothing to eliminate edge texture in the target area, and set the gradient to zero. Step 3.5: Weight the gradient of the target region to eliminate the brightness difference between the target region and the background image. The specific formula is as follows: in It is a region 0.2 times the internal average gradient; ; Step 3.6: Discretize the Poisson equation and use the finite difference method to transform the gradient field constraint into a system of linear equations for solution; recover the target region. The pixel values ​​are then used to seamlessly integrate the image with the background, creating a new scene.

Images