A fast simulation method of high-resolution sar image building segmentation dataset

Abstract

The application discloses a kind of high-resolution SAR image building segmentation data set fast simulation method.This application is based on ray tracing principle, using building three-dimensional model realizes high-resolution SAR image simulation, and proposes the automatic labeling method of SAR image building facade, shadow area, realizes the fast simulation of high-resolution SAR image data set.This method can simulate high-resolution SAR image according to building three-dimensional model, and automatically label building facade, shadow area in high-resolution SAR image, provide data set production method for the semantic segmentation neural network applied to SAR image, provide strong support for the development of SAR image field deep learning.

Description

Technical Field

[0001] This invention belongs to the field of synthetic aperture radar technology, and particularly relates to a fast simulation method for high-resolution SAR image building segmentation datasets. Background Technology

[0002] Synthetic Aperture Radar Tomography (TomoSAR) technology acquires 2D SAR images of the same area under different baselines through multiple reorbit observations, forming a synthetic aperture in the height direction to achieve high-resolution 3D imaging in the range-azimuth-height directions. TomoSAR technology can perform all-weather, all-day 3D imaging over large areas, solving the overlay problem in 2D SAR images and achieving great success in 3D reconstruction tasks in complex areas such as cities. With the improvement of spaceborne synthetic aperture radar (SAR) imaging resolution, building scattering characteristics in SAR images are becoming more apparent. Research on extracting building information from 2D SAR images based on SAR imaging mechanisms and target scattering characteristics to assist in SAR 3D imaging has developed rapidly. Traditional building information extraction methods are based on shallow features of SAR images, resulting in poor performance and robustness. Convolutional neural networks can extract deep semantic features from images, achieving great success in the field of semantic segmentation of natural images. The application of semantic segmentation networks to SAR images has become a focus of research. However, due to the significant differences between SAR imaging mechanisms and optical images, SAR image acquisition and annotation are difficult, leading to slow development of semantic segmentation networks based on SAR images.

[0003] Currently available SAR image datasets are mostly for ship target detection, aircraft target detection, and large-scale terrain and building region segmentation; datasets for building semantic segmentation of high-resolution SAR images are relatively scarce. Methods for creating datasets can be divided into two types: simulated SAR images and real SAR images. Manual annotation of real SAR images is time-consuming and labor-intensive, and the quality of annotation results varies greatly depending on the annotator's skill. Simulated SAR image data can be automatically annotated by computers, which is less time-consuming; however, the building scattering characteristics of simulated SAR images differ from those of real SAR images, which may affect the performance of semantic segmentation results. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a method for rapidly simulating a high-resolution SAR image semantic segmentation dataset. This method generates building point clouds from a radar perspective based on a 3D building model, then generates SAR images according to SAR imaging principles, extracts building facades by projecting the point clouds from the radar perspective onto a horizontal plane, and finally annotates the building facades and shadows in the SAR images.

[0005] The fast simulation method for high-resolution SAR image semantic segmentation datasets of the present invention includes:

[0006] Step 1: Input parameters and use POV-Ray software to obtain single-view point clouds and the scattering intensity and scattering count of scattering points. First, input the building model data and place the model at the center of the scene. Then, input the light source position, ground parameters, camera position, and the roughness of the building and the ground. Import the building model into POV-Ray software, render it, and obtain the point cloud from the radar viewpoint and the scattering intensity and scattering count of each point. Eight single-view point clouds are generated for each building model at different angles.

[0007] Step 2: Simulate the SAR image based on the SAR imaging principle. First, set the imaging parameters, such as the phase center position at the radar imaging time, bandwidth, baseline position, radar platform flight direction and speed, and scattering intensity of each point. Then, set the imaging plane size and the pixel spacing in the range and azimuth directions. Next, project the single-view point cloud onto the imaging plane, mapping each point to its nearest grid cell in the imaging plane, and record the grid position of each point on the imaging plane. Then, calculate the distance from each point in the point cloud to the phase center at the radar imaging time, and convolve it with the two-way echo delay phase using the previously obtained point spread function matrix to obtain the SAR image result. Then, select a signal from a ground area, calculate the power of that area, and add 15dB of random complex noise to the signal power of the image based on the power of that area. Finally, crop the SAR image to a size commonly used in neural networks.

[0008] Step 3, extract building facade points. First, based on the radar viewpoint point cloud... shaft and Axis range, generating mesh size is The horizontal plane of m. The single-view point cloud from the radar perspective is directed towards... The horizontal grid surface is projected, and the number of spatial points projected onto each grid is set as the density value of that grid. Since points on the facade are stacked in the same grid, the density value of the grid corresponding to the facade part is higher. Setting an appropriate threshold can separate the grid corresponding to the facade from other grids. The points projected onto the grid with the higher density value are selected as the facade points of the building.

[0009] Step 4, Annotate SAR Image. First, project the extracted point cloud of the building facade onto a range-azimuth grid surface. Grid values ​​for points within the grid are set to 1, and other grid values ​​are set to 0. Extract the coordinates of the grids with values ​​of 1 to form a point set, and use... The algorithm extracts the contour of the point set. Using the same method, the single-view point cloud is projected onto a grid surface on the range-azimuth plane. Grids without points are set to a value of 1, and other grids are set to 0. The algorithm then uses... The algorithm extracts the outline of the shaded area. The facade area and the shaded area are placed on the same grid map and labeled with different values ​​to obtain the labeled SAR image.

[0010] Step 5: Using the two generated SAR images, calculate the interferometric phase map, which serves as the input to another channel of the neural network, enabling the neural network to utilize both amplitude and phase information.

[0011] Step 6, Dataset Augmentation. Random rotation, scaling, and translation are applied simultaneously to the generated SAR images and labels to expand the number of images in the dataset.

[0012] This completes all the steps.

[0013] The beneficial effects of this invention are as follows:

[0014] This invention uses POY-Ray software to obtain single-view point clouds, scattering times and scattering intensities of scattering points from building models, generates SAR images based on the back projection algorithm, and annotates the SAR images based on the detection of facade points in the point cloud, thus realizing the rapid creation of a SAR image building semantic segmentation dataset. Attached Figure Description

[0015] Figure 1 Simulate a flowchart for the dataset;

[0016] Figure 2 Example diagram of architectural model;

[0017] Figure 3 The point cloud obtained from POV-Ray software is a single-viewpoint cloud.

[0018] Figure 4 To simulate SAR images;

[0019] Figure 5 The interferogram is obtained by interferometry processing of two simulated SAR images;

[0020] Figure 6 The projection image is obtained by projecting a single-view point cloud onto a horizontal plane.

[0021] Figure 7 These are the extracted building facade points;

[0022] Figure 8 The projection diagram obtained by projecting a point on the building facade into the distance-azimuth plane.

[0023] Figure 9 for The elevation area obtained by the algorithm.

[0024] Figure 10The annotation results for the SAR image. Detailed Implementation

[0025] The invention will now be described in detail with reference to the accompanying drawings.

[0026] This invention mainly consists of two parts: SAR image simulation and SAR image annotation. The flowchart is as follows: Figure 1 As shown. The specific steps of this invention are as follows:

[0027] Step 1, Input Parameters. First, you need to obtain various types of architectural model data, such as architectural models... Figure 2 As shown, the model format was converted to POV format. The bottom height of the building model was set to 0m, and the model position was centered on the horizontal plane.

[0028] Next, the building model is imported into POV-Ray software. In the software, the light source position is set as the phase center position for radar imaging, the camera position is the same as the light source position, and the ground is set as a quadratic surface, which can be customized according to needs. Based on the standard "Surface Roughness Parameters and Their Values ​​in Surface Structure Contour Method GB / T 1031-2009", different roughness and scattering coefficient parameters are set for the ground and building respectively to better match the real scene. Then, after rendering with POV-Ray software, the point cloud and the scattering intensity and scattering number of each point are obtained from the radar viewpoint, based on the principle of ray tracing. For each building model, eight single-view point clouds at 45-degree intervals can be generated at different angles. A single-view point cloud at a certain angle is shown below. Figure 3 As shown.

[0029] Step 2: Simulate SAR images based on SAR imaging principles. First, set imaging parameters such as the phase center position at the radar imaging time, bandwidth, baseline position, radar platform flight direction and velocity, scattering intensity of each point, and random phase. Then, set the imaging plane size, range and azimuth pixel spacing, ensuring the imaging plane can encompass the entire point cloud projection onto the range-azimuth plane. Next, project the single-view point cloud onto the imaging plane, mapping each point to its nearest corresponding grid cell on the imaging plane, and recording the grid cell position of each point. Then, construct a two-dimensional point spread function, as shown in the following formula.

[0030] (1)

[0031] in, and These are the range and azimuth resolutions, respectively. A Hamming window is used to reduce the impact of sidelobes. Then, the slant range (distance) is calculated from each point in the point cloud to the phase center at the imaging time. Assume the radar location is... The target location is The formula for SAR three-dimensional imaging is as follows:

[0032] (2)

[0033] in, For carrier frequency, At the speed of light, The distance from the point to the phase center at the moment of radar imaging. for The echo signal at a pixel. As the formula shows, the energy projected onto points within the same grid in the point cloud is superimposed on that grid. Simultaneously, the echo signal within that grid is influenced by the point spread function of other targets at that grid. The distance from each point in the point cloud to the phase center at the radar imaging time is calculated. Convolving the previously calculated point spread function matrix with the two-way echo delay phase yields the SAR image result. The calculation formula is as follows:

[0034] (3)

[0035] After generating the SAR image, random complex noise needs to be added. Select a signal from a ground region, calculate the power of that region, and add 15 dB of random complex noise to the image signal power, using that region's power as the base signal power. Finally, crop the SAR image to a size commonly used in neural networks. A SAR image simulating a building is shown below. Figure 4 As shown.

[0036] Step 3, extract building facade points. First, based on the radar viewpoint point cloud... shaft and Axis range, create a grid with a size of A horizontal grid surface of m is used. The single-view point cloud from the radar perspective is projected onto this horizontal grid surface. The number of spatial points projected into each grid is set as the density value of that grid. Because points on the facade are stacked in the same grid, the density value of the grid corresponding to the facade is high. An appropriate threshold is set to separate the grid corresponding to the facade from other grids; grids with a density value higher than the threshold have a value of 1, and other grids have a value of 0. Figure 6 As shown. Points projected onto the grid with higher density values ​​are selected as the building's facade points. Figure 7 The extracted facade points are displayed.

[0037] Step 4, annotate the SAR image. First, set a grid surface of the same size as the SAR image, located on the range-azimuth plane. The center of the grid surface is the projection position of the scene center onto the range-azimuth plane. The grid size is set to... m. Then, the extracted point cloud of the building facade is projected onto the grid surface. The grid value of the points within the grid that are projected is set to 1, and the grid value of other grids that are not projected to is set to 0. This yields a building facade projection consisting only of 0s and 1s. However, as Figure 8 As shown, the grids with a value of 1 in the projection are not tightly connected. To obtain the outline of the building facade on the projection, the coordinates of the grids with a value of 1 are extracted to form a point set, and then... The algorithm extracts the contour of the point set and sets the value of all grid cells within the contour to 1, such as... Figure 9 As shown, the building facade area has been extracted.

[0038] Project the single-view point cloud onto a grid surface in the range-azimuth plane using the same method. Grids without points are set to a value of 1, grids with points are set to a value of 0, and grids without points are designated as shaded areas. Then, similarly... The algorithm extracts the outline of the point set formed by the shaded area and sets the value of all grid points within the outline to 1. By placing the facade area and the shaded area in the same grid map and labeling them with different values, a labeled SAR image is obtained, such as... Figure 10 As shown.

[0039] Step 5: Generate interferograms. Using the two generated SAR images at different baselines, calculate their interferometric phase maps. The interferogram results are as follows: Figure 5 As shown, as the input of another channel of the neural network, the neural network can utilize both amplitude information and phase information.

[0040] Step 6, Data Augmentation. Random rotation, scaling, and translation are applied simultaneously to the generated SAR images, interferograms, and labeled images. This expands the number of images in the dataset, improves the generalization ability and performance of the neural network, and avoids overfitting.

[0041] This completes all the steps.

[0042] The following provides an implementation example with specific parameters.

[0043] In this example, a common building model is used as the initial data, and POV-Ray software version 3.7 is employed. The parameter values ​​used to simulate the SAR image are shown in Table 1.

[0044] Table 1 SAR image simulation parameters

[0045]

[0046] First, following step 1, import the building model into POV-Ray software to obtain a single-view point cloud and the number of scattering points and scattering intensity. The obtained single-view point cloud is as follows: Figure 2 As shown.

[0047] Step 2 is executed. Based on the imaging principle of the back projection algorithm, SAR images are generated using the parameters shown in Table 1. The image results are as follows: Figure 4 As shown.

[0048] Step 3 involves projecting the single-view point cloud onto a horizontal plane to generate a projection map. The grid containing the building facade points is then extracted based on the density threshold, thus obtaining the point cloud of the building facade.

[0049] Perform step 4, project the building facade point cloud onto the distance-azimuth plane, and use... The algorithm extracts the outline of the facade area and labels the facade area. Then, the same method is used to label the shaded areas. The labeling results are as follows: Figure 10 As shown.

[0050] Perform step 5 to generate an interferogram using two SAR images with different baselines, such as... Figure 5 As shown.

[0051] Perform step 6 to augment the dataset.

[0052] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A fast simulation method for a high-resolution SAR image building segmentation dataset, the method comprising the following steps: Step 1: Input parameters and use POV-Ray software to generate a point cloud from the radar viewpoint; Step 2: Simulate SAR images based on SAR imaging principles: Based on a single-view point cloud, set the intensities of primary and secondary scattering points respectively, and generate SAR images using a two-dimensional point spread function and two-way time delay phase convolution; calculate the distance from each point in the point cloud to the phase center at the radar imaging time, and convolve the previously obtained point spread function matrix with the two-way echo time delay phase to obtain the SAR image result. The calculation formula is as follows: ； in, For carrier frequency, At the speed of light, The distance from the target point to the phase center at the moment of radar imaging. For a two-dimensional point extension function, the formula is: ； in, and These represent the range and azimuth resolutions, respectively. Step 3: Extract building facade points based on the projection density threshold method; Step 4, based on The algorithm extracts and labels the building facades and shadow areas by projecting the point cloud onto the distance-azimuth plane. Step 5: Calculate the interferometric phase map using the two generated SAR images; Step 6, data augmentation.

2. The fast simulation method for a high-resolution SAR image building segmentation dataset as described in claim 1, characterized in that: In step 1, the building model and scene setting parameters are input, and POV-Ray software is used to obtain the point cloud of the building scene from the radar perspective and the scattering intensity and scattering number of each point in the point cloud; For each building model, point clouds with 8 different angles can be generated at 45-degree intervals.

3. The fast simulation method for a high-resolution SAR image building segmentation dataset as described in claim 1, characterized in that: In step 3, the single-view point cloud is projected onto the horizontal plane. Based on the number of spatial points projected into different grids, a threshold is set to select the grids corresponding to the building facade points, thereby obtaining the building facade points in the point cloud.

4. The fast simulation method for a high-resolution SAR image building segmentation dataset as described in claim 1, characterized in that: In step 4, the point cloud of the building facade is projected onto the distance-azimuth plane, and then... The algorithm extracts the contour of the facade projection points, and the internal region of the extracted contour is the building facade region in the SAR image; then the same method is used to obtain the shadow region in the SAR image.

5. The fast simulation method for a high-resolution SAR image building segmentation dataset as described in claim 1, characterized in that: In step 5, based on the two generated SAR images under different baselines, their interferometric phase maps are calculated and used as input data for the second channel of the neural network.

Images