Satellite feature component image automatic labeling and database construction method

Abstract

The present application relates to a kind of satellite feature component image automatic labeling and database construction method, comprising the following steps: (1) key component marking is carried out to satellite three-dimensional model;(2) in unreal engine, build earth background, satellite orbit, camera and other scene components, build simulation space scene;(3) for the satellite three-dimensional model marked in step (1) and the simulation space scene obtained in step (2), batch generates satellite picture and its key component semantic labeling diagram;(4) for the semantic labeling diagram obtained in step (3), label remapping is carried out, standard database format, complete database construction.The present application provides satellite three-dimensional model key component automatic labeling and batch drawing method, and accurately and efficiently realizes the construction of satellite simulation image key component semantic segmentation database.

Description

Technical Field

[0001] This invention is applicable to the field of satellite key component identification in space situational awareness, and specifically relates to a method for automatic image annotation and database construction of satellite feature components. Background Technology

[0002] In recent years, with the continuous development of space technology and the increasing frequency of human activities in space, space situational awareness has become a crucial foundation for space security and control. Space target identification is an important component of space situational awareness, intersecting with several other important fields and possessing significant military value. With various types of satellites launched by different countries and the number of man-made spacecraft in space surging, the reconnaissance and identification of key satellite components has become a vital part of space target identification technology.

[0003] Typical satellite components mainly include the satellite body, antennas, solar panels, and payloads. These observable components exhibit diverse geometric configurations. The satellite body is generally a regular shape with a clearly distinguishable outline, making it relatively easy to extract its shape from visual images. Solar panels, also known as solar cells or solar arrays, can be broadly categorized into bulk-mounted and deployable types. For bulk-mounted panels, their outline largely overlaps with the satellite's main outline, making identification difficult solely based on morphological features; optical reflection characteristics are also necessary. For deployable solar panels, they typically form regular rectangular assemblies when deployed, and their installation position is a certain distance from the satellite body, making outline extraction and identification relatively easy. Satellite antennas are generally the most difficult components to detect and identify. These communication antennas include global beam, hemispherical beam, regional beam, spot beam, multi-beam, and variable beam types. Satellites carry different antennas depending on their mission, and often more than one. The reflector antenna has a relatively clear outline, but its close proximity to the transmitter and receiver can easily interfere with the identification process. The horn antenna is relatively small in size compared to the satellite and is generally mounted on a bracket, maintaining a certain distance from the satellite surface. Phased array antennas and helical antennas are also mounted on brackets, but are generally relatively close to the satellite surface. Therefore, some prior knowledge about the satellite can be used to assist in the effective identification of the components.

[0004] Segmentation of key components in satellite images often utilizes deep learning-based semantic segmentation algorithms. Dataset preparation is essential before model training, and the quality of the dataset directly impacts the prediction accuracy of the algorithm. For key target identification in satellite images, a large number of diverse real-world satellite images are needed, along with pixel-level annotation of key components. This poses a challenge to dataset preparation. First, the number of actual satellite images acquired through space-based or ground-based cameras is limited, and most real-world images suffer from defects such as motion blur, overexposure, and small satellite targets, making them unsuitable for key component segmentation. Second, pixel-level annotation for semantic segmentation requires significantly more work than polygon annotation for target detection. Annotating key satellite components demands substantial manpower, requiring annotators with specialized knowledge of satellite structure and function. Manual annotation can reduce the accuracy of the dataset. In conclusion, a cost-effective, highly automated, and accurately annotated method for constructing a semantic segmentation dataset for key satellite components will provide effective data support for the field of space situational awareness and has high application value. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies and address the common problems in constructing semantic segmentation datasets for key satellite components, such as the difficulty in obtaining real image data and the large workload of labeling key component information, this invention builds a simulation environment and provides a method for automatic annotation of satellite feature component images and database construction. This method automatically annotates key components of satellite 3D models and accurately and efficiently constructs a semantic segmentation database of key components from satellite simulation images.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for automatic image annotation and database construction of satellite feature components includes the following steps:

[0008] (1) Mark the key components of the satellite 3D model;

[0009] (2) Construct scene components including Earth background, satellite orbit, and camera in Unreal Engine to build a simulated space scene;

[0010] (3) Based on the labeled satellite 3D model obtained in step (1) and the simulated space scene obtained in step (2), generate satellite images and semantic annotation maps of their key components in batches;

[0011] (4) For the semantic annotation map obtained in step (3), perform label remapping, standardize the database format, and complete the database construction.

[0012] Furthermore, in step (1), key components of the satellite 3D model are marked, including:

[0013] 1.1) Obtain the satellite's 3D model and open the model;

[0014] 1.2) Based on the actual structural information of the satellite, distinguish key components including solar panels, solar panel supports, antennas, optical payloads, propulsion components, and star sensors in the satellite 3D model, and assign different materials to each type of component and mark them.

[0015] 1.3) Export the original model and the labeled model separately and then import them into Unreal Engine.

[0016] Further, step (2) includes:

[0017] 2.1) Construct the Earth and satellite orbits in Unreal Engine at a scale of 1:20, and draw the starry sky background;

[0018] 2.2) Configure the camera according to the actual camera parameters and preset multiple camera shooting positions and shooting angles.

[0019] Further, step (3) includes:

[0020] 3.1) Import the original satellite model and the marked satellite model obtained in step (1) into Unreal Engine, set the attitude and motion of the two models to be consistent, and use the two cameras configured in step 2.2) to take pictures of the two models respectively, and use two rendering targets to capture the obtained images;

[0021] 3.2) Edit the rendering target. For the original satellite image, set the capture source to LDR final color, display all scene components except the labeled satellite model, and the display effect is the satellite image taken by the space-based camera; for the labeled satellite image, set the capture source to RGB basic color, hide all scene components except the labeled satellite model, turn off all lighting, and the display effect is the annotation result of the key components of the original satellite image.

[0022] 3.3) Design the program to generate images in batches, including functions such as changing the model, changing the background, adjusting the satellite attitude, adjusting the shooting angle, and taking screenshots at set times; design an animation sequence to combine the above functions to achieve batch automatic screenshot function and automatically save the original satellite image and semantically annotated image.

[0023] Further, step (4) includes:

[0024] 4.1) Based on the material label list formed by different materials corresponding to different feature components in step 1.2), map the satellite semantic annotation map generated in batches in step 3.3) from the RGB image to the grayscale image of the semantic label;

[0025] 4.2) Set up the dataset directory according to the format of the CityScapes dataset, standardize image naming, and generate index files.

[0026] Beneficial effects:

[0027] The method employed in this invention is simple in principle and clear in its approach, enabling highly automated generation of simulated satellite images and pixel-level semantic annotation of key components. It reduces the workload of manual annotation while avoiding errors. The constructed dataset meets the requirements of large dataset size, high diversity, and fine-grained annotation for semantic segmentation of key satellite components, providing data support for the field of space situational awareness. Attached Figure Description

[0028] Figure 1 This is a flowchart of a method for automatic annotation of satellite feature component images and database construction according to the present invention. Detailed Implementation

[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0030] like Figure 1 As shown, the specific implementation steps of the automatic annotation and database construction method for satellite feature component images of the present invention are as follows:

[0031] Step 1: Mark key components of the satellite 3D model. First, obtain the satellite 3D model, which was created by a professional 3D modeling company using 3ds Max software. The process consists of four steps: reference material search, satellite geometric modeling, satellite material mapping, and rendering. Specifically, reference material search involves collecting relevant satellite information from well-known satellite or aerospace websites, including the satellite model's scale, dimensions, and materials. Next, through analysis of various aspects of the product and dimensional calculations, a scaled-down model is created, and key names are labeled for the satellite layers to facilitate subsequent material replacement. Then, satellite texture mapping is performed, unfolding the model's texture coordinates, drawing material textures and details, and exporting the finished texture map. Finally, the 3D model and texture map are rendered, and the satellite model is exported in .max format.

[0032] The next step, based on the actual structural information of the satellite and the layer annotation information in the satellite's 3D model, is to distinguish key components in the 3D model, including solar panels, solar panel supports, antennas, optical payloads, propulsion components, star sensors, docking rings, robotic arms, the main body, and other equipment, totaling 10 categories. In 3ds Max software, different solid color materials are assigned to different key components, and the mapping from key components to colors is recorded in a table. The same mapping is performed on all satellite models to complete the labeling of the satellite's 3D model. Finally, the original model and the labeled model are exported as .fbx format for easy import into Unreal Engine.

[0033] Step 2: Construct scene components such as the Earth background, satellite orbits, and cameras in Unreal Engine to build a simulated space scene. To simulate the imaging effect of satellites in the space environment, Unreal Engine is used to build a simulated space environment, incorporating as much realistic physical information as possible. First, the Earth background is constructed. A new scene component blueprint control is created, and a blank sphere at a 1:20 scale is added as the Earth's skeleton. 8K ultra-high-definition day and night Earth textures, normal maps, and cloud maps are applied to the material sphere and attached to the Earth skeleton. The skeleton's rotation speed and its interaction with time and lighting are set. The simulated Earth blueprint is now complete. Following the same approach, an 8K ultra-high-definition starry sky background blueprint is constructed using an 8K ultra-high-definition starry sky background image. Multiple circular and elliptical tracks are set around the Earth, covering various altitudes from near-Earth orbit (2000 km) to geostationary orbit (36,000 km), and various tilt angles and orbit shapes are set according to the actual satellite operation. The simulated space environment is now complete.

[0034] The next step involves constructing a simulated shooting environment using Unreal Engine's camera controls. To simulate the imaging effect of a space-based camera capturing satellite images, the Unreal Engine camera components are configured with real camera parameters, including a focal length of 2000mm, a field of view of 0.4 degrees, an aperture of 610mm, and a film size of 14mm. A camera boom control is created on the satellite orbit, and the camera and satellite model are bound to the boom to ensure that the camera can always capture the satellite model during satellite movement. The camera position is adjusted, setting multiple shooting distances from close range (within 100 meters) to ultra-long range (50km), and various shooting angles are also set. This step is to simulate the uncertainties of space satellite imaging and to improve the diversity of satellite semantic segmentation datasets.

[0035] Step 3: Based on the labeled satellite 3D models obtained in Step 1 and the simulated space scene obtained in Step 2, generate satellite images and semantic annotation maps of their key components in batches. First, import all original satellite models and their labeled key component models into Unreal Engine for later use. Create two new static mesh scene components to hold the models. Embed two models of a specific satellite model into the mesh components. Place the models on the simulated satellite orbit designed in Step 2, ensuring the positions and attitudes of the two models are consistent. Use two configured cameras to capture the two models respectively. To facilitate simultaneous imaging and image saving of both models, use Unreal Engine's "Render Target" function to hold the satellite images generated by the cameras for subsequent processing. A render target is a texture in Unreal Engine that can be written to at runtime. It can store information such as base color, normals, and ambient occlusion, allowing users to convert the engine's rendering results into material textures for further editing, rendering, or storage operations. For example, a scene can be captured pointing at an object and the image stored in the render target. Then, the render target can be displayed on the mesh to simulate camera functionality. In the specific application of this invention, two rendering targets are used to capture satellite images generated by two cameras. The rendering targets are edited: for the original satellite image, the capture source is set to LDR final color, all scene components except the labeled satellite model are displayed, the gamma value is adjusted to simulate ambient light in space, and motion blur is applied to simulate image degradation caused by satellite motion; for the labeled satellite image, the capture source is set to RGB basic color, all scene components except the labeled satellite model are masked, all lighting is turned off, and other settings remain consistent with the first rendering target. The display effect is the annotation result of key components from the original satellite image. The two rendering targets are then packaged into a rendering target texture for later use.

[0036] The main functions of the program are designed below: batch generation of satellite images and their semantic annotation maps. The main interface is designed using control blueprints, and buttons are added and implemented using blueprint programming to achieve five sub-functions: model replacement, background replacement, satellite attitude adjustment, shooting angle adjustment, and timed screenshot. The model replacement function is implemented based on the function "get all instantiated components of a class," which retrieves all static mesh components and creates two lists: the original satellite model and the labeled satellite model. These lists are used to replace the corresponding static mesh scene components, allowing the replacement of the original and labeled models for a specified satellite model during program execution. The background replacement function changes the Earth background in the satellite images, including Earth size, Earth's rotation position, shadow area, cloud thickness, etc. This function is achieved by adjusting the satellite orbit and its position in the orbit, combined with simulated sunlight effects, to physically simulate diverse Earth backgrounds. The satellite attitude adjustment function is achieved by adding horizontal and vertical rotational movements of satellite mesh components in the control blueprint, ensuring consistent movement between the original and labeled models. The shooting angle adjustment function adds the shooting angle settings from step 2 to the control blueprint to simulate the uncertainties of space satellite imaging. The timed screenshot function is implemented based on the conversion between Unreal Engine material textures and images. It converts the rendering target texture of satellite images into .png format images and writes them locally. The original image and the labeled image are stored in pairs and categorized according to satellite model.

[0037] Finally, based on the Unreal Engine's level sequence function, an animation sequence was designed, combining the sub-functions of the above program to achieve batch automatic screenshotting and automatic saving of original satellite images and semantically annotated maps. Specifically, for each satellite model, over 100 background scenarios, 10 horizontal attitudes, 10 vertical attitudes, and 20 shooting angles were set, generating a total of 200,000 images. These parameters can be flexibly adjusted according to actual needs and satellite model conditions. The batch automatic screenshotting function can significantly reduce the configuration time before screenshotting, improve database construction efficiency, and reduce manual labor consumption. The generated images will be used as the raw database for further processing.

[0038] Step 4: For the semantically annotated images obtained in Step 3, perform label remapping, standardize the database format, and complete the database construction. The Cityscapes dataset focuses on semantic understanding of urban street scenes and consists of street scenes from 50 different cities. The dataset includes 5,000 precisely annotated images and 20,000 roughly annotated images, making it an important dataset in the field of image semantic segmentation. The satellite key component semantic segmentation dataset of this invention will be constructed according to the format of the Cityscapes dataset.

[0039] The satellite semantic annotation map generated in step 3 is an RGB color image, where each color represents a type of component, while the semantic annotation map of the Cityscapes dataset is a grayscale image, where each grayscale value represents a category. Following the mapping table from key components to colors used in the satellite 3D model annotation in step 1, the three-channel color image is remapped to a grayscale image to obtain the final satellite semantic annotation map.

[0040] Next, the dataset directory is set up according to the Cityscapes dataset format. All images taken by each type of satellite are treated as a sequence, and each image is considered a frame in the sequence. The root directory distinguishes between original images and labeled images, the first-level directories distinguish between sequences, and the second-level directories store all images in that sequence. Images are named in the format "satellite model_sequence number_frame number.file extension". The design script automatically generates an index file that maps image names to storage locations. At this point, the semantic segmentation dataset for key satellite components is complete.

[0041] Contents not described in detail in this specification are common knowledge to those skilled in the art. Although illustrative specific embodiments of the invention have been described above to facilitate understanding by those skilled in the art, it should be understood that the invention is not limited to the scope of the specific embodiments. Various modifications will be readily apparent to those skilled in the art as long as they fall within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the inventive concept are protected.

Claims

1. A method for automatic image annotation and database construction of satellite feature components, characterized in that, Includes the following steps: Step (1) Mark the key components of the satellite 3D model, including: Step (1.1) Obtain the satellite 3D model and open the model; Step (1.2) Based on the actual structural information of the satellite, distinguish key components including solar panels, solar panel supports, antennas, optical payloads, propulsion components, and star sensors in the satellite 3D model, and assign different materials to each type of component for marking. Step (1.3) involves exporting the original model and the labeled model separately and then importing them into Unreal Engine; Step (2) Construct scene components including Earth background, satellite orbit, and camera in Unreal Engine to build a simulated space scene; Step (3) Based on the labeled satellite 3D model obtained in step (1) and the simulated space scene obtained in step (2), generate satellite images and semantic annotation maps of their key components in batches, including: Step (3.1) Import the original satellite model and the marked satellite model obtained in step (1) into Unreal Engine, set the pose and motion of the two models to be consistent, use the cameras configured in step (2.2) to take pictures of the two models respectively, and use two rendering targets to capture the obtained images; Step (3.2) Edit the rendering target. For the original satellite image, set the capture source to LDR final color, display all scene components except the marked satellite model, and the display effect is the satellite image taken by the space-based camera; for the marked satellite image, set the capture source to RGB basic color, hide all scene components except the marked satellite model, turn off all lighting, and the display effect is the annotation result of the key components of the original satellite image. Step (3.3) Design the batch image generation function of the program to realize the functions of changing the model, changing the background, adjusting the satellite attitude, adjusting the shooting angle, and taking screenshots at time; design an animation sequence to combine the above functions to realize the batch automatic screenshot function and automatically save the original satellite image and semantic annotation image; Step (4) involves remapping the semantic annotation map obtained in step (3), standardizing the database format, and completing the database construction, including: Step (4.1) Map the satellite semantic annotation map generated in batches in step (3.3) from the RGB image to the grayscale image of the semantic label according to the material label list formed by the different materials corresponding to the different feature components in step (1.2); Step (4.2) Set up the dataset directory according to the format of the CityScapes dataset, standardize the image naming, and generate the index file.

2. The method for automatic annotation of satellite feature component images and database construction according to claim 1, characterized in that: Step (2) includes: 2.1) Construct the Earth and satellite orbits in Unreal Engine at a scale of 1:20, and draw the starry sky background; 2.2) Configure the camera according to the actual camera parameters and preset multiple camera shooting positions and shooting angles.

Images