Space target sequence image generation method and device for deep space background

Abstract

The application provides a space target sequence image generation method and device for a deep space background, electronic equipment and a medium, and relates to the field of space target optical image simulation. The method comprises the following steps: obtaining pose data of a space target in an observation period through simulation based on predefined initial parameters; selecting imageable stars from a reference star library according to the field of view and detection capability of an on-orbit optical camera; generating a deep space background image matched with the observation field of view of the space target through brightness simulation and coordinate system transformation based on the position data of the imageable stars; generating a time sequence rendering image through a rendering engine by using a three-dimensional model and pose data of the space target; and superimposing the deep space background image and the time sequence rendering image according to the foreground and background distribution, and adding noise to obtain a simulation sequence image.

Description

Technical Field

[0001] This invention relates to the field of optical image simulation of space targets, and specifically to a method, apparatus, electronic device, and medium for generating sequence images of space targets against a deep space background. Background Technology

[0002] Space target identification technology, as a core means of ensuring the safety of space activities, is becoming increasingly important with the increasing frequency of such activities. Key tasks of this technology include rapid detection, feature extraction, and precise identification of on-orbit targets, requiring accurate acquisition of characteristic parameters such as the target's size, configuration, and attitude. These parameters not only provide crucial support for tasks such as on-orbit repair of failed satellites and recovery of space targets, but also effectively prevent potential collisions, improve the utilization rate of orbital resources, and provide a safe space environment.

[0003] Space surveillance systems mainly consist of two parts: space-based surveillance systems and ground-based surveillance systems. The payloads of space-based surveillance systems primarily operate in the visible light band. Due to the unique characteristics of space targets and the sensitivity of space regions, obtaining accurate images is difficult, and the number of samples is limited. Therefore, when identifying space targets, there is a lack of a sufficient number of labeled samples for training classifiers. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of the above problems, the present invention provides a method, apparatus, device and medium for generating spatial target sequence images with deep space background, so as to at least partially solve the technical problems of insufficient dynamics of simulated targets and backgrounds, high simulation complexity and lack of fine-grained target labels for algorithm training.

[0006] (II) Technical Solution

[0007] This invention provides a method for generating a sequence of space target images against a deep-space background, comprising: acquiring the pose data of the space target during the observation period through simulation based on predefined initial parameters; selecting imageable stars from a reference star library according to the field of view and detection capability of the on-orbit optical camera; generating a deep-space background image matching the observation field of view of the space target through brightness simulation and coordinate system transformation based on the position data of the imageable stars; generating a time-series rendered image using a rendering engine using the three-dimensional model and pose data of the space target; and superimposing the deep-space background image and the time-series rendered image according to the foreground and background distribution, and adding noise to obtain a simulated sequence of images.

[0008] According to an embodiment of the present invention, the method further includes: selecting multiple moments from the observation period when screening imageable stars; and performing multi-moment and multi-view simulations on the on-orbit optical camera based on the pose data of the space targets at different moments to obtain the position data of imageable stars at the corresponding moments.

[0009] According to an embodiment of the present invention, generating a deep-space background image that matches the observation field of view of a space target based on the position data of an imageable star through brightness simulation and coordinate system transformation includes: constructing a coordinate system and performing coordinate system transformation based on the position data of the imageable star to obtain the position coordinates of the imageable star on a two-dimensional imaging plane; performing brightness simulation of the imageable star through a Gaussian diffusion model and combining it with the position coordinates to obtain a deep-space background image.

[0010] According to an embodiment of the present invention, generating a time-series rendered image using a rendering engine based on the 3D model and pose data of a space target includes: importing the 3D model of the space target into the rendering engine and determining the position and orientation of the on-orbit optical camera; calculating the bounding box of each space target and its important components in the view of the on-orbit optical camera, converting the coordinates of the bounding box into the target format, and saving it to the corresponding component label file; updating the position and orientation of the space target and the on-orbit optical camera according to the pose data of the space target in the current frame, rendering the scene, and obtaining a time-series rendered image and the corresponding component label file.

[0011] According to an embodiment of the present invention, the deep space background image and the time-series rendered image are superimposed according to the foreground and background distribution, and noise is added to obtain a simulated sequence image. This includes: using a resampling algorithm to scale the deep space background image and the time-series rendered image; performing semi-transparent superposition on the scaled deep space background image and the time-series rendered image to obtain a superimposed image; performing coordinate mapping and label normalization processing on the superimposed image, and adding noise to the processed superimposed image to obtain a simulated sequence image.

[0012] According to an embodiment of the present invention, the method further includes: extracting target data from the original star catalog to obtain a reference star library.

[0013] According to an embodiment of the present invention, the initial parameters include time parameters, orbital parameters, and sensor parameters.

[0014] A second aspect of the present invention provides a space target sequence image generation device for deep space background, comprising: a data parsing module for acquiring the pose data of a space target during the observation period through simulation based on predefined initial parameters; a star selection module for selecting imageable stars from a reference star library according to the field of view and detection capability of the on-orbit optical camera; a background generation module for generating a deep space background image matching the observation field of view of the space target based on the position data of the imageable stars through brightness simulation and coordinate system transformation; a target simulation module for generating a time-series rendered image through a rendering engine using the three-dimensional model and pose data of the space target; and an image synthesis module for superimposing the deep space background image and the time-series rendered image according to the foreground and background distribution, and adding noise to obtain a simulated sequence image.

[0015] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements various steps in a method for generating spatial target sequence images against a deep space background.

[0017] (III) Beneficial Effects

[0018] 1. The method for generating space target sequence images for deep space background provided by the present invention simulates the optical imaging process of space targets based on space-based observation, and can provide data support for the development and verification of various algorithms such as ground-based space target classification and fine-grained detection.

[0019] 2. Establish a real-time update mechanism to simulate the dynamic changes of the target and background by changing the camera position, satellite attitude and star background, and generate simulated sequence images of the space target.

[0020] 3. An automated generation process for space target simulation sequence images and component label files was constructed, which can realize batch automatic rendering of images and synchronous generation of component-level labels, with higher efficiency and accuracy compared to manual annotation.

[0021] 4. The simulated sequence images are scalable, and can be used to simulate the dynamic interaction process of multiple targets according to actual needs, and simulate the relative motion, collision risk assessment and collaborative operation between multiple satellites based on orbit information and other factors. Attached Figure Description

[0022] To gain a more complete understanding of the invention and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, wherein:

[0023] Figure 1 The flowchart illustrates a method for generating spatial target sequence images against a deep space background, as provided in an embodiment of the present invention.

[0024] Figure 2 The schematic diagram illustrates the overall flowchart of the spatial target sequence image generation method for deep space background provided by an embodiment of the present invention;

[0025] Figure 3 The illustrations show deep space background images at different times provided in embodiments of the present invention;

[0026] Figure 4 The illustrations show spatial target rendering images at different times provided in embodiments of the present invention;

[0027] Figure 5 This illustration schematically shows an image generated by overlaying a deep space background image and a space target rendering image, as provided in an embodiment of the present invention.

[0028] Figure 6 The illustration schematically shows a simulated sequence image with added noise provided in an embodiment of the present invention;

[0029] Figure 7 The illustration schematically shows a visual image of a spatial target component label provided in an embodiment of the present invention;

[0030] Figure 8 This schematic diagram illustrates the structural block diagram of the spatial target sequence image generation device for deep space background provided in an embodiment of the present invention;

[0031] Figure 9 The diagram illustrates the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0032] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0033] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0034] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0035] The accompanying drawings show some block diagrams and / or flowcharts. It should be understood that some blocks or combinations thereof in the block diagrams and / or flowcharts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions can create means for implementing the functions / operations described in these block diagrams and / or flowcharts.

[0036] Research has revealed that current space target simulations primarily face challenges such as insufficient dynamics of simulated targets and backgrounds, high simulation complexity, and a lack of fine-grained target labels for algorithm training. Some technologies construct datasets of small high-orbit space targets by simulating different signal-to-noise ratios, clutter environments, and target grayscale values. However, this process uses simple long rectangles and circles to simulate space targets and stars, with the star positions being randomly determined, resulting in discrepancies with real-world images. Furthermore, other technologies generate rich datasets through stellar background modeling, space target imaging modeling, and noise and stray light simulation, but these primarily focus on static image simulation, lacking simulation of dynamic changes in targets and backgrounds. Some related technologies generate simulated space target images against complex space backgrounds, including the Earth, Moon, Sun, and stars, using techniques such as ray tracing, star catalog calculation, projected coordinate transformation, stray light empirical formulas, and texture mapping. However, these also lack simulation of dynamic changes in targets and backgrounds. A more effective approach involves generating dynamic digital images through steps such as target imaging model design, target digital library construction, stellar background mapping, illumination background modeling, noise superposition, and comprehensive motion simulation, simulating the imaging effects of space-based observation platforms in real-world space scenarios. However, the simulation process involves multiple steps such as model design and digital library construction, which is highly complex and may lead to low simulation efficiency. Computer-generated imagery (CVA) and image processing technologies can simulate the imaging process of an on-orbit optical camera, generating simulated images with component-level labels, providing data support for the ground-based development and verification of space target recognition algorithms. To address the above problems, this invention provides a method for generating sequence images of space targets against a deep-space background.

[0037] like Figure 1 As shown, the flowchart of the method for generating spatial target sequence images for deep space background includes operations S1 to S5.

[0038] In operation S1, based on predefined initial parameters, the pose data of the space target during the observation period are obtained through simulation.

[0039] In operation S2, imageable stars are selected from the reference star library based on the field of view and detection capabilities of the on-orbit optical camera.

[0040] In operation S3, based on the position data of imageable stars, a deep space background image matching the observation field of view of the space target is generated through brightness simulation and coordinate system transformation.

[0041] In operation S4, the three-dimensional model and pose data of the spatial target are used to generate time-series rendered images through the rendering engine.

[0042] In operation S5, the deep space background image and the time-series rendered image are superimposed according to the foreground and background distribution, and noise is added to obtain the simulation sequence image.

[0043] In some exemplary embodiments, the simulation object is mainly an on-orbit satellite, which simulates the process of near-field space-based observation. The dataset is constructed by simulating space targets, stellar background, clutter environment and target gray values, and is used for training, testing and verification of machine learning algorithms or deep learning algorithms.

[0044] The method for generating a sequence of space target images against a deep-space background, provided in this invention, first involves constructing the observation scene and coordinate system. Space-based observation primarily involves passive detection against a relatively simple deep-space background. During measurement, an optical system collects the spectral energy reflected from the target and projects it onto the focal plane of the detector, acquiring a sequence of observation images through continuous shooting. Elements in the observation scene include the space target, the observation satellite, the on-orbit optical camera (mounted on the observation satellite), and the observation background. Based on these observation elements, a geocentric inertial coordinate system (such as the J2000 coordinate system), a satellite body coordinate system, a camera coordinate system, and an imaging plane coordinate system can be constructed.

[0045] Specifically, interdisciplinary engineering software can be used to create simulation scenarios to model, analyze, and simulate complex systems. Within the simulation software, appropriate initial parameters can be set, including but not limited to time parameters, orbital parameters, and sensor parameters, as well as the tracking modes for the sensors and the observation satellite. Time parameters can include the simulation start time, simulation end time, and simulation step size. Next, by importing a two-line element (TLE) file, the orbital parameters for both the observation satellite and the observed satellite (i.e., the space target) can be set. The TLE file can contain parameters such as the satellite's semi-major axis, eccentricity, orbital inclination, right ascension of the ascending point, argument of perigee, and mean perigee angle. Then, sensors can be added to the observation satellite, and the sensor type (such as an on-orbit optical camera), resolution, field of view, and pointing angle can be set, along with the tracking modes for the sensors and the observation satellite.

[0046] Figure 2 The schematic diagram illustrates the overall flowchart of the spatial target sequence image generation method for deep space background provided by an embodiment of the present invention.

[0047] like Figure 2 As shown, the method for generating a sequence of space target images can include three modules: sequence star background generation, target simulation, and foreground / background overlay. Initial star catalogs, analytical data, and 3D models can be used as initial inputs. First, based on the processed satellite position and attitude data, imageable stars at the corresponding time are obtained and projected onto a 2D plane using coordinate system transformation and a Gaussian diffusion model. Then, the position and attitude of the 3D model of the space target in the rendering engine are controlled to obtain the rendered image of the target at the corresponding time. Finally, the star background image and the rendered image of the space target at the corresponding time are overlaid to obtain the final sequence of images.

[0048] Furthermore, such as Figure 2 As shown, based on the simulation platform and the set initial parameters, the position and attitude data (i.e., pose data) of the observed and observed satellites at each simulation moment can be accurately calculated. Specifically, based on the set initial parameters, the position and attitude data of the observed and observed satellites within one simulation cycle can be obtained through simulation software. The position data can be output in Cartesian coordinates, including the satellite's x, y, and z coordinates in the J2000 coordinate system and the satellite's velocities Vx, Vy, and Vz in the x, y, and z directions. The attitude data can be output in Euler angles or quaternions, including the satellite's yaw, pitch, and roll angles relative to the J2000 coordinate system. After obtaining the reference star library, stars that can be imaged on the camera's focal plane can be screened based on the equipment's detection capabilities and the camera's field of view, resulting in a preliminary list of imageable stars.

[0049] In the process of screening for imageable stars, the maximum apparent magnitude that can be obtained can be determined by the detection capabilities of the detection instruments. The larger the magnitude of a star, the lower its brightness; therefore, the magnitude of a star... A star can be captured by a detection instrument if the following equation is satisfied.

[0050]

[0051] A star can only be imaged on the camera's imaging plane when its right ascension and declination, along with the optical axis of the observation camera, satisfy the camera's field of view constraints.

[0052]

[0053] in, and The right ascension and declination of stars. and Right ascension and declination are the points of view of the camera's optical axis at a given moment; FOV is the camera's field of view.

[0054] After two rounds of selection, stars that can be imaged at the camera's focal plane at certain times can be identified. For these imageable stars, deep-space background images can be generated through brightness simulation and coordinate system transformation. Then, space target simulation is performed, using a 3D model and rendering engine based on the satellite, combined with the satellite's position and attitude data to obtain time-series rendered images. Finally, the generated deep-space background images and the simulation sequence images are overlaid according to foreground and background distribution, and noise is added to the overlaid image to obtain the final space target simulation sequence image facing the deep-space background.

[0055] It is understood that the embodiments of the present invention generate a series of space target simulation sequence images with fine-grained labels by simulating satellite and star backgrounds at different shooting times and superimposing foreground and background images, thus providing data support for the development and verification of ground-based space target classification algorithms and fine-grained detection algorithms.

[0056] In an embodiment of the present invention, the method further includes: extracting target data from the original star catalog to obtain a reference star library.

[0057] For example, the SAO (Smithsonian Astrophysical Observatory) catalog can be chosen as the original catalog. It contains data on 258,997 stars, covering a wide area of ​​the sky, providing a rich data foundation for the preparation of the reference star library.

[0058] While the original star catalog contains a wealth of celestial information, it also contains a significant amount of redundant data. For example, celestial parameters irrelevant to the research objective, duplicated data, or descriptions requiring unnecessary detail for the current task increase the complexity and computational burden of data processing, reducing the efficiency of subsequent analysis. Therefore, the necessary effective information (target data) can be extracted from the original star catalog to obtain a sub-catalog, i.e., a reference star library, free of redundant data. This effective information can revolve around stars, such as their identification numbers, right ascension, declination, and apparent magnitude. Star identification numbers are crucial for recognizing and distinguishing different stars, right ascension and declination determine a star's position in the sky, and apparent magnitude reflects its brightness.

[0059] Understandably, the preparation of the reference star library provides an efficient and concise data source for stellar background simulation and related astronomical research.

[0060] Based on the above embodiments, in this embodiment, the method further includes: selecting multiple moments from the observation period when screening imageable stars; and performing multi-moment and multi-view simulations on the on-orbit optical camera based on the pose data of the space targets at different moments to obtain the position data of imageable stars at the corresponding moments.

[0061] When selecting the field of view for an on-orbit optical camera, multiple moments within the observation period can be chosen. This process comprehensively considers the motion characteristics of the target celestial body, the satellite's orbital characteristics, and the requirements of the observation mission. Then, based on the pose data of the space target at different moments—namely, the satellite's position (longitude, latitude, altitude) and attitude (pitch angle, yaw angle, roll angle)—multi-moment, multi-view simulations are performed on the on-orbit optical camera. During the simulation, for each moment and each viewpoint, the position data of imageable stars within the camera's field of view can be recorded, such as the star's right ascension, declination, and apparent star size. Finally, the acquired imageable star position data can be saved in a specific format using a database, text file, or binary file, along with the corresponding time, satellite pose data, and simulation parameters, so that the simulation process and results can be accurately reconstructed when needed.

[0062] Understandably, by analyzing the simulation results at multiple times and from multiple perspectives, we can understand the observation capabilities of the on-orbit optical camera at different times and perspectives, discover blind spots and areas with poor imaging quality, and thus optimize the camera's observation strategy.

[0063] Based on the above embodiments, in this embodiment, generating a deep-space background image that matches the observation field of view of a space target by means of brightness simulation and coordinate system transformation based on the position data of imageable stars includes: constructing a coordinate system and performing coordinate system transformation based on the position data of imageable stars to obtain the position coordinates of imageable stars on a two-dimensional imaging plane; performing brightness simulation of imageable stars through a Gaussian diffusion model and combining the position coordinates to obtain a deep-space background image.

[0064] Based on the selected imageable stars, the position coordinates of the stars on the two-dimensional imaging plane can be obtained by constructing a coordinate system and performing coordinate transformation. Then, the brightness of the stars can be simulated using a Gaussian diffusion model, and combined with the position coordinates, a deep-space background image can be obtained.

[0065] Specifically, based on the right ascension, declination, and apparent star data of the selected stars, the coordinates and grayscale values ​​of the stars are calculated.

[0066] The unit vector of a star in the J2000 coordinate system can be expressed as:

[0067]

[0068] Among them, U star V star and W star This represents the unit vector component of the star in the J2000 coordinate system.

[0069] By rotating and translating the coordinate axes, the coordinates of the star in the star's body coordinate system are obtained as follows:

[0070]

[0071] Where R1 is the rotation matrix from the J2000 coordinate system to the satellite body coordinate system; X star Y star and Z star The coordinates of the star in the satellite's body coordinate system; U sat V sat and W sat This represents the position vector components (translation part) of the satellite in the J2000 coordinate system.

[0072] The rotation matrices are calculated using Euler angles. The basic rotation matrices corresponding to rotations of the coordinate system around the X, Y, and Z axes are as follows:

[0073]

[0074]

[0075]

[0076] in, For roll angle, The pitch angle, This is the yaw angle.

[0077] Euler angles describe the rotation process of a coordinate system based on rotation matrices. The rotation order of the rotation matrix is ​​divided into external rotation (x, y, z) and internal rotation (z, y, x). The rotation matrix corresponding to internal rotation is:

[0078]

[0079] Transformation from satellite coordinate system to camera coordinate system:

[0080]

[0081] Where X, Y, and Z are the coordinates of the star in the camera coordinate system; R2 is the rotation matrix from the satellite body coordinate system to the camera coordinate system.

[0082] Since the distance between the satellite coordinate system and the camera coordinate system is relatively short, when performing coordinate system transformation, the error caused by coordinate system translation can be ignored, and only the rotation transformation of the coordinate system needs to be performed.

[0083] During imaging, the pinhole imaging model is used to calculate the imaging position of the target. The coordinates of the star in the imaging plane coordinate system are:

[0084]

[0085] Where, N X and N y Where is the camera resolution, and FOV is the camera's field of view.

[0086] Stellar brightness simulation can be achieved using a Gaussian diffusion model, where the combination of the number of pixels (diffusion range) and pixel brightness (amplitude of the Gaussian kernel) jointly determines the simulation effect of a single star. Specifically, a baseline brightness value is returned based on the magnitude, then a two-dimensional Gaussian diffusion is applied at a specified location to simulate the brightness distribution of the star, and finally, the brightness value is scaled to an 8-bit grayscale range of [0, 255] through normalization.

[0087] The brightness is determined by the amplitude of the Gaussian kernel and the magnitude through a function mapping relationship, as shown in Table 1.

[0088] Table 1. Mapping Relationship between Star Magnitude and Reference Brightness

[0089]

[0090] For each pixel (x, y) in the image, calculate its distance from the center of the star, and use the Gaussian formula to calculate the brightness contribution of that point, g, given by the following equation:

[0091]

[0092] Where A is the base brightness value of the star. Let be the standard deviation of the Gaussian function, and (x0, y0) be the center point of the star.

[0093] By calculating the coordinates and grayscale values ​​of stars on a two-dimensional imaging plane, simulated images of the stellar background at different times can be obtained, i.e., deep-space background images, such as... Figure 3 As shown, (a)-(e) are stellar background images at five different times. That is, based on the position and attitude data output by the satellite toolkit software and the right ascension and declination data in the star catalog, the stellar background at different times can be obtained. When generating the stellar background, a frame can be set to 6 seconds, and each image can correspond to the stellar background at a specific time.

[0094] Based on the above embodiments, in this embodiment, generating a time-series rendered image using the 3D model and pose data of the space target through a rendering engine includes: importing the 3D model of the space target into the rendering engine and determining the position and orientation of the on-orbit optical camera; calculating the bounding box of each space target and its important components in the view of the on-orbit optical camera, converting the coordinates of the bounding box into the target format, and saving it to the corresponding component label file; updating the position and orientation of the space target and the on-orbit optical camera according to the pose data of the space target in the current frame, rendering the scene, and obtaining the time-series rendered image and the component label file corresponding to the image.

[0095] The 3D model of the satellite is imported into the rendering engine, and the temporal position and attitude data in the coordinate system of the simulation software are converted to a right-handed coordinate system. Then, the camera position can be set based on the observed satellite's position and attitude data. The camera's orientation can be calculated from the target satellite's position, ensuring the camera always points towards the target satellite. The light source position is randomly generated and located near the camera. The light source intensity is randomly set; additionally, the ambient light color and intensity can be set. By calculating the bounding box of each object in the camera view, the bounding box coordinates are converted to a target format (such as YOLO format), i.e., normalized values ​​of the center point coordinates, width, and height, and saved to the corresponding part label file. In each frame, the position and attitude of the target satellite and camera can be updated based on the satellite data of the current frame. Then, the scene is rendered and the image is saved, resulting in an example of a space target rendering image, such as... Figure 4 As shown, (a)-(e) are five different time-lapse renderings of the spatial target generated using 3D graphics software. During image rendering, the pose and position data of the camera and the target can be adjusted every 6 seconds, and a corresponding rendering image is generated every 6 seconds. The position, pose, and lighting conditions of the target change in each rendering image.

[0096] Through the above operations, we can simulate real shooting scenarios and obtain multiple satellite target images (i.e., sequentially rendered images) taken at different times in a continuous shooting process, image category labels, and corresponding component-level labels (stored in component label files).

[0097] Based on the above embodiments, in this embodiment, the deep space background image and the time-series rendered image are superimposed according to the foreground and background distribution, and noise is added to obtain the simulation sequence image. This includes: using a resampling algorithm to scale the deep space background image and the time-series rendered image; performing semi-transparent superposition on the scaled deep space background image and the time-series rendered image to obtain the superimposed image; performing coordinate mapping and label normalization processing on the superimposed image, and adding noise to the processed superimposed image to obtain the simulation sequence image.

[0098] The final simulated image of the space target is obtained by overlaying the star background as the background image and the space target simulation image as the foreground image.

[0099] For example, the LANCZOS resampling algorithm can be used for image scaling. This algorithm achieves smooth resampling by calculating a pixel-weighted average using a convolution kernel based on the sinc function, effectively preserving high-frequency details and reducing jagged edges when reducing image size. The core of this algorithm is the Lanczos kernel, which is defined as:

[0100]

[0101] Where x is the relative distance between the current pixel position and the target pixel position; L(x) is the weight value of the Lanczos kernel at position x; Let a be the normalized sine function, and a be the radius of the Lanczos window.

[0102] Secondly, an alpha blending algorithm is used to achieve semi-transparent overlay. By separating the transparency weights of the foreground and background, linear interpolation is performed on each RGB channel. This preserves the texture of the stellar background while highlighting the contour features of the spatial targets, thus obtaining an overlaid image, i.e., a sequence of spatial target images, such as... Figure 5 As shown. Specifically, according to Figure 2 The process described in the text can overlay the rendered images of stars and space objects at corresponding moments, that is,... Figure 3 and Figure 4 The images are overlaid using a semi-transparent overlay method. Each frame in the generated image period is 6 seconds, and the final image is identical to the generated image, also with each frame lasting 6 seconds.

[0103] Finally, through precise coordinate mapping and label normalization, we achieve a dual guarantee of visual quality and consistency of labeled data.

[0104] like Figure 6 As shown, to simulate the real space environment and imaging process, noise is added to the sequence of space target images; that is, different types of noise are added to the superimposed images to simulate the noisy environment in space-based observation. The main types of noise added include Gaussian noise, salt-and-pepper noise, and Poisson noise. Gaussian noise is used to simulate common random noise in space-based observation, such as sensor noise. Salt-and-pepper noise simulates random pixel failures that may occur in space-based observation, such as sensor thermal noise or the effects of cosmic rays. Poisson noise simulates noise in the photon counting process and is proportional to the signal strength.

[0105] Figure 7 The illustration schematically shows a visual image of a spatial target component label provided in an embodiment of the present invention.

[0106] like Figure 7 As shown, the visualized images of space target component labels can display the various components of a space target and their corresponding label information, allowing researchers or operators to quickly identify and understand the composition and structure of the space target.

[0107] While existing technologies generate rich datasets, they primarily focus on simulating static images, lacking simulation of dynamic changes in targets and backgrounds. However, the spatial target sequence image generation method for deep-space backgrounds provided in this invention, through simulation and real-time update mechanisms, can simulate the dynamic changes in targets and backgrounds, generating a series of temporal optical images of spatial targets that more closely resemble real-world space-based observation scenarios. Furthermore, the simulation process in existing technologies is highly complex, involving multiple steps such as target imaging model design and target digital library construction, resulting in low simulation efficiency. This invention simplifies the simulation process by directly utilizing simulation software parsing and rendering engines, reducing complex steps, improving simulation efficiency, and ensuring the quality of the simulated images. In addition, existing technologies lack the generation of fine-grained target labels, simulating spatial targets only through simple geometric shapes, lacking component-level labels. This invention, however, can generate simulated images with component-level labels by calculating the bounding box of each object in the camera view and converting it into a target format, providing richer training data for machine learning and deep learning algorithms. Furthermore, existing technologies are relatively simplistic in their noise simulation, only mentioning noise superposition without detailing the noise types and simulation methods. In contrast, the embodiments of this invention not only add Gaussian noise and salt-and-pepper noise but also introduce Poisson noise, providing a more comprehensive simulation of the noise environment in space-based observations and improving the realism of the simulated images. Finally, the simulated images of existing technologies lack scalability, making it difficult to simulate multi-target dynamic interaction processes according to actual needs. However, the simulated images of the embodiments of this invention are scalable, allowing for subsequent simulations of multi-target dynamic interaction processes based on actual requirements, simulating the relative motion between multiple satellites, collision risk assessment, and collaborative operations.

[0108] Figure 8 The diagram illustrates the structure of a spatial target sequence image generation device for deep space background provided in an embodiment of the present invention.

[0109] like Figure 8 As shown, the space target sequence image generation device 800 for deep space background in this embodiment includes a data parsing module 801, a star screening module 802, a background generation module 803, a target simulation module 804, and an image synthesis module 805.

[0110] The data parsing module 801 is used to obtain the pose data of a space target during the observation period through simulation based on predefined initial parameters.

[0111] The star selection module 802 is used to select imageable stars from the reference star library based on the field of view and detection capabilities of the on-orbit optical camera.

[0112] Background generation module 803 is used to generate a deep space background image that matches the field of view of the space target by means of brightness simulation and coordinate system transformation based on the position data of imageable stars.

[0113] The target simulation module 804 is used to generate time-series rendered images by using the three-dimensional model and pose data of the spatial target through the rendering engine.

[0114] The image synthesis module 805 is used to overlay deep space background images and time-series rendered images according to the foreground and background distribution, and add noise to obtain simulated sequence images.

[0115] It is understood that the data parsing module 801, star screening module 802, background generation module 803, target simulation module 804, and image synthesis module 805 can be implemented in one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the data parsing module 801, star screening module 802, background generation module 803, target simulation module 804, and image synthesis module 805 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable method of integrating or packaging circuitry, or as hardware or firmware implementation, or as a suitable combination of software, hardware, and firmware implementations. Alternatively, at least one of the data parsing module 801, star screening module 802, background generation module 803, target simulation module 804, and image synthesis module 805 can be at least partially implemented as a computer program module, which can perform the functions of the corresponding module when the program is run by a computer.

[0116] Figure 9 The diagram illustrates the hardware structure of an electronic device provided in an embodiment of the present invention.

[0117] like Figure 9As shown, an electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0118] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 902 and / or RAM 903. It should be noted that programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0119] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0120] The present invention also provides a computer-readable medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0121] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0122] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating spatial target sequence images against a deep space background, characterized in that, include: Based on predefined initial parameters, the pose data of the space target during the observation period are obtained through simulation. Based on the field of view and detection capabilities of the on-orbit optical camera, imageable stars are selected from the reference star library; When performing the imageable star screening, multiple moments are selected from the observation period; Based on the pose data of the space target at different times, the on-orbit optical camera is simulated at multiple times and from multiple perspectives to obtain the position data of the imageable star at the corresponding time. Based on the position data of the imageable star, a coordinate system is constructed and a coordinate system transformation is performed to obtain the position coordinates of the imageable star on the two-dimensional imaging plane; the brightness of the imageable star is simulated using a Gaussian diffusion model, and combined with the position coordinates, a deep space background image matching the observation field of view of the space target is obtained. Using the 3D model of the spatial target and the pose data, a temporal rendering image is generated through a rendering engine; The deep space background image and the time-series rendered image are superimposed according to the foreground and background distribution, and noise is added to obtain a simulation sequence image.

2. The method according to claim 1, characterized in that, The step of generating a time-series rendered image using a rendering engine based on the 3D model of the spatial target and the pose data includes: The three-dimensional model of the space target is imported into the rendering engine, and the position and attitude of the on-orbit optical camera are determined. Calculate the bounding box of each space target and its components in the view of the on-orbit optical camera, convert the coordinates of the bounding box into target format, and save it to the corresponding component label file; Based on the pose data of the spatial target in the current frame, update the position and attitude of the spatial target and the on-orbit optical camera, and render the scene to obtain a time-series rendered image and the corresponding component label file.

3. The method according to claim 1, characterized in that, The step of superimposing the deep space background image and the time-series rendered image according to the foreground and background distribution, and adding noise to obtain the simulation sequence image includes: A resampling algorithm is used to scale the deep space background image and the time-series rendered image. The scaled deep space background image and the time-series rendered image are semi-transparently overlaid to obtain an overlaid image; The superimposed images are subjected to coordinate mapping and label normalization, and noise is added to the processed superimposed images to obtain the simulation sequence images.

4. The method according to claim 1, characterized in that, The method further includes: The target data is extracted from the original star catalog to obtain the reference star library.

5. The method according to claim 1, characterized in that, The initial parameters include time parameters, orbital parameters, and sensor parameters.

6. A device for generating spatial target sequence images against a deep space background, characterized in that, include: The data parsing module is used to obtain the pose data of a space target during the observation period through simulation based on predefined initial parameters. The star selection module is used to select imageable stars from the reference star library based on the field of view and detection capabilities of the on-orbit optical camera. The background generation module is used to select multiple moments from the observation period when screening imageable stars; and to simulate the on-orbit optical camera at multiple moments and from multiple perspectives based on the pose data of the space target at each different moment to obtain the position data of the imageable stars at the corresponding moment. Based on the position data of the imageable star, a coordinate system is constructed and a coordinate system transformation is performed to obtain the position coordinates of the imageable star on the two-dimensional imaging plane; the brightness of the imageable star is simulated using a Gaussian diffusion model, and combined with the position coordinates, a deep space background image matching the observation field of view of the space target is obtained. The target simulation module is used to generate time-series rendered images using the three-dimensional model of the spatial target and the pose data through the rendering engine. The image synthesis module is used to superimpose the deep space background image and the time-series rendered image according to the foreground and background distribution, and add noise to obtain a simulated sequence image.

7. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements each step of the spatial target sequence image generation method for deep space background as described in any one of claims 1 to 5.

Images