Space target sequence image generation method and device facing deep space background

Abstract

The invention provides a deep space background-oriented space target sequence image generation method and device, electronic equipment and a medium, and relates to the field of space target optical image simulation. The method comprises the steps of obtaining pose data of a space target in an observation time period through simulation based on pre-defined initial parameters; screening imageable fixed stars from the reference star library according to the field angle and the detectability of the on-orbit optical camera; based on the position data of the imageable fixed star, generating a deep space background image matched with the observation view field of the space target through brightness simulation and coordinate system transformation; generating a time sequence rendering image through a rendering engine by using the three-dimensional model and the pose data of the space target; and superposing the deep space background image and the time sequence rendering image according to foreground and background distribution, and adding noise to obtain a simulation sequence image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of space target optical image simulation, in particular to a space target sequence image generation method and device for deep space background, electronic equipment and medium. BACKGROUND

[0002] Space target recognition technology is a core means to ensure the safety of space activities, and its importance is constantly increasing with the increasing frequency of space activities. The key tasks of this technology include rapid detection, feature extraction and fine recognition of on-orbit targets, which require accurate acquisition of feature parameters such as target size, configuration and attitude. These feature parameters not only provide key support for on-orbit maintenance of failed satellites, space target recovery and other tasks, but also effectively prevent potential collisions, improve the utilization rate of orbital resources, and provide a safe space environment.

[0003] The space monitoring system mainly includes a space-based monitoring system and a ground-based monitoring system, and the payload of the space-based monitoring system mainly works in the visible light band. Due to the particularity of space targets and the sensitivity of space regions, it is difficult to obtain real images, and the number of samples is small. Therefore, when recognizing space targets, there is a lack of sufficient number of labeled samples for training the classifier. SUMMARY

[0004] (I) Technical problems to be solved

[0005] In view of the above problems, the present application provides a space target sequence image generation method and device for deep space background, equipment and medium, to at least partially solve the technical problems of insufficient dynamicity of simulated targets and background, high simulation complexity, and lack of target fine-grained labels for algorithm training.

[0006] (II) Technical solutions

[0007] The present application provides a space target sequence image generation method for deep space background, comprising: based on the initial parameters defined in advance, obtaining the pose data of the space target in the observation period through simulation; selecting the imageable stars from the reference star library according to the field of view angle and detection capability of the on-orbit optical camera; based on the position data of the imageable stars, generating a deep space background image matching the observation field of view of the space target through brightness simulation and coordinate system transformation; using the three-dimensional model and pose data of the space target, generating a time sequence rendering image through a rendering engine; superimposing the deep space background image and the time sequence rendering image according to the foreground and background distribution, and adding noise to obtain a simulation sequence image.

[0008] According to an embodiment of the present application, the method further comprises: selecting multiple time instants from the observation period when performing the imageable star screening; and performing multi-time and multi-view simulation on the in-orbit optical camera according to the pose data of the space target at each time instant to obtain position data of the imageable star at the corresponding time instant.

[0009] According to an embodiment of the present application, the deep space background image matching the observation field of view of the space target is generated by luminance simulation and coordinate system transformation based on the position data of the imageable star, comprising: constructing a coordinate system and performing coordinate system transformation based on the position data of the imageable star to obtain position coordinates of the imageable star on a two-dimensional imaging plane; and performing luminance simulation of the imageable star by a Gaussian diffusion model and combining the position coordinates to obtain the deep space background image.

[0010] According to an embodiment of the present application, the time sequence rendering image is generated by a rendering engine using the three-dimensional model and the pose data of the space target, comprising: importing the three-dimensional model of the space target into the rendering engine and determining the position and pose of the in-orbit optical camera; calculating the bounding box of each space target and its important components in the view of the in-orbit optical camera, converting the coordinates of the bounding box into a target format, and saving to a corresponding component label file; updating the position and pose of the space target and the in-orbit optical camera according to the pose data of the space target of the current frame, and rendering the scene to obtain the time sequence rendering image and the component label file corresponding to the image.

[0011] According to an embodiment of the present application, the deep space background image and the time sequence rendering image are superimposed according to the foreground and background distribution, and noise is added to obtain a simulation sequence image, comprising: using a resampling algorithm to perform image scaling on the deep space background image and the time sequence rendering image; performing semi-transparent superimposition on the scaled deep space background image and the time sequence rendering image to obtain a superimposed image; performing coordinate mapping and label normalization processing on the superimposed image, and adding noise to the processed superimposed image to obtain the simulation sequence image.

[0012] According to an embodiment of the present application, the method further comprises: extracting target data in the original star catalog to obtain a reference star library.

[0013] According to an embodiment of the present application, the initial parameters include time parameters, orbit parameters and sensor parameters.

[0014] The second aspect of the present application provides a space target sequence image generation device for a deep space background, comprising: a data analysis module, configured to obtain pose data of a space target in an observation period through simulation based on predefined initial parameters; a star screening module, configured to screen imageable stars from a reference star library according to a field of view angle and a detection capability of an on-orbit optical camera; a background generation module, configured to generate a deep space background image matching an observation field of view of the space target through brightness simulation and coordinate system transformation based on position data of the imageable stars; a target simulation module, configured to generate a time sequence rendering image through a rendering engine by using a three-dimensional model and the pose data of the space target; and an image synthesis module, configured to superimpose the deep space background image and the time sequence rendering image according to a foreground and background distribution, and add noise to obtain a simulation sequence image.

[0015] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0016] The fourth aspect of the present application provides a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement each step of the space target sequence image generation method for a deep space background.

[0017] (Three) beneficial effects

[0018] 1. The space target sequence image generation method for a deep space background provided by the present application simulates the space target optical imaging process based on space-based observation, and can provide data support for the development and verification of various algorithms such as ground-based space target classification and fine-grained detection.

[0019] 2. A real-time updating mechanism is established to simulate the dynamic change process of the target and the background by changing the camera position, satellite attitude and star background, and generate the simulation sequence image of the space target.

[0020] 3. An automatic generation process of the space target simulation sequence image and the component label file is constructed, which can realize batch automatic rendering of the image and synchronous generation of the component-level label, and has higher efficiency and accuracy than manual annotation.

[0021] 4. The simulation sequence image has scalability, and subsequent simulation of the multi-target dynamic interaction process can be performed according to actual needs, and the relative motion between multiple satellites, collision risk assessment and cooperative operation can be simulated according to orbit information. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more completely understand the present application and its advantages, reference will now be made to the following description taken together with the accompanying drawings, in which:

[0023] Figure 1 A flow chart of a method for generating a space target sequence image facing a deep space background is shown schematically;

[0024] Figure 2 A general flow chart of a method for generating a space target sequence image facing a deep space background is shown schematically;

[0025] Figure 3 Deep space background images at different time are shown schematically;

[0026] Figure 4 Space target rendering images at different time are shown schematically;

[0027] Figure 5 An image generated by superimposing a deep space background image and a space target rendering image is shown schematically;

[0028] Figure 6 A simulated sequence image after adding noise is shown schematically;

[0029] Figure 7 A space target component label visualization image is shown schematically;

[0030] Figure 8 A structure block diagram of a device for generating a space target sequence image facing a deep space background is shown schematically;

[0031] Figure 9 A hardware structure diagram of an electronic device is shown schematically. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, the drawings are designed for a description only, and are not intended to limit the scope of the present application. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to one skilled in the art that one or more embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and functions have been omitted to avoid obscuring the present application.

[0033] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the present application. The terms "include", "comprise" and the like used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0034] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art unless otherwise defined. It should be further understood that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification, and should not be interpreted in an overly idealized or overly formal sense.

[0035] Some of the diagrams illustrated in the drawings are block diagrams and / or flowcharts. It should be understood that some of the blocks in the block diagrams and / or flowcharts, or combinations thereof, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that these instructions executed by the processor can create a means for implementing the functions / operations described in the block diagrams and / or flowcharts.

[0036] It is found through research that the current space target simulation mainly faces problems such as insufficient dynamicity of simulation targets and background, high simulation complexity, and lack of target fine-grained labels for algorithm training. In the related art, a high-orbit space small target data set is constructed by simulating different signal-to-noise ratios, clutter environments, and target gray scale values. However, this process simulates space targets and stars in the form of simple long rectangles and circles, and the positions of the stars are randomly determined, which still has a gap with real shooting images. In addition, there is a technology that generates a rich data set through star background modeling, space target imaging modeling, noise and stray light simulation, but it mainly focuses on the simulation of static images, and lacks simulation of dynamic changes of targets and backgrounds. Some related technologies generate space target simulation images in a complex space background containing the Earth, the Moon, the Sun, and stars through light tracing, star table calculation, projection coordinate transformation, stray light empirical formula, and texture mapping technologies, but also lack simulation of dynamic changes of targets and backgrounds. Dynamic digital images are generated through target imaging model design, target digital library construction, star background mapping, illumination background modeling, noise superposition, and comprehensive motion simulation, to simulate the imaging effect of a space-based observation platform in a real space scene. However, the simulation process involves multiple steps such as model design and digital library construction, and has high complexity, which may result in low simulation efficiency. The imaging process of an in-orbit optical camera can be simulated through computer vision simulation technology and image processing technology to generate simulation images with component-level labels, providing data support for ground-based development and verification of space target recognition algorithms. In view of the above problems, an embodiment of the present application provides a space target sequence image generation method for a deep space background.

[0037] As shown in Figure 1 The flowchart of the space target sequence image generation method for a deep space background includes operations S1-S5.

[0038] In operation S1, based on predefined initial parameters, the pose data of space targets in an observation period is obtained through simulation.

[0039] In operation S2, the observable stars are screened from a reference star library according to the field of view and detection capability of the on-orbit optical camera.

[0040] In operation S3, based on the position data of the observable stars, a deep space background image matching the observation field of view of the space target is generated through brightness simulation and coordinate system transformation.

[0041] In operation S4, a time sequence rendering image is generated through a rendering engine using the three-dimensional model and pose data of the space target.

[0042] In operation S5, the deep space background image and the time sequence rendering image are superimposed according to the foreground and background distribution, and noise is added to obtain a simulation sequence image.

[0043] In some example embodiments, the simulation object is mainly an on-orbit satellite, and the simulation is performed for a close-range space observation process. The data set is constructed by simulating the space target, star background, clutter environment, and target gray value, etc., for training, testing, and verification of machine learning algorithms or deep learning algorithms.

[0044] The space target sequence image generation method for deep space background provided by the embodiment of the application can first construct an observation scene and a coordinate system. During space observation, the deep space background is mainly passively detected, and the background is relatively simple. During measurement, the optical system is used to collect the spectral energy reflected by the target, project it onto the focal plane of the detector, and obtain an observation image sequence through continuous shooting. The elements in the observation scene include a space target, an observation satellite, an on-orbit optical camera (carried on the observation satellite), and an observation background, etc. According to the observation elements, an earth-centered inertial coordinate system (such as J2000 coordinate system), a satellite body coordinate system, a camera coordinate system, and an imaging plane coordinate system can be constructed.

[0045] Specifically, a simulation scene can be created by using interdisciplinary engineering software to model, analyze, and simulate complex systems. In the simulation software, appropriate initial parameters can be set, including but not limited to time parameters, orbit parameters, and sensor parameters, etc., and the tracking mode of the sensor and the observation satellite can be set. The time parameters can include the simulation start time, the simulation end time, and the simulation step length, etc. Then, the orbit parameter setting of the observation satellite and the observed satellite (i.e., the space target) can be completed by importing a two-line orbit element (TLE) file. The TLE file can contain parameters such as the semi-major axis, eccentricity, orbit inclination, ascending node right ascension, argument of perigee, and mean anomaly of the satellite. Then, the sensor (such as the on-orbit optical camera) can be added to the observation satellite, and the type, resolution, field of view, and pointing angle of the sensor can be set, and the tracking mode of the sensor and the observation satellite can be set.

[0046] Figure 2 An overall flowchart of a space target sequence image generation method provided by an embodiment of the present application is shown.

[0047] As shown in Figure 2 , the space target sequence image generation method can include three modules of sequence star background generation, target simulation, and foreground and background superposition. An initial star catalog, parsed data, and a three-dimensional model, etc. can be used as initial inputs. First, based on processed satellite position and attitude data, the imaged stars at the corresponding time are obtained and projected onto a two-dimensional plane through coordinate system transformation and a Gaussian diffusion model. Then, the position and attitude of the space target three-dimensional model in the rendering engine are controlled to obtain the rendering image of the target at the corresponding time. Finally, the star background image and the space target rendering image at the corresponding time are superimposed to obtain the final sequence image.

[0048] Further, as shown in Figure 2 , the position data and attitude data (i.e. pose data) of the observation satellite and the observed satellite at each simulation time can be accurately calculated based on the simulation platform and the set initial parameters. Specifically, based on the set initial parameters, the position data and attitude data of the observation satellite and the observed satellite within a simulation period can be obtained through simulation software. The position data can be output in the form of a Cartesian coordinate system, including the x, y, z coordinates of the satellite in the J2000 coordinate system and the velocity Vx, Vy, Vz of the satellite in the x, y, z direction. The attitude data can be output in the form of Euler angles or quaternions, including the yaw angle, pitch angle, and roll angle of the satellite relative to the J2000 coordinate system. After obtaining the reference star library, the stars that can be imaged on the camera focal plane can be selected based on the detection capability of the device and the camera field of view angle to obtain the preliminarily screened imaged stars.

[0049] In the process of screening the imaged stars, the maximum apparent magnitude that can be obtained can be determined by the detection capability of the detection instrument. The larger the apparent magnitude of the star, the lower the brightness, so the apparent magnitude of the star satisfies the following formula, and the star can be captured by the detection instrument.

[0050]

[0051] When the right ascension and declination of the star and the pointing direction of the observation camera satisfy the camera field of view angle range constraint, the star can be imaged on the camera imaging plane.

[0052]

[0053] wherein, and are the right ascension and declination of the star, and Observe the right ascension and declination of the camera optical axis at a certain moment; FOV is the field of view of the camera.

[0054] After two screenings, a number of stars that can be imaged at the focal plane of the camera at some time can be obtained. For the screened imageable stars, a deep space background image can be generated through brightness simulation and coordinate system transformation. Then, spatial target simulation is performed, i.e., based on the three-dimensional model and rendering engine of the satellite, combined with the position and attitude data of the satellite, a time sequence rendering image is obtained. Finally, the generated deep space background image and the simulation sequence image are superimposed according to the foreground and background distribution, and noise is added to the superimposed image to obtain the final space target simulation sequence image facing the deep space background.

[0055] It can be understood that the embodiment of the application generates a series of space target simulation sequence images with fine-grained labels by simulating the satellite and star background at different shooting moments, superimposes the foreground and background, and provides data support for the development and verification of ground space target classification algorithms and fine-grained detection algorithms.

[0056] In the embodiment of the application, the method further comprises: extracting target data in the original star catalog to obtain a reference star library.

[0057] Exemplarily, the SAO (Smithsonian Astrophysical Observatory) star catalog can be selected as the original star catalog. It contains data of 258997 stars, and the star information covers a wide range of sky areas, providing a rich data basis for the preparation of the reference star library.

[0058] Although the original star catalog contains a large amount of celestial body information, there is a large amount of redundant data in it. For example, celestial body parameters irrelevant to the research target, repeatedly recorded data, or descriptions unnecessary in detail for the current task, their existence will increase the complexity and computational amount of data processing, and reduce the efficiency of subsequent analysis. Therefore, the required effective information (target data) can be extracted from the original star catalog to obtain a sub-star catalog, i.e., a reference star library, which does not contain redundant data. Among them, the effective information can revolve around the star, such as star number, right ascension, declination and apparent magnitude, etc. The star number is an important identifier for identifying and distinguishing different stars, the right ascension and declination determine the position of the star in the sky, and the apparent magnitude reflects the brightness of the star.

[0059] It can be understood that the preparation of the reference star library provides an efficient and concise data source for star background simulation and related astronomical research.

[0060] On the basis of the above-mentioned embodiments, in this embodiment, the method further comprises: when performing the screening of the imageable star, selecting multiple time instants within the observation period; and performing multi-time and multi-view simulation on the in-orbit optical camera according to the pose data of the space target at each time instant, to obtain the position data of the imageable star at the corresponding time instant.

[0061] When performing the field of view screening of the in-orbit optical camera, multiple time instants can be selected within the observation period. In this process, the motion characteristics of the target celestial body, the satellite orbit characteristics, and the observation task requirements can be comprehensively considered. Then, based on the pose data of the space target at each time instant, i.e., the position (longitude, latitude, and altitude) and attitude (pitch angle, yaw angle, and roll angle) of the satellite, multi-time and multi-view simulation is performed on the in-orbit optical camera. In the simulation process, for each time instant and each view, the position data of the imageable star within the camera field of view can be recorded, such as the right ascension, declination, and apparent magnitude of the star. Finally, the obtained position data of the imageable star can be saved in a certain format, such as a database, a text file, or a binary file, while recording the corresponding time instant, satellite pose data, and simulation parameters, so as to accurately restore the simulation process and results when needed.

[0062] It can be understood that through analysis of the multi-time and multi-view simulation results, the observation capability of the in-orbit optical camera at different times and views can be understood, and the observation blind area and the area with poor imaging quality can be found, so as to optimize the observation strategy of the camera.

[0063] On the basis of the above-mentioned embodiments, in this embodiment, based on the position data of the imageable star, the deep space background image matching the observation field of view of the space target is generated through brightness simulation and coordinate system transformation, which comprises: based on the position data of the imageable star, a coordinate system is constructed and coordinate system transformation is performed to obtain the position coordinates of the imageable star on the two-dimensional imaging plane; and through a Gaussian diffusion model, the brightness simulation of the imageable star is performed, and combined with the position coordinates, the deep space background image is obtained.

[0064] Based on the screened imageable star, the position coordinates of the star on the two-dimensional imaging plane can be obtained by constructing a coordinate system and performing coordinate system transformation. Then, the brightness simulation of the star can be performed through a Gaussian diffusion model, and combined with the position coordinates, the deep space background image is obtained.

[0065] Specifically, based on the right ascension, declination, and apparent magnitude data of the screened star, the coordinate calculation and gray value calculation of the star are performed.

[0066] The unit vector of the star in the J2000 coordinate system can be expressed as:

[0067]

[0068] Wherein, U star , V star and W star are the unit vector components of the star in the J2000 coordinate system.

[0069] Through the coordinate axis rotation and translation transformation, the coordinates of the star in the satellite body coordinate system are obtained as follows:

[0070]

[0071] Wherein, R1 is the rotation matrix from the J2000 coordinate system to the satellite body coordinate system; X star , Y star and Z star are the coordinates of the star in the satellite body coordinate system; U sat , V sat and W sat are the position vector components (translation part) of the satellite in the J2000 coordinate system.

[0072] The rotation matrix is calculated in the form of Euler angles. When the coordinate system rotates around the X, Y and Z axes, the corresponding basic rotation matrices are as follows:

[0073]

[0074]

[0075]

[0076] Wherein, is the roll angle, is the pitch angle, is the yaw angle.

[0077] Euler angles describe the rotation process of the coordinate system based on the rotation matrix. The rotation order of the rotation matrix is divided into external rotation (x, y, z) and internal rotation (z, y, x). The rotation matrix corresponding to the internal rotation is as follows:

[0078]

[0079] The conversion from the satellite body coordinate system to the camera coordinate system is as follows:

[0080]

[0081] Wherein, X, Y and Z are the coordinates of the star in the camera coordinate system; R2 is the rotation matrix from the satellite body coordinate system to the camera coordinate system.

[0082] The distance from the satellite body coordinate system to the camera coordinate system is relatively short. When performing coordinate system conversion, the error caused by coordinate system translation can be ignored, and only coordinate system rotation transformation is performed.

[0083] The imaging position of the target is calculated using a pinhole imaging model during imaging, and the coordinates of the star in the imaging plane coordinate system are:

[0084]

[0085] where N X and N y are the camera resolution, and FOV is the camera field of view.

[0086] Star brightness simulation can be achieved through a Gaussian diffusion model, in which the combination of the number of pixels (diffusion range) and the pixel brightness (amplitude of the Gaussian kernel) jointly determines the simulation effect of a single star. Specifically, a reference brightness value is returned according to the magnitude, then a two-dimensional Gaussian diffusion is applied at the specified position to simulate the brightness distribution of the star, and finally the brightness value is scaled to the 8-bit grayscale range of [0, 255] through normalization processing.

[0087] The brightness is determined by the amplitude of the Gaussian kernel and the magnitude through a function mapping relationship, and the specific relationship is shown in Table 1.

[0088] Table 1. Magnitude and reference brightness mapping table

[0089]

[0090] For each pixel point (x, y) in the image, the distance from the center point of the star is calculated, and the brightness contribution of the point is calculated using the Gaussian formula. The brightness contribution g is given by:

[0091]

[0092] where A is the reference brightness value of the star, is the standard deviation of the Gaussian function, and (x0, y0) is the center point of the star.

[0093] Through the calculation of the coordinates of the star in the two-dimensional imaging plane and the calculation of the gray scale, the star background simulation images at different times, i.e., deep space background images, can be obtained, as shown in Figure 3 Fig. 5. (a)-(e) are five star background images at different times. That is, based on the position and attitude data output by the satellite tool kit software and the right ascension and declination data in the star catalog, the star background at different times can be obtained. When generating the star background, 6s can be set as one frame, and each image can correspond to a star background at a time.

[0094] On the basis of the above-mentioned embodiments, in this embodiment, by using the three-dimensional model and the pose data of the space target, the time sequence rendering image generated by the rendering engine includes: importing the three-dimensional model of the space target into the rendering engine, and determining the position and attitude of the on-orbit optical camera; calculating the bounding box of each space target and its important component in the view of the on-orbit optical camera, converting the coordinates of the bounding box into a target format, and saving it to the corresponding component label file; according to the pose data of the space target of the current frame, updating the position and attitude of the space target and the on-orbit optical camera, and rendering the scene to obtain the time sequence rendering image and the component label file corresponding to the image.

[0095] The three-dimensional model of the satellite is imported into the rendering engine, and the time sequence position and attitude data in the coordinate system of the simulation software are converted into the right-hand coordinate system. Then, the position of the camera can be set according to the position and attitude data of the observation satellite. The orientation of the camera can be calculated by the position of the target satellite, ensuring that the camera always points to the target satellite. The position of the light source is randomly generated near the camera. The intensity of the light source is randomly set, and in addition, the color and intensity of the ambient light can also be set. By calculating the bounding box of each object in the camera view, the bounding box coordinates are converted into a target format (such as YOLO format), i.e. the normalized values of the center point coordinates, width and height, and saved to the corresponding component label file. In each frame, the position and attitude of the target satellite and the camera can be updated according to the satellite data of the current frame, and then the scene is rendered and the image is saved to obtain a space target rendering image example, as shown in Figure 4 (a)-(e) are five space target rendering images at different times generated by using three-dimensional graphics software. When rendering the image, the attitude and position data of the camera and the target can be adjusted every 6s, and a corresponding rendering image is generated every 6s. The position, attitude and lighting conditions of the target in each rendering image will change.

[0096] Through the above operation, a real shooting scene can be simulated, and a plurality of satellite target images at different times (i.e. time sequence rendering images), image category labels and corresponding component level labels (stored in the component label file) can be obtained.

[0097] On the basis of the above-mentioned embodiments, in this embodiment, the deep space background image and the time sequence rendering image are superimposed according to the foreground and background distribution, and noise is added to obtain a simulation sequence image, which includes: using a resampling algorithm to scale the deep space background image and the time sequence rendering image; semi-transparently superimposing the scaled deep space background image and the time sequence rendering image to obtain a superimposed image; performing coordinate mapping and label normalization processing on the superimposed image, and adding noise to the processed superimposed image to obtain a simulation sequence image.

[0098] The stellar background is taken as a background image, and the space target simulation image is taken as a foreground image, and the final space target simulation image is obtained by superimposing the foreground and the background.

[0099] Exemplarily, first, the LANCZOS resampling algorithm can be used for image scaling, which realizes smooth resampling by calculating the pixel weighted average value based on the sinc function convolution kernel, and can effectively retain high-frequency details and reduce the sawtooth effect when reducing the image. The core of the algorithm is the Lanczos kernel, which is defined as:

[0100]

[0101] Wherein, x is the relative distance between the current pixel position and the target pixel position; L(x) is the weight value of the Lanczos kernel at position x; is a normalized sine function, and a is the radius of the Lanczos window.

[0102] Secondly, the Alpha blending algorithm is used to realize semi-transparent superposition, the transparency weight of the foreground and the background is separated, linear interpolation calculation is performed on each RGB channel, the texture of the stellar background is retained while the contour features of the space target are highlighted, and thus a superposition image, i.e. a space target sequence image, as shown in Figure 5 is obtained. Specifically, according to the process in Figure 2 , the stellar background and the space target rendering image at the corresponding moment can be superimposed, i.e. Figure 3 and Figure 4 are superimposed in a semi-transparent superposition manner. When generating an image, 6s can be a frame, and the finally obtained image is consistent with the generated image, also 6s a frame.

[0103] Finally, through accurate coordinate mapping and label normalization processing, the consistency of visual quality and labeled data is realized.

[0104] As shown in Figure 6 , in order to simulate the real space environment and the shooting process, noise is added to the space target sequence image, i.e. different types of noise are added to the superposition image to simulate the noise environment in space observation. The main added noises include Gaussian noise, salt and pepper noise and Poisson noise. Gaussian noise is used to simulate the random noise commonly seen in space observation, such as sensor noise. Salt and pepper noise simulates random pixel faults that may occur in space observation, such as sensor thermal noise or cosmic ray influence. Poisson noise simulates the noise in the photon counting process, which is proportional to the signal strength.

[0105] Figure 7 The space target component label visualization image provided by the embodiment of the application is schematically shown.

[0106] AsFigure 7 As shown, the space target component label visualization image can show each component of the space target and its corresponding label information, so that researchers or operators can quickly identify and understand the composition structure of the space target.

[0107] Although the prior art generates a rich dataset, it mainly focuses on the simulation of static images and lacks simulation of dynamic changes of targets and backgrounds, but the space target sequence image generation method for deep space background provided by the embodiment of the present application can simulate the dynamic change process of the target and the background through the simulation and real-time updating mechanism, generate a series of space target time sequence optical images, and be closer to the real space observation scene. At the same time, the simulation process of the prior art has high complexity, involves target imaging model design, target database construction and other steps, resulting in low simulation efficiency. The embodiment of the present application simplifies the simulation process, directly uses the simulation software analysis and rendering engine, reduces the complex steps, improves the simulation efficiency, and at the same time ensures the quality of the simulation image. In addition, the prior art lacks the generation of target fine-grained labels, and only simulates space targets through simple geometric shapes, lacking component-level labels. However, the embodiment of the present application can convert the bounding box of each object in the camera view into a target format to generate a simulation image with component-level labels, providing more abundant training data for machine learning and deep learning algorithms. In addition, the prior art is relatively single in noise simulation, only mentioning noise superposition, but not detailing the noise type and simulation method, while the embodiment of the present application not only adds Gaussian noise and salt and pepper noise, but also introduces Poisson noise, more comprehensively simulating the noise environment in space observation, improving the authenticity of the simulation image. Finally, the simulation image of the prior art lacks scalability and is difficult to simulate the dynamic interaction process of multiple targets according to actual needs. However, the simulation image of the embodiment of the present application has scalability, and subsequent simulation of the dynamic interaction process of multiple targets can simulate the relative motion between multiple satellites, collision risk assessment and cooperative operation.

[0108] Figure 8 The structure block diagram of the space target sequence image generation device for deep space background provided by the embodiment of the present application is schematically shown.

[0109] As Figure 8 shown, the space target sequence image generation device for deep space background 800 of this embodiment includes a data analysis module 801, a star screening module 802, a background generation module 803, a target simulation module 804 and an image synthesis module 805.

[0110] The data analysis module 801 is used to obtain the pose data of the space target in the observation period through simulation based on the pre-defined initial parameters.

[0111] The star selection module 802 is used to select imageable stars from the reference star library based on the field of view and detection capabilities of the on-orbit optical camera.

[0112] Background generation module 803 is used to generate a deep space background image that matches the field of view of the space target by means of brightness simulation and coordinate system transformation based on the position data of imageable stars.

[0113] The target simulation module 804 is used to generate time-series rendered images by using the three-dimensional model and pose data of the spatial target through the rendering engine.

[0114] The image synthesis module 805 is used to overlay deep space background images and time-series rendered images according to the foreground and background distribution, and add noise to obtain simulated sequence images.

[0115] It is understood that the data parsing module 801, star screening module 802, background generation module 803, target simulation module 804, and image synthesis module 805 can be implemented in one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the data parsing module 801, star screening module 802, background generation module 803, target simulation module 804, and image synthesis module 805 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable method of integrating or packaging circuitry, or as hardware or firmware implementation, or as a suitable combination of software, hardware, and firmware implementations. Alternatively, at least one of the data parsing module 801, star screening module 802, background generation module 803, target simulation module 804, and image synthesis module 805 can be at least partially implemented as a computer program module, which can perform the functions of the corresponding module when the program is run by a computer.

[0116] Figure 9 The diagram illustrates the hardware structure of an electronic device provided in an embodiment of the present invention.

[0117] like Figure 9As shown, an electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0118] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 902 and / or RAM 903. It should be noted that programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0119] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0120] The present invention also provides a computer-readable medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0121] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0122] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating spatial target sequence images against a deep space background, characterized in that, include: Based on predefined initial parameters, the pose data of the space target during the observation period are obtained through simulation. Based on the field of view and detection capabilities of the on-orbit optical camera, imageable stars are selected from the reference star library; Based on the position data of the imageable stars, a deep space background image matching the observation field of view of the space target is generated through brightness simulation and coordinate system transformation; Using the 3D model of the spatial target and the pose data, a temporal rendering image is generated through a rendering engine; The deep space background image and the time-series rendered image are superimposed according to the foreground and background distribution, and noise is added to obtain a simulation sequence image.

2. The method according to claim 1, characterized in that, The method further includes: When performing the imageable star screening, multiple moments are selected from the observation period; Based on the pose data of the space target at different times, the on-orbit optical camera is simulated at multiple times and from multiple perspectives to obtain the position data of the imageable star at the corresponding time.

3. The method according to claim 2, characterized in that, The process of generating a deep-space background image that matches the observation field of view of the space target based on the position data of the imageable star, through brightness simulation and coordinate system transformation, includes: Based on the position data of the imageable star, a coordinate system is constructed and a coordinate system transformation is performed to obtain the position coordinates of the imageable star on the two-dimensional imaging plane. The brightness of the imageable star is simulated using a Gaussian diffusion model, and the deep space background image is obtained by combining the position coordinates.

4. The method according to claim 1, characterized in that, The step of generating a time-series rendered image using a rendering engine based on the 3D model of the spatial target and the pose data includes: The three-dimensional model of the space target is imported into the rendering engine, and the position and attitude of the on-orbit optical camera are determined. Calculate the bounding box of each space target and its key components in the view of the on-orbit optical camera, convert the coordinates of the bounding box into target format, and save it to the corresponding component label file; Based on the pose data of the spatial target in the current frame, update the position and attitude of the spatial target and the on-orbit optical camera, and render the scene to obtain a time-series rendered image and the corresponding component label file.

5. The method according to claim 1, characterized in that, The step of superimposing the deep space background image and the time-series rendered image according to the foreground and background distribution, and adding noise to obtain the simulation sequence image includes: A resampling algorithm is used to scale the deep space background image and the time-series rendered image. The scaled deep space background image and the time-series rendered image are semi-transparently overlaid to obtain an overlaid image; The superimposed images are subjected to coordinate mapping and label normalization, and noise is added to the processed superimposed images to obtain the simulation sequence images.

6. The method according to claim 1, characterized in that, The method further includes: The target data is extracted from the original star catalog to obtain the reference star library.

7. The method according to claim 1, characterized in that, The initial parameters include time parameters, orbital parameters, and sensor parameters.

8. A device for generating spatial target sequence images against a deep space background, characterized in that, include: The data parsing module is used to obtain the pose data of a space target during the observation period through simulation based on predefined initial parameters. The star selection module is used to select imageable stars from the reference star library based on the field of view and detection capabilities of the on-orbit optical camera. The background generation module is used to generate a deep space background image that matches the observation field of view of the space target based on the position data of the imageable star through brightness simulation and coordinate system transformation. The target simulation module is used to generate time-series rendered images using the three-dimensional model of the spatial target and the pose data through the rendering engine. The image synthesis module is used to superimpose the deep space background image and the time-series rendered image according to the foreground and background distribution, and add noise to obtain a simulated sequence image.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements each step of the spatial target sequence image generation method for deep space background as described in any one of claims 1 to 7.

Images