A method and system for real-time simulation of optical imagery of a non-cooperative target for close-in operations

Abstract

The application discloses a kind of real-time simulation method and system of optical image of empty target suitable for close operation, the method includes real-time receiving by track or attitude generator generated target and observation star orbit data;Camera optical axis direction is calculated;In combination with the limit detection capability of camera, the star that can be detected by camera is filtered out in SAO star table, complete star simulation;According to data, the relative pose of observation star and target is calculated;According to the characteristics of space target, the imaging simulation of target is carried out;According to whether the moon appears in the field of view, and the moon in the field of view is simulated;Add noise and stray light in image;It is output by UDP transmission mode through Ethernet port.The application can simulate a variety of space targets, and considers the relative change of space illumination, calculates real-time high efficiency, has fast image generation and transmission capacity, can provide more realistic target image input for ground semi-physical performance verification of space visual navigation system.

Description

Technical Field

[0001] This invention belongs to the field of satellite technology, specifically relating to a real-time simulation method and system for optical images of airborne targets suitable for close-range operations. Background Technology

[0002] In the fields of aerospace technology and space exploration, the observation, identification, and analysis of space targets are of paramount importance. Ground stations play a crucial role in acquiring image data of space targets, which provides fundamental information for various applications such as orbit determination, shape analysis, attitude estimation, and fault diagnosis. With the increasing frequency of space activities, the demand for high-precision, high-fidelity space target imaging simulation technology is becoming increasingly urgent.

[0003] Traditional space target imaging relies primarily on actual optical observation equipment. However, these devices are limited by various factors, such as weather conditions, observation time windows, equipment costs, and maintenance. Furthermore, real-world observations often struggle to flexibly adjust parameters and reproduce scenes under various preset conditions, posing significant challenges to algorithm development, system testing, and the verification of new imaging technologies. While some current software-based imaging simulation methods can simulate the imaging process of space targets to some extent, most do not consider integration with ground-based semi-physical co-simulation environments. These purely software simulations lack the ability to interact with actual hardware systems (such as sensor models and signal processing hardware), resulting in significantly reduced accuracy and effectiveness of simulation results in real-world applications. When applying simulation results to optimize actual equipment and verify new algorithms, substantial deviations occur, failing to accurately reflect the imaging effects under real-world semi-physical co-simulation conditions. Therefore, a real-time simulation method for visible light images of space targets for ground-based semi-physical verification of satellite visual navigation systems is needed, addressing the technical challenges of obtaining real-time on-orbit images of targets and the difficulty of recreating complex space environments and target states on-orbit using optical cameras on the ground. Summary of the Invention

[0004] Purpose of the invention: This invention provides a real-time simulation method and system for optical images of airborne targets suitable for close-range operations, forming a stable and continuous image output throughout the process, achieving the real-time performance and dynamism required for co-simulation.

[0005] Technical solution: The present invention provides a real-time simulation method for optical images of airborne targets suitable for close-range operations, specifically including the following steps:

[0006] S1. Receive real-time data on the orbital motion of the target and the observed star generated by the orbit or attitude generator;

[0007] S2. Calculate the camera optical axis pointing based on the received motion trajectory data;

[0008] S3. Based on the calculated camera optical axis direction and combined with the camera's ultimate detection capability, select stars that can be detected by the camera from the SAO star catalog to complete the star simulation.

[0009] S4. Calculate the relative pose of the observed satellite and the target based on the data received in S1;

[0010] S5. Based on the characteristics of the space target, perform imaging simulation of the target;

[0011] S6. Determine whether the moon will appear in the field of view based on the orbital motion data, and simulate the situation where the moon is in the field of view;

[0012] S7. Add noise and stray light to the image simulated in S5;

[0013] S8. Output the image after adding noise and stray light in S7 via Ethernet port in UDP transmission mode.

[0014] Furthermore, the target and observed star orbit data mentioned in step S1 are as follows:

[0015] Position and velocity vectors of the target in the ECI coordinate system, position and velocity vectors of the observed star in the ECI coordinate system, attitude quaternions of the observed star in the ECI coordinate system, position vector of the moon in the ECI coordinate system, and direction vector of sunlight in the ECI coordinate system.

[0016] Further, step S2 includes the following steps:

[0017] S201. Calculate the rotation matrix M1 from the ECI coordinate system to the observed star's body system using the quaternion of the observed star's body attitude in the ECI coordinate system from the received data.

[0018] S202. Define the camera mounting matrix M2, and calculate the rotation matrix M3 from the ECI coordinate system to the observation camera using the rotation matrix M1 and the camera mounting matrix M2.

[0019] Further, step S3 includes the following steps:

[0020] S301. Based on the camera's detection capabilities, select stars in the SAO catalog that are bright enough to be detected. The selection criteria are as follows:

[0021] Stars = {VM i ≤VM0,i=1,2,3...N}

[0022] Here, Stars is the collection of stars in the SAO catalog that can be detected within the camera's limit of detection range, VM iLet be the apparent magnitude of the i-th star in the SAO catalog, and VM0 be the apparent magnitude of the faintest star that can be detected by the camera's detection limit.

[0023] S302. Based on the right ascension and declination information of stars in the SAO catalog, calculate the azimuth vector (Vector) of the stars in the Stars list obtained in S301 in the camera coordinate system:

[0024]

[0025] Where Vector is the direction vector of the star in the camera coordinate system, and RA and RD are the right ascension and declination information of the star, respectively;

[0026] S303. Based on the Vector obtained in S302, further filter the Stars obtained in S301 to find Stars-In-Img that are within the camera's field of view. The filtering criteria are as follows:

[0027]

[0028] Among them, FOV x FOV is the field of view of the camera in the x-direction. y Camera field of view in the y-direction;

[0029] S304. Calculate the pixel coordinates of the stars (Stars-In-Img) selected in S303 within the image. The calculation formula is as follows:

[0030]

[0031] Among them, (N) x N y ) represents the pixel coordinates of the star in the image, f represents the focal length of the camera used, and d represents the focal length of the star. x With d y These are the pixel dimensions of the camera in the x and y directions, respectively;

[0032] S305. The simulated camera's detection capability range is from magnitude -1 to magnitude 5. The total gray value of the faintest star that the camera can detect in the image is set. The total grayscale value of stars of other magnitudes is:

[0033]

[0034] in, For star level is VM i The total grayscale value of the stars in the image. This represents the total grayscale value of a star with a magnitude of VM0 in the image.

[0035] S306. Using the star pixel coordinates obtained from S304 as the center, perform grayscale dispersion on the star points within a pixel range of 3×3 to 7×7. The dispersion formula is as follows:

[0036]

[0037] Where A is the total gray value of the star, δ is the Gaussian diffusion radius, (x0,y0) are the coordinates of the diffusion center, and I(x,y) is the gray value at pixel (x,y).

[0038] Further, step S4 includes the following steps:

[0039] S401, Target star position vector R based on ECI coordinate system Tar and the observed star position vector R Cha Find the relative positions of the target and the observation platform in the camera coordinate system;

[0040] S402, Based on the target star's position vector R under ECI Tar With velocity vector V Tar Calculate the rotation matrix M4 from the target system to the observed star system;

[0041] S403. Using the rotation matrix M4, calculate the target's attitude angles δ1, δ2, and δ3 relative to the observation camera to obtain the target's relative attitude.

[0042] Further, step S5 includes the following steps:

[0043] S501. Based on the three-dimensional geometric configuration of the target, select a certain number of feature points to reflect its own structural characteristics. Obtain the coordinates of these feature points in the target's own system according to the target size information, and convert the feature point coordinates in the target's own system into pixel coordinates in the camera coordinate system through the rotation matrix M4.

[0044] S502. Analyze the reflection characteristics of the target surface, and use different reflection models to describe the different materials used in different parts of the target.

[0045] S503. Treating the sun as a blackbody radiator, calculate the radiant exitance M within the effective wavelength range λ1~λ2 of a visible light camera. Calculate the radiant exitance of sunlight at a distance D from the sun using the inverse square law of distance. Tar-sun Spectral irradiance E at a distant target surface sun The target surface is divided into facets based on the feature points selected in S501, and the spectral irradiance dE of each facet is calculated. Tar ; Set the apparent magnitude VM of the Sun Sun The equivalent apparent magnitude VM of the surface element dA is calculated. dA-TarBased on the total gray value of stars with a magnitude of 5 Further calculation yields the total grayscale value I of the face element. dA-Tar ;

[0046] S504. Obtain the number of pixels occupied by the surface element based on the feature point pixel coordinates in S501, and then... dA-Tar The target is evenly divided into surface cells to obtain its grayscale value in the image.

[0047] Further, step S6 includes the following steps:

[0048] S601. Based on the lunar orbit data, calculate its position relative to the observed star in the camera coordinate system, determine whether the angle between the moon and the camera's line of sight in the x and y directions is greater than the camera's field of view angle, and whether the angle formed by the geocenter-camera-moon center is less than the angle formed by the geocenter-camera-camera and the point of tangency between the camera and the edge of the earth, and then determine the visibility of the moon.

[0049] S602. After determining that the moon is within the field of view, simulate the moon: Based on the relative positions of the moon, Earth, and the observed stars, and the change in the angle θ between the direction of sunlight (i.e., the vector perpendicular to the sun's rays) and the position vector of the moon relative to the observed stars, determine the method for calculating the illuminated area of ​​the moon in the simulated image. Calculate the imaged area S of the moon in the image using the appropriate method based on θ; further, calculate the number of pixels S it occupies in the image based on the pinhole camera imaging principle. Pic :

[0050]

[0051] Among them, S Pic R is the number of pixels the moon occupies in the image, S is the area of ​​the moon's image in the image, and R is the area of ​​the moon's image in the image. Cam-moon To observe the relative position of the camera and the moon;

[0052] S603. Based on the imaging area of ​​the moon in the image and its apparent magnitude in the full moon state, the apparent magnitude of the moon is calculated, and the total gray value and gray distribution of the moon are further calculated.

[0053] Further, step S7 includes the following steps:

[0054] S701. Determine the angle β between the observation platform and the tangent at the edge of the Earth's atmosphere and the camera's line of sight, and the angle FOV (1 / 2 of the camera's field of view). y The size relationship, if β < FOV y / 2 suggests that atmospheric radiation from the ground affects camera imaging;

[0055] S702. Earth's atmospheric radiation is mainly the reflection of sunlight. Calculate the solar irradiance E received by the Earth's surface.SC ;

[0056] S703. According to Lambert's cosine law, the Earth is considered as a Lambert radiator with constant radiant brightness in all directions. Its radiation intensity varies with the angle θ between the observation direction and the surface source normal, and follows the cosine law. Its surface is divided into countless surface elements, and the irradiance dE of each surface element at the entrance pupil of the detector camera is calculated. e-Cam ;

[0057] S704, Setting the Sun's Equivalent Apparent Magnitude (VM) Sun The equivalent apparent magnitude (VM) of the surface element at the entrance pupil of the detector camera was calculated based on the quantitative relationship between magnitude and irradiance. e Further, the apparent magnitude of the surface element is converted into grayscale values, and the total grayscale value and grayscale distribution of the Earth-atmosphere radiation are calculated.

[0058] The present invention provides a real-time simulation system for optical images of airborne targets suitable for close-range operations, comprising:

[0059] The RS422 serial port data receiving module is used to connect the target optical image simulator to the orbit or attitude generator and receive the motion data of the observed star and space target generated by it in real time.

[0060] The camera optical axis calculation module calculates the line-of-sight direction of the observation camera based on the data received from the RS422 serial port, including the rotation matrix from the ECI coordinate system to the observation star's local system and the rotation matrix from the ECI coordinate system to the observation camera.

[0061] The star simulation module, based on the calculated camera optical axis direction and the camera's ultimate detection capability, selects stars that can be detected by the camera from the SAO star catalog and performs simulations.

[0062] The relative pose calculation module uses the position and velocity vectors of the observed star and the target star in the ECI coordinate system from the received data to calculate the relative pose, including relative position and relative attitude.

[0063] The target simulation imaging module is used to calculate feature points that reflect the structural characteristics of the target based on its properties, and to perform imaging simulation of the target.

[0064] The Moon Determination and Simulation Imaging Module determines the visibility of the Moon by analyzing its orbital data, whether it is within the camera's field of view, and whether it is obscured by the Earth. If the Moon is determined to be within the field of view, the module calculates the Moon's pixel coordinates, the number of pixels it occupies, and the total grayscale value and grayscale distribution in the image.

[0065] The noise and stray light addition module calculates the irradiance of the Earth's surface from sunlight, the irradiance of each Earth surface element at the entrance pupil of the detector, the equivalent apparent magnitude of the element at the entrance pupil of the detector camera, and the total gray value and gray distribution of Earth's atmospheric radiation, based on the stray light imaging model.

[0066] The Ethernet port image transmission module outputs the simulated generated image via Ethernet port using UDP transmission.

[0067] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows:

[0068] This invention fully considers the real deep space environment, including noise and stray light. Compared with other existing technologies, it also considers the moon, which may appear in the field of view of the optical camera in actual deep space exploration, and simulates it, making the simulation scenario more realistic and credible.

[0069] This invention can simulate the entire process of a target moving from far to near during a mission. It can generate continuously updated simulated images based on the dynamic changes of the target (such as the trajectory and attitude quaternion information of the space target), achieving full coverage of medium- and long-range and short-range images, and forming a stable and coherent full-process image output.

[0070] This invention has strong data interaction capabilities with other components, enabling real-time data reception, real-time generation of the latest images and real-time transmission, and has a high speed, achieving the real-time and dynamic performance required for co-simulation.

[0071] This invention enables semi-physical verification on the ground, that is, simulating the motion image of a space target without actual cameras, satellite targets, or other equipment, and applying it to subsequent attitude and orbit control analysis, etc. This method solves the problem that optical cameras in traditional simulation technology are difficult to reproduce complex space environments on the ground. Attached Figure Description

[0072] Figure 1 A flowchart for a real-time simulation method of optical images of empty targets suitable for close-range operations;

[0073] Figure 2 A geometric diagram showing the relationship between the bright and dark areas of the moon in the camera image;

[0074] Figure 3 The following are the simulation results of the on-orbit visual scene of the space target under various observation platform distances according to the present invention. Among them, (a) is the distance between the target and the observation platform of 2.441km; (b) is the distance between the target and the observation platform of 0.167km; and (c) is the distance between the target and the observation platform of 0.019km. Detailed Implementation

[0075] The present invention will now be described in further detail with reference to the accompanying drawings.

[0076] This invention proposes a real-time simulation method for optical images of empty targets suitable for close-range operations, the specific implementation process of which is as follows:

[0077] The S1 and RS422 serial ports receive real-time target and observation satellite motion orbit data generated by the orbit or attitude generator.

[0078] The received data mainly includes the target's position and velocity vectors in the ECI coordinate system, the observed star's position and velocity vectors in the ECI coordinate system, the observed star's attitude quaternions in the ECI coordinate system, the moon's position vector in the ECI coordinate system, and the direction vector of sunlight in the ECI coordinate system.

[0079] S2. Calculate the line-of-sight direction of the observation camera based on the data received from the RS422 serial port.

[0080] S201. Calculate the rotation matrix M1 from the ECI coordinate system to the observed star's body system using the quaternions of the observed star's attitude in the received ECI coordinate system, as follows:

[0081]

[0082] Where Q(q0,q1,q2,q3) is the attitude quaternion of the observed star in the ECI coordinate system, and M1 is the rotation matrix from the ECI coordinate system to the observed star's body system.

[0083] S202. Define the camera mounting matrix M2 as follows:

[0084]

[0085] From the rotation matrix M1 from the ECI coordinate system obtained in S201 to the observation satellite's local system and the camera mounting matrix M2, the rotation matrix M3 from the ECI coordinate system to the observation camera can be calculated. Therefore, the optical axis orientation of the camera in the ECI coordinate system is known, as calculated below:

[0086] M3 = M2 × M1

[0087] Where M3 is the rotation matrix from the ECI coordinate system to the observation camera.

[0088] S3. Based on the calculated camera optical axis direction and the camera's ultimate detection capability, select stars that can be detected by the camera from the SAO star catalog to complete the star simulation.

[0089] S301. Based on the camera's detection capabilities, select stars in the SAO catalog that are bright enough to be detected. The selection criteria are as follows:

[0090] Stars = {VM i≤VM0,i=1,2,3...N}

[0091] Here, Stars is the collection of stars in the SAO catalog that can be detected within the camera's limit of detection range, VM i Let be the apparent magnitude of the i-th star in the SAO catalog, and VM0 be the apparent magnitude of the faintest star that can be detected by the camera's detection limit.

[0092] S302. Based on the right ascension and declination information of stars in the SAO catalog, calculate the azimuth vector (Vector) of the stars in the Stars list obtained in S301 in the camera coordinate system:

[0093]

[0094] Where Vector is the direction vector of the star in the camera coordinate system, and RA and RD are the right ascension and declination information of the star, respectively.

[0095] S303. Based on the Vector obtained in S302, further filter the Stars obtained in S301 to find Stars-In-Img that are within the camera's field of view. The filtering criteria are as follows:

[0096]

[0097] Among them, FOV x FOV is the field of view of the camera in the x-direction. y Camera field of view in the y-direction.

[0098] S304. Calculate the pixel coordinates of the stars selected in S303 (Stars-In-Img) in the image. The calculation formula is as follows:

[0099]

[0100] Among them, (N) x N y ) represents the pixel coordinates of the star in the image, f represents the focal length of the camera used, and d represents the focal length of the star. x With d y These are the pixel sizes of the camera in the x and y directions, respectively.

[0101] S305. The simulated camera's detection capability range is from magnitude -1 to magnitude 5. The total grayscale value A of the faintest magnitude star that the camera can detect in the image is set. VM0 If the value is 60, then the total gray value of stars of other magnitudes can be calculated using the following formula:

[0102]

[0103] in, For star level is VM i The total grayscale value of the stars in the image. This represents the total grayscale value of a star with magnitude VM0 in the image.

[0104] S306. Using the star pixel coordinates obtained from S304 as the center, perform grayscale dispersion on the star points within a pixel range of 3×3 to 7×7. The dispersion formula is as follows:

[0105]

[0106] Where A is the total gray value of the star, δ is the Gaussian diffusion radius, (x0,y0) are the coordinates of the diffusion center, and I(x,y) is the gray value at pixel (x,y). The diffusion pixel range depends on the star's magnitude, i.e., its total gray value in the image.

[0107] This invention is based on multiple experimental tests and statistical data analysis: to ensure that a star is not overwhelmed by a noisy background due to its low grayscale value, the grayscale value at the center of the star's brightness should not be lower than 30. Taking a 5th magnitude star as an example, since it is a relatively dim star, it is diffused into a 3×3 pixel area. Therefore, more than 95% of its star's energy is concentrated within this 3×3 area. According to the Gaussian distribution law, the probability of the numerical distribution falling within (μ-2σ, μ+2σ) is 95%, where μ is the pixel position of the star's brightness center and σ is the Gaussian diffusion radius δ. Since the star's brightness is diffused within a 3×3 pixel area, and the center of the pixel area is the star's brightness center, the Gaussian diffusion radius of the star's point should satisfy the following relationship:

[0108]

[0109] Where δ is the Gaussian dispersion radius.

[0110] Therefore, the Gaussian dispersion radius of the star point at this time is approximately 0.5 pixels. To ensure that the center grayscale value of a 5th magnitude star is not less than 30, the total grayscale value of the 5th magnitude star can be estimated to be approximately 60 according to the grayscale dispersion formula. Therefore, the total grayscale value of the 5th magnitude star is set to 60 in this invention. The total grayscale value set in this invention is not absolute, but a relative value that can be changed within an appropriate range based on different simulation requirements and operating conditions.

[0111] For stars of other magnitudes, calculate their total grayscale value based on S305, and determine which pixel range to use for grayscale dispersion. The determination criteria are as follows:

[0112] a. When a star is relatively dim and its total gray value is less than 255×9, diffuse it into a 3×3 pixel.

[0113] b. For bright stars with a total grayscale value greater than 255×9 and less than 255×49, calculating A / 255 yields the minimum number of pixels required for the simulation. By taking the adjacent odd number upwards, the diameter value of the number of pixels diffused by the star can be further calculated.

[0114] c. For bright stars with a total gray value greater than 255×49, diffuse them into 7×7 pixels.

[0115] S4. Calculate the relative pose of the observed satellite and the target based on the data received in step S1.

[0116] S401. Using the target star position vector R in the ECI coordinate system from the data received in step 1 Tar and the observed star position vector R Cha Find the relative position D of the target and the observation platform in the camera coordinate system:

[0117] D = M3(R) Tar -R Cha )

[0118] Where D represents the relative position of the target and the observation platform in the camera coordinate system, and R... Tar R is the position vector of the target star under ECI. Cha This is the position vector of the observed star in the ECI coordinate system.

[0119] S402. Using the target star's position vector R under ECI from the data received in step 1. Tar With velocity vector V Tar The rotation matrix M4 from the target constellation to the observed star constellation is calculated as follows:

[0120]

[0121] Among them, V Tar This is the velocity vector of the target star under ECI.

[0122] S403. Using the rotation matrix M4 obtained in S402, solve the following equations to calculate the target's attitude angles δ1, δ2, and δ3 relative to the observation camera. From this, the target's relative attitude can be determined:

[0123]

[0124] S5. Based on the characteristics of the spatial target, simulate the target image, including:

[0125] S501. Based on the three-dimensional geometric configuration of the target, select a certain number of feature points to reflect its own structural characteristics. Obtain the coordinates of these feature points in the target's intrinsic system according to the target size information, and convert the feature point coordinates in the target's intrinsic system into pixel coordinates in the camera coordinate system through the rotation matrix M4 obtained by S402.

[0126] S502. Analyze the reflection characteristics of the target surface, and use different reflection models to describe the different materials used in different parts of the target. The body part is approximated based on the Blinn-Phong model, with a two-way distribution function f. Phong As shown below:

[0127]

[0128] Where, k d k is the diffuse reflectance coefficient. d When k is set to 0, the model is a pure specular reflection. s ω is the specular reflection coefficient, n is the target surface normal vector, h is the half-angle vector, and ω is the reflectance coefficient. l It is the direction vector of the light ray, ω o It is the direction vector of the viewing axis.

[0129] The antenna section is approximated based on the Lambert model, with a two-way distribution function f. Lambert As shown below:

[0130]

[0131] The solar panel section is based on AB g The model is approximated using the bidirectional distribution function f. ABg As shown below:

[0132]

[0133] Where, β in β is the scattering angle. out Let A be the reflection angle, A be a scale constant reflecting the intensity of the reflected energy from the target, and B and g reflect the degree of concentration of the reflected energy.

[0134] S503. Treating the sun as blackbody radiation at a temperature of 5778 K, the radiant exitance M within the effective wavelength range λ1~λ2 of a visible light camera can be calculated. Using the inverse square law of distance, the radiant exitance of sunlight at a distance D from the sun can be calculated. Tar-sun Spectral irradiance E at a distant target surface sun .

[0135] The target surface is divided into facets based on the feature points selected in S501, and the spectral irradiance dE of each facet is calculated. Tar :

[0136] dE Tar =E sun ×cosθ×f RBD ×dA

[0137] Among them, dE Tar E represents the spectral irradiance radiated outward by each surface element. sun The distance of sunlight from the sun at a distance D is calculated based on the inverse square law of distance. Tar-sun Spectral irradiance at a distant target surface, f RBD Given a two-way reflection distribution function, f is calculated for different parts of the target based on S502. RBD The selection of dA is given by θ, where dA is the area of ​​the infinitesimal element and θ is the angle between the sunlight and the normal to the infinitesimal element.

[0138] Setting the apparent magnitude VM of the Sun Sun The equivalent apparent magnitude VM of the surface element dA is approximately -26.7. dA-Tar The calculation is as follows:

[0139]

[0140] Among them, dE Sun VM represents the solar spectral irradiance received by surface element dA. Sun VM is the apparent magnitude of the Sun. dA-Tar The equivalent apparent magnitude of the surface element dA.

[0141] Based on the total gray value of stars with a magnitude of 5 Further calculation yields the total grayscale value I of the face element. dA-Tar The calculation is as follows:

[0142]

[0143] Among them, VM Star-5 The apparent magnitude is 5. I represents the total grayscale value of a star with a magnitude of 5. dA-Tar This represents the total grayscale value of the face element.

[0144] The number of pixels occupied by the surface element is obtained based on the feature point pixel coordinates in S501, and I is then used to... dA-Tar The target is evenly divided into facets to obtain its grayscale value in the image. Based on the size information of each component of the target, the pixel coordinates of feature points and their grayscale values ​​in the image, the components of the target are drawn onto the image.

[0145] S6. Determine whether the moon will appear in the field of view based on the orbital data, and simulate the situation where the moon is in the field of view.

[0146] S601. Based on the lunar orbit data, calculate its position relative to the observed star in the camera coordinate system, determine whether the angle between the moon and the camera's line of sight in the x and y directions is greater than the camera's field of view angle, and whether the angle formed by the geocenter-camera-moon center is less than the angle formed by the geocenter-camera-camera and the point of tangency between the camera and the edge of the Earth, and then determine the visibility of the moon.

[0147] S602. After determining that the moon is within the field of view, simulate the moon. Based on the relative positions of the moon and Earth, the observed star, and the changing direction of sunlight (the angle θ between the vector perpendicular to the sunlight vector and the position vector of the moon relative to the observed star), the bright-dark interface of the moon in the imaging plane will also change. Correspondingly, the calculation method for the bright area of ​​the moon in the simulated image will also differ. Select an appropriate method based on θ to calculate the imaging area S of the moon in the image.

[0148] by For example, according to Figure 2 Given the geometric relationships shown and the moon's radius r, we can obtain the radius R of the arc formed by the moon's light-dark interface in the image, and the image area (bright area) S of the moon in the image, as calculated below:

[0149]

[0150] Where R is the radius of the arc that the moon's light-dark interface appears as in the image, r is the radius of the moon, θ is the angle between the vector perpendicular to the sun's light vector and the position vector of the moon relative to the observed star, and S is the image area (bright area) of the moon in the image.

[0151] Similarly, we can obtain R and S when θ is in other ranges, as shown in Table 1:

[0152] Table 1. Image size of the moon in the images.

[0153]

[0154] Furthermore, based on the principle of pinhole imaging in cameras, the number of pixels S it occupies in the image is calculated. Pic The calculation is as follows:

[0155]

[0156] Among them, S Pic R is the number of pixels the moon occupies in the image, S is the area of ​​the moon's image in the image, and R is the area of ​​the moon's image in the image. Cam-moon To observe the relative position of the camera and the moon.

[0157] S603. Based on the moon's imaging area in the image and its apparent magnitude during a full moon, the moon's apparent magnitude is calculated. Further, using the formula for calculating stellar brightness, the moon's total grayscale value and grayscale distribution are calculated. Based on the moon's pixel coordinates, total grayscale value, and grayscale distribution in the image, the moon is drawn onto the image.

[0158] S7. Add noise and stray light to the image.

[0159] S701. Determine the angle β between the observation platform and the tangent at the edge of the Earth's atmosphere and the camera's line of sight, and the angle FOV (1 / 2 of the camera's field of view). y The size relationship, if β < FOV y / 2 suggests that atmospheric radiation from the ground affects camera imaging.

[0160] S702. Earth's atmospheric radiation is mainly the reflection of sunlight. Calculate the solar irradiance E received by the Earth's surface. SC The calculation is as follows:

[0161]

[0162] Among them, E SC A represents the irradiance of the Earth's surface from sunlight. S λ is the surface area of ​​the sun, λ1~λ2 is the wavelength range of visible light from the sun, h is Planck's constant, c is the speed of light, k is Boltzmann's constant, T is the temperature, and R is the surface area of ​​the sun. SC This represents the Earth-Sun distance.

[0163] S703. According to Lambert's cosine law, the Earth is considered as a Lambert radiator with constant radiant brightness in all directions. Its radiation intensity varies with the angle θ between the observation direction and the surface source normal, and follows the cosine law. Its surface is divided into countless surface elements, and the irradiance dE of each surface element at the entrance pupil of the detector camera is calculated. e-Cam .

[0164] S704, Setting the Sun's Equivalent Apparent Magnitude (VM) Sun The value is -26.7. Based on the quantitative relationship between magnitude and irradiance, the equivalent apparent magnitude (VM) of the surface element at the entrance pupil of the detector camera is calculated. e The calculation is as follows:

[0165]

[0166] Among them, E sun For sunlight at a distance D from the sun Tar-sun Spectral irradiance at a distant target surface, dE e-Cam Radiance at the entrance pupil of the probe camera for each surface element of the Earth, VM eVM is the equivalent apparent magnitude of the surface element at the entrance pupil of the detector camera. Sun This is the equivalent apparent magnitude of the Sun.

[0167] Furthermore, based on the formula for calculating stellar brightness, the apparent magnitude of the surface element is converted into grayscale values, and the total grayscale value and grayscale distribution of Earth-atmosphere light radiation are calculated.

[0168] S8. The generated image is sent to the image processing module via Ethernet port using UDP transmission, including:

[0169] S801. Set the receiving end IP address and port number.

[0170] S802. Close any existing UDP sockets and create a new UDP object and set the OutputBufferSize property.

[0171] S803: Open a UDP object and send image size information, then send image data in batches.

[0172] S804, Close the UDP object.

[0173] The present invention also provides a real-time simulation system for optical images of airborne targets suitable for close-range operations, comprising:

[0174] The RS422 serial port data receiving module connects the target optical image simulator to the orbit or attitude generator, receiving real-time motion data of the observed star and space target. The camera optical axis calculation module calculates the line-of-sight pointing of the observation camera based on the data received via the RS422 serial port, including the rotation matrix from the ECI coordinate system to the observed star's local frame and the rotation matrix from the ECI coordinate system to the observation camera. The star simulation module, based on the calculated camera optical axis pointing and the camera's ultimate detection capabilities, selects stars detectable by the camera from the SAO star catalog and performs simulations. The relative pose calculation module calculates the relative pose, including relative position and relative attitude, using the position and velocity vectors of the observed and target stars in the ECI coordinate system from the received data. The target simulation imaging module... The system is used to calculate feature points that reflect the structural characteristics of a space target based on its properties, and to simulate the imaging of the target. The moon detection and simulation imaging module determines the visibility of the moon by using its orbital data, whether it is within the camera's field of view, and whether it is obscured by the Earth. If the moon is determined to be within the field of view, it calculates the pixel coordinates, number of pixels, total gray value, and gray distribution of the moon in the image. The noise and stray light addition module calculates the irradiance of the Earth's surface from sunlight, the irradiance of each Earth surface element at the detector's entrance pupil, the equivalent apparent magnitude of the element at the detector's entrance pupil, and the total gray value and gray distribution of atmospheric radiation, based on the stray light imaging model. The Ethernet port image transmission module outputs the simulated image via Ethernet port using UDP transmission.

[0175] Figure 3 The simulation results are based on the on-orbit visual scene simulation of a space target, using the change in distance between the target and the observation platform from far to near as the simulation condition. In (a), the distance between the target and the observation platform is 2.441 km; (b), the distance is 0.167 km; and (c), the distance is 0.019 km. Some stars are represented within the red boxes. The simulation results demonstrate that the simulation method involved in this invention can clearly and realistically reflect the target's shape, movement process, and the relative positional relationship between the target and the stars in the background. This indicates that this invention can simulate the entire process of a target's movement from far to near during a mission.

[0176] After testing, the method proposed in this invention is applicable to image simulation of different satellite targets and different trajectories. Based on the different positions, velocities, and attitude quaternion information of the target and observed satellites, the pixel coordinates of the target's feature points can be changed to simulate different relative poses and trajectories. Depending on the camera parameters and the target size, the position, number, and pixel area of ​​the feature points reflected on the imaging plane also vary, thus achieving the purpose of simulating different target satellites.

[0177] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention. Any other corresponding changes and variations made in accordance with the technical concept of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A real-time simulation method for optical images of empty targets suitable for close-range operations, characterized in that, Includes the following steps: S1. Receive real-time data on the orbital motion of the target and the observed star generated by the orbit or attitude generator; S2. Calculate the camera optical axis pointing based on the received motion trajectory data; S3. Based on the calculated camera optical axis direction and combined with the camera's ultimate detection capability, select stars that can be detected by the camera from the SAO star catalog to complete the star simulation. S4. Calculate the relative pose of the observed satellite and the target based on the data received in S1; S5. Based on the characteristics of the space target, perform imaging simulation of the target; S6. Determine whether the moon will appear in the field of view based on the orbital motion data, and simulate the situation where the moon is in the field of view; S7. Add noise and stray light to the image simulated in S5; S8. Output the image after adding noise and stray light in S7 via Ethernet port in UDP mode. Step S6 includes the following steps: S601. Based on the lunar orbit data, calculate its position relative to the observed star in the camera coordinate system, determine whether the angle between the moon and the camera's line of sight in the x and y directions is greater than the camera's field of view angle, and whether the angle formed by the geocenter-camera-moon center is less than the angle formed by the geocenter-camera-camera and the point of tangency between the camera and the edge of the earth, and then determine the visibility of the moon. S602. After determining that the moon is within the field of view, simulate the moon: Based on the relative positions of the moon, Earth, and the observed star, and the angle between the direction of sunlight (i.e., the vector perpendicular to the sun's rays) and the position vector of the moon relative to the observed star. The changes in the moon's illuminated area in the simulated image were used to determine the calculation method based on these changes. Calculate the area of ​​the moon in the image using the appropriate method. Furthermore, based on the principle of pinhole imaging in cameras, the number of pixels it occupies in the image is calculated. ; S603. Based on the imaging area of ​​the moon in the image and its apparent magnitude in the full moon state, the apparent magnitude of the moon is calculated, and the total gray value and gray distribution of the moon are further calculated. Step S7 includes the following steps: S701. Determine the angle between the observation platform and the tangent at the edge of the Earth's atmosphere and the camera's line of sight. With 1 / 2 camera field of view The size relationship, if it satisfies It is believed that atmospheric radiation from the ground affects camera imaging; S702. Earth's atmospheric radiation is mainly the reflection of sunlight. Calculate the irradiance of the Earth's surface from sunlight. ; S703. According to Lambert's cosine law, if the Earth is considered as a Lambert radiator with constant radiative brightness in all directions, its radiation intensity varies with the angle between the observation direction and the surface source normal. The irradiance varies with the Earth's surface and follows a cosine law. Its surface is divided into countless elements, allowing for the calculation of the irradiance of each element at the entrance pupil of the detector camera. ; S704, setting the equivalent apparent magnitude of the Sun. The equivalent apparent magnitude of the surface element at the entrance pupil of the detector camera was calculated based on the quantitative relationship between magnitude and irradiance. Further, the apparent magnitude of the surface element is converted into grayscale values, and the total grayscale value and grayscale distribution of the Earth-atmosphere radiation are calculated.

2. The real-time simulation method for optical images of an empty target suitable for close-range operations according to claim 1, characterized in that, The target and observed satellite orbit data mentioned in step S1 are as follows: Position and velocity vectors of the target in the ECI coordinate system, position and velocity vectors of the observed star in the ECI coordinate system, attitude quaternions of the observed star in the ECI coordinate system, position vector of the moon in the ECI coordinate system, and direction vector of sunlight in the ECI coordinate system.

3. The real-time simulation method for optical images of an empty target suitable for close-range operations according to claim 1, characterized in that, Step S2 includes the following steps: S201. Calculate the rotation matrix from the ECI coordinate system to the observed star's body system using the quaternions of the observed star's attitude in the received ECI coordinate system. ; S202, Define the camera mounting matrix M2, which is derived from the rotation matrix. and camera mounting matrix Calculate the rotation matrix from the ECI coordinate system to the observation camera. .

4. The real-time simulation method for optical images of an empty target suitable for close-range operations according to claim 1, characterized in that, Step S3 includes the following steps: S301. Based on the camera's detection capabilities, select stars in the SAO catalog that are bright enough to be detected. The selection criteria are as follows: in, This is the collection of stars in the SAO catalog that can be detected within the camera's detection limits. The first in the SAO star table The apparent magnitude of a star. The faintest star magnitude that can be detected by cameras; S302. Based on the right ascension and declination information of stars in the SAO catalog, calculate the values ​​obtained from S301. The azimuth vector of the star in the camera coordinate system : in, This represents the direction vector of the star in the camera coordinate system. , These are the right ascension and declination information of stars, respectively; S303, Results obtained from S302 The result obtained in S301 Further screening of stars within the camera's field of view (Stars-In-Img) was conducted using the following criteria: in, For camera Directional field of view, camera Directional field of view; S304. Calculate the pixel coordinates of the stars (Stars-In-Img) selected in S303 within the image. The calculation formula is as follows: in, These are the pixel coordinates of the star in the image. The focal length of the camera used. and The camera is in and Pixel size in the direction; S305. The simulated camera's detection capability range is -1 to 5 magnitude stars. The total gray value of the faintest star that the camera can detect in the image is set. Then the total gray value of stars of other magnitudes is: in, For star magnitude The total grayscale value of the stars in the image. For star magnitude The total grayscale value of the stars in the image; S306. Using the star pixel coordinates obtained from S304 as the center, perform grayscale dispersion on the star points within a pixel range of 3×3 to 7×7. The dispersion formula is as follows: in, This represents the total grayscale value of the star. Let be the Gaussian dispersion radius. Coordinates of the diffusion center For pixels The grayscale value at that location.

5. The real-time simulation method for optical images of an empty target suitable for close-range operations according to claim 1, characterized in that, Step S4 includes the following steps: S401, Target Star Position Vector Based on ECI Coordinate System and the position vector of the observed star Find the relative positions of the target and the observation platform in the camera coordinate system; S402, Based on the target star's position vector under ECI With velocity vector Calculate the rotation matrix from the target system to the observed star system. ; S403, Using a rotation matrix The target's attitude angle relative to the observation camera was calculated. , , This allows us to obtain the relative posture of the target.

6. The real-time simulation method for optical images of an empty target suitable for close-range operations according to claim 1, characterized in that, Step S5 includes the following steps: S501. Based on the target's three-dimensional geometric configuration, select a certain number of feature points to reflect its structural characteristics. Obtain the coordinates of these feature points in the target's intrinsic system based on the target's size information, and then use a rotation matrix... Convert the feature point coordinates in the target's local coordinate system to pixel coordinates in the camera's coordinate system; S502. Analyze the reflection characteristics of the target surface, and use different reflection models to describe the different materials used in different parts of the target. S503. Treating the sun as blackbody radiation, the effective wavelength range of the visible light camera is calculated. Internal radiation exitance The inverse square law of distance is used to calculate the distance of sunlight from the sun. Spectral irradiance at distant target surface The target surface is divided into facets based on the feature points selected in S501, and the spectral irradiance radiated outward by each facet is calculated. ; Set the apparent magnitude of the sun The surface element is calculated. Equivalent apparent magnitude Based on the total gray value of stars with a magnitude of 5 The total gray value of the face element is further calculated. ; S504. Obtain the number of pixels occupied by the surface element based on the feature point pixel coordinates in S501, and... The target is evenly divided into surface cells to obtain its grayscale value in the image.

7. The real-time simulation method for optical images of an empty target suitable for close-range operations according to claim 1, characterized in that, The number of pixels for: in, This represents the number of pixels the moon occupies in the image. The area of ​​the moon in the image. To observe the relative position of the camera and the moon.

8. A real-time simulation system for optical images of an airborne target suitable for close-range operations, employing the method described in any one of claims 1 to 7, characterized in that, include: The RS422 serial port data receiving module is used to connect the target optical image simulator to the orbit or attitude generator and receive the motion data of the observed star and space target generated by it in real time. The camera optical axis calculation module calculates the line-of-sight direction of the observation camera based on the data received from the RS422 serial port, including the rotation matrix from the ECI coordinate system to the observation star's local system and the rotation matrix from the ECI coordinate system to the observation camera. The star simulation module, based on the calculated camera optical axis direction and the camera's ultimate detection capability, selects stars that can be detected by the camera from the SAO star catalog and performs simulations. The relative pose calculation module uses the position and velocity vectors of the observed star and the target star in the ECI coordinate system from the received data to calculate the relative pose, including relative position and relative attitude. The target simulation imaging module is used to calculate feature points that reflect the structural characteristics of the target based on its properties, and to perform imaging simulation of the target. The Moon Determination and Simulation Imaging Module determines the visibility of the Moon by analyzing its orbital data, whether it is within the camera's field of view, and whether it is obscured by the Earth. If the Moon is determined to be within the field of view, the module calculates the Moon's pixel coordinates, the number of pixels it occupies, and the total grayscale value and grayscale distribution in the image. The noise and stray light addition module calculates the irradiance of the Earth's surface from sunlight, the irradiance of each Earth surface element at the entrance pupil of the detector, the equivalent apparent magnitude of the element at the entrance pupil of the detector camera, and the total gray value and gray distribution of Earth's atmospheric radiation, based on the stray light imaging model. The Ethernet port image transmission module outputs the simulated generated image via Ethernet port using UDP transmission.

Images