Method and system for daytime infrared space surveillance

Abstract

The present invention relates to a space surveillance method for detecting space objects in Earth orbit in images captured during the day, said method comprising the steps of capturing a plurality of infrared images of the daytime sky using a camera equipped with at least one infrared sensor, detecting space objects in Earth orbit based on said images, wherein the detection of bright spots is achieved by a deep learning artificial intelligence system, and identifying each detected object from a catalogue of known space objects in Earth orbit.

Description

[Technical Field]

[0001] The present invention relates to the monitoring of objects in Earth orbit using short wavelength infrared sensors during the day, and more particularly to an improved monitoring method.

[0002] The system allows tracking the evolution of objects in their trajectories, cataloging these objects and updating their trajectories. [Background technology]

[0003] Near Earth space is defined as the region of space extending hundreds of thousands of kilometers from Earth, and thus surveillance of near Earth space is essentially, but not exclusively, concerned with the detection of objects in Earth orbit.

[0004] The gist of the present invention is that an increasing number of objects in Earth orbit are observed, which may be, for example, debris or operational satellites.

[0005] The large number of space objects in Low Earth Orbit (LEO), located between 200 and 2000 km above the Earth's surface, increases the risk of collisions, which in the long term could lead to worsening situations, but above all to risks for operational space resources, whether military, scientific, or commercial. However, with the large number of satellites launched into orbit every year, the space environment is becoming increasingly crowded and contested. To control these risks, it is of utmost importance to catalog all potentially hazardous objects and to attribute valid orbital parameters that allow describing their trajectories.

[0006] Space objects in low Earth orbit, as observed from a fixed point on Earth, have the property of moving rapidly across the sky. Furthermore, at any given moment, there are multiple objects crossing the sky in multiple locations. According to their orbital parameters, each object crosses the local sky at more or less regular time intervals ranging from tens of minutes to several hours.

[0007] Various effects affect the orbits, such as tides, atmospheric drag, radiation pressure, and irregularities in the Earth's gravitational field, which prevents accurate long-term description of these orbits with an invariant set of orbital parameters.

[0008] Furthermore, the size distribution of objects ranges from a characteristic radius of a few millimeters, for example propulsion, paint, or residue from meteorites, to tens of meters, especially in the case of artificial satellites or artificial orbital systems, whether they are in operation or not.

[0009] Conducting surveillance of low-orbit space objects Detecting low orbit space objects without prior knowledge of their existence or location; Defining orbits or orbital parameters with a high degree of accuracy suitable for the intended use; and updating the orbital parameters of the detected objects over time.

[0010] It is also necessary to reacquire the same objects and periodically reread the measurements of their orbital parameters so that their accuracy remains appropriate according to the application to which they have to be put, for example to carry out collision risk assessment procedures.

[0011] Finally, the system must be able to improve the accuracy of knowledge of the orbital parameters of a given object on demand, so as to be able to accurately predict its position in the near future, typically several days, for the purpose of, for example, identifying collision risks and planning potential avoidance maneuvers.

[0012] Telescopes and radar systems are commonly used to monitor space objects in Earth orbit.

[0013] Although cheaper and easier to use than radar systems, ground-based telescopes generally cannot see high altitude satellites during the day due to the bright sky background.

[0014] Also, tracking satellites using radar systems can be difficult due to their limited geographical distribution and their limited range.

[0015] As a result, there are unobserved periods each day during which potentially dangerous and / or harmful space objects can maneuver undetected from the ground, resulting in loss of oversight by satellite operators and potentially endangering nearby satellites.

[0016] Ground-based optical telescopes are constrained to operate at night due to increased photon emission noise, i.e., quantum noise, and the potential for saturation of the daytime sky background. While certain ground-based systems have overcome these challenges, these systems are generally expensive. Space-based surveillance satellites can also be used during the day and do not encounter photon emission noise detection problems, but they are expensive and limited due to observation patterns, the need to address sun avoidance, and relatively long latency times in transmitting tracking data to the ground. Passive ground-based radio frequency (RF) systems can detect stationary space objects (RSOs) during the day, but these RSOs must actively transmit data to ground-based satellite stations. As a result, the majority of RSOs are not observed during the day, leaving nearby RSOs vulnerable to dangerous and / or harmful activity.

[0017] The Graz Observatory in Austria performs laser detection of certain space objects during the day, but observations of these objects are based on visible light sensors and are therefore limited to very bright objects in low Earth orbit.

[0018] The use of short-wave infrared radiation to monitor space objects in Earth orbit is known, as shown in the article "Daytime GEO Tracking with Aquila: Approach and Results from a New Ground-Based SWIR Small Telescope System" by Numerica Corporation. Most technologies using such waves use very large telescopes (greater than 1 meter in diameter) and / or cryogenically cooled optics and generally have not been able to demonstrate the ability to observe objects in orbits farther away than daytime low Earth orbit.

[0019] Systems for the daytime detection of space objects in Earth orbit using infrared sensors are also known from US Pat. No. 1,074,0609 and US Pat. No. 9,423,341.

[0020] Although these systems have advantages over previously known systems, the systems described in these documents remain complex and expensive. The methods implemented involve the use of telescopes specifically calibrated and optimized for operation in the infrared (which is more expensive). They also use complex calibration methods for dealing with noise, particularly methods that use specific developments. [Prior art documents] [Patent documents]

[0021] [Patent Document 1] U.S. Patent No. 10,740,609 [Patent Document 2] U.S. Patent No. 9,423,341 Summary of the Invention [Problem to be solved by the invention]

[0022] A primary object of the present invention is to provide a system and method for detecting space objects in Earth orbit that can operate during the day and that is less expensive than known systems.

[0023] One object of the present invention is a space surveillance method for detecting space objects in Earth orbit in images captured during the day, the method comprising: capturing a plurality of infrared shots of the daytime sky with a camera comprising at least one infrared sensor, each infrared shot comprising an array of pixels each associated with an intensity of light received by the infrared sensor; detecting a space object in Earth orbit based on the captured shots; and identifying each detected object from a catalog of known space objects in Earth orbit. [Means for solving the problem]

[0024] According to a general feature of the invention, the step of detecting a space object in Earth orbit is performed by a deep learning artificial intelligence system, comprising multiple layers of an artificial neural network connected together to analyze information from previous layers of neurons, and the step of detecting a space object in Earth orbit comprises: Detecting bright spots in each shot; Identifying the detected bright spots, which identification comprises tracking each detected bright spot that remains stationary in successive shots and recording the coordinates of the detected bright spots, possibly in different positions, and grouping them by this tracking, wherein for each detected bright spot, recording is performed following its disappearance in the following shot.

[0025] Said infrared sensors, such as short-wave infrared cameras (SWIR), provide images that can be very different from visible images in terms of noise, especially inhomogeneities and defect features, which is why conventional detection algorithms derived directly from visible imaging applications give poor results.

[0026] To obtain better detection results, including for objects with a low signal-to-noise ratio (SNR), the method according to the present invention comprises an artificial intelligence-based algorithm. It is based on training a neural network on a set of simulated images specifically designed to represent as closely as possible real infrared images in the optical configuration used to acquire the images. A set of real images acquired under real conditions in a laboratory setup was studied to build realistic models of noise, defects, response, background, and non-uniformity. These models were then used to simulate a set of images with known ground truth (ground truth data) and used as a training set for the neural network.

[0027] By using deep learning artificial intelligence for processing infrared images, the method according to the invention has the advantage that it can operate on the basis of a simple calibration carried out in the field of view of the sky, without requiring specific modules or deployments.

[0028] Deep learning is one of the main techniques in machine learning, implementing algorithms that can mimic the workings of the human brain, using artificial neural networks consisting of tens or hundreds of "layers" of neurons, each receiving and interpreting information from the previous layer. Deep learning networks are trained based on the complex data structures they encounter.

[0029] The method according to the invention also has the advantage that it can be used with systems that are dedicated to conventional reflecting telescopes, which were not optimized for operation with short wavelength infrared waves, thus facilitating space surveillance from Earth in order to detect space objects in Earth orbit from images captured during the day.

[0030] In the monitoring method of the first embodiment according to the present invention, the layers of the artificial neural network can be calibrated by a supervised learning method based on different databases of images, which allows the artificial intelligence system to determine the typical features of space objects (features that differentiate bright spots corresponding to real space objects from image backgrounds of hot pixels, electronic noise, etc.) before using them to detect space objects (satellites, debris, stars, etc.) present in infrared images.

[0031] In a second embodiment monitoring method according to the present invention, the method may further comprise applying a non-uniformity correction in the captured shot immediately after capturing the shot.

[0032] In a third embodiment of the monitoring method according to the present invention, the method may further comprise filtering each of the obtained stacked images.

[0033] In a fourth embodiment of the surveillance method according to the present invention, the method may further comprise training a stacked image from a superposition of a plurality of said shots, wherein each pixel of the stacked image is associated with a received light intensity corresponding to the average of the intensities of the superposed shots for the same pixel, and performing the detection of space objects using the stacked image as the shot to be processed.

[0034] In a fifth embodiment of the monitoring method according to the present invention, the method may further comprise a step of removing streak defects in each of the stacked images to remove streak defects in the stacked images before the step of detecting a space object in Earth orbit.

[0035] Another object of the invention is to propose a space surveillance system for detecting space objects in Earth orbit, said system comprising a reflecting telescope mounted on a mechanical support with motorized displacement, a camera equipped with at least one infrared sensor attached to the output of the reflecting telescope and configured to capture a series of shots of the daytime sky at a frequency between 1 Hz and several hundred Hertz, and a processing unit receiving each shot captured by the camera.

[0036] According to a general feature of the space surveillance system according to the invention, the processing unit may comprise a deep learning artificial intelligence system comprising a plurality of artificial neural network layers connected together to analyze information of the preceding layers of neurons, the processing unit being adapted to perform, based on the received images: detecting a space object in Earth orbit based on the captured shots, the detection being performed by a deep learning artificial intelligence system; identifying each detected object from a catalog of known space objects in Earth orbit; detecting space objects in Earth orbit, the step comprising detecting bright points in each shot and identifying the detected bright points, the identification comprising tracking each detected bright point that remains stationary in successive shots; The method comprises recording the coordinates of the detected bright spots, possibly in different positions, and grouping them by this tracking, the recording being configured to carry out, for each detected bright spot, the steps to be carried out following its disappearance in the following shot.

[0037] In one embodiment of the space surveillance system, the camera may further comprise at least one visible light sensor attached to the output of the reflecting telescope and configured to capture a series of shots of the night sky, and the space surveillance system further comprises a day / night alternation module that enables the type of sensor that receives light from the sky to be changed depending on the ambient light intensity.

[0038] Other characteristics and advantages of the invention will become apparent from the description given below, with reference to the attached drawings, which show exemplary, non-limiting embodiments. [Brief explanation of the drawings]

[0039] [Figure 1] FIG. 1 illustrates a space surveillance method according to one embodiment of the present invention. [Figure 2] FIG. 2 illustrates a space surveillance system according to a first embodiment of the present invention. [Figure 3] FIG. 3 illustrates a space surveillance system according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0040] FIG. 2 shows a schematic diagram of a space surveillance system according to a first embodiment of the present invention.

[0041] The space surveillance system 1 is configured to detect space objects in Earth orbit. The system 1 comprises a reflecting telescope 2 mounted on a mechanical support 3 with motorized displacement, a camera 4 attached to the output of the reflecting telescope 2 and comprising at least one infrared sensor 40, and a processing unit 5 receiving each image captured by the camera 4.

[0042] The camera 4 is configured to capture successive shots of the daytime sky at a frequency between 1 Hz and several hundred Hertz. The exposure time of each shot is adjusted to limit saturation of the infrared sensor 40.

[0043] The processing unit 5 comprises a deep learning artificial intelligence 50 comprising multiple artificial neural network layers connected together to analyze the information of the preceding neuron layers, and is configured to implement a space surveillance method on images captured during the day.

[0044] FIG. 1 illustrates a monitoring method according to one embodiment of the present invention.

[0045] In a first step 100 of the method, multiple infrared images of the sky are captured during the day using the camera 4 and its infrared sensor 40. Each infrared image comprises an array of pixels, each associated with an intensity of light received by the infrared sensor.

[0046] In the second step 110 of the method, a correction for pixel response non-uniformity is performed for each captured image. This correction is performed via the pre-acquisition of a uniform reference image taken directly on the sky in an area free of luminous objects (stars or satellites). This calibration does not require the removal of sensors or additional equipment.

[0047] In a subsequent step 120 of the method, a stack of successively captured images is created. The resulting stacked image is generated by superimposing the infrared images of the stack. Each pixel of the stacked image is associated with a received light intensity corresponding to the average of the intensities of the superimposed infrared images for the same pixel.

[0048] In a subsequent step 130, a streak removal is performed on each stacked image, which makes it possible to remove streaks that may appear in the stacked images due to the overlay of infrared images.

[0049] In a subsequent step 140, bright spot detection, or "plot detection", is performed on each of the stacked images from which the streaks have been removed. This step consists of extracting the bright spots from the background noise.

[0050] This bright spot detection step 140 can be preceded by conventional (averaging type) image filtering to reduce background noise and improve detection performance.

[0051] The step 140 of detecting bright spots is performed by a deep learning artificial intelligence system 50 comprising multiple artificial neural network layers connected together to analyze the information of the preceding neuron layers.

[0052] In a subsequent step 150, the bright points detected in the preceding step 140 are identified as stars or satellites. This identification comprises tracking the detected bright points that are stationary in the image (these stationary points therefore belong to the same space object). Then, once all points from a given space object are listed, the coordinates of the bright points detected in potentially different positions and grouped together by this tracking are recorded. That is, once the detected bright points disappear in the following image, a recording is made.

[0053] Thus, detecting space objects in step 150 allows for the determination of space objects in orbit around the Earth that are present in the image.

[0054] In a subsequent step 160, the detected object is compared to objects listed in a catalog of known space objects in Earth orbit, which lists known space objects in Earth orbit indicating various characteristics of the space object, i.e., its size, orbit, and various unique features.

[0055] In a subsequent step 170, new features are extracted from objects detected in the stacked images in order to update the information associated with this space object if it is already listed in the catalog, or to add a new space object to the catalog.

[0056] Before being used to detect space objects (stars, satellites, debris, etc.), several layers of the artificial neural network are calibrated by supervised learning from a base of various images, which allows the artificial intelligence system to determine the typical characteristics of space objects (stars, satellites, debris, etc.). The calibration is performed on a set of simulated images generated to reproduce a typical image from the sensor. Each simulated image comprises a background and background noise, bright spots corresponding to stars or space objects or sensor defects. Each image created in this way is associated with its true, i.e., the position of the actual objects (stars and satellites) in the image. A large number of images are created in this way and serve as a training base for the neural network.

[0057] Thus, the present invention provides a system and method for detecting space objects in Earth orbit that is capable of daytime operation and is less expensive than known systems.

Claims

1. 1. A space surveillance method for detecting space objects in Earth orbit in daytime captured images, comprising: The space surveillance method includes: capturing (100) a plurality of infrared shots of a daytime sky using a camera comprising at least one infrared sensor, each infrared shot comprising an array of pixels each associated with an intensity of light received by the infrared sensor; detecting (140, 150) space objects in Earth orbit based on these infrared shots; and identifying (160) each detected object from a catalog of known space objects in Earth orbit; the step of detecting space objects in Earth orbit is performed by a deep learning artificial intelligence system comprising multiple layers of an artificial neural network connected to each other to analyze information from previous layers of neurons, the deep learning being based on simulated images generated to reproduce typical images coming from an infrared sensor, the typical images comprising background and background noise, spots of light corresponding to either stars or space objects, or spots of light corresponding to defects in the infrared sensor, each of the simulated images being associated with a truth based on the position of a real object in the image; The step of detecting a space object in Earth orbit includes: Detecting bright points in each shot (140); A space surveillance method comprising: a step (150) of identifying detected bright points, which identification comprises tracking each detected bright point that is stationary in successive shots, and recording the coordinates of the detected bright points, possibly in different positions, and grouping them by this tracking, said recording being performed for each detected bright point following its disappearance in a subsequent shot.

2. 2. The space surveillance method of claim 1, wherein the multiple layers of the artificial neural network are calibrated before being used to detect space objects by a supervised learning method using a base of various images that enable the deep learning artificial intelligence system to determine typical characteristics of space objects.

3. 3. The space surveillance method of claim 1, further comprising applying non-uniformity correction to the captured infrared shots (110) immediately after the step of capturing the infrared shots (100).

4. A space surveillance method according to any one of claims 1 to 3, further comprising filtering each shot.

5. The space surveillance method according to any one of claims 1 to 4, further comprising a step (120) of forming a stacked image from an overlay of the plurality of infrared shots, wherein each pixel of the stacked image is associated with a received light intensity corresponding to an average of the intensities of the overlaid shots for the same pixel, and the stacked image comprises a detection of a space object (140 and 150) used as a shot to be processed.

6. 6. The space surveillance method of claim 5, further comprising a step (130) of removing streak defects in each of the stacked images to remove streak defects in the stacked images before the step of detecting the bright points.

7. A space surveillance system (1, 10) for detecting space objects in Earth orbit, said space surveillance system (1) comprising: a reflecting telescope (2) mounted on a mechanical support (3) with motorized displacement; a camera (4, 4') equipped with at least one infrared sensor (40) attached to the output of the reflecting telescope (2) and configured to capture a series of shots of the daytime sky at a frequency between 1 Hz and several hundred Hertz; a processing unit (5) for receiving each shot captured by said camera, the processing unit comprises a deep learning artificial intelligence system (50) comprising multiple layers of an artificial neural network connected together to analyze information from a previous layer of neurons; The processing unit, based on the received image, The system detects space objects in Earth orbit based on captured shots, and the detection of bright spots is performed by a deep learning artificial intelligence system. identifying each detected object from a catalog of known space objects in Earth orbit; Detecting a space object in Earth orbit, comprising: detecting (140) a bright spot in each shot; and identifying (150) the detected bright spot; The space surveillance system (1, 10) is configured to perform the steps of: tracking each detected bright spot that remains stationary in successive shots; recording the coordinates of the detected bright spots, possibly in different locations; and grouping them according to this tracking, the recording being performed for each detected bright spot after its disappearance in the following shot.

8. 8. The space surveillance system (10) of claim 7, wherein the camera (4') further comprises at least one visible light sensor (42) attached to the output of the reflecting telescope (2) and configured to capture a series of shots of the night sky, and the space surveillance system (10) further comprises a day / night alternation module (44) that enables changing the type of sensor that receives light from the sky depending on the ambient light intensity.

Images