Infrared surveillance system for military aircraft and military aircraft, including missiles, equipped with such a system
The optronic system with fixed sensors and inertial units aligns images in a common frame to enhance detection precision and reduce noise, addressing the challenge of accurately identifying thermal traces on moving aircraft.
Patent Information
- Application Number
- FR2022003130
- Authority / Receiving Office
- FR · FR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-04-06
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2042-04-06
AI Technical Summary
Existing optronic surveillance systems on military aircraft face challenges in accurately detecting thermal traces of missiles due to image noise and false alarms, particularly when the aircraft is in motion, complicating the detection of thermal traces across multiple images.
An optronic system with fixed, non-orientable sensors and inertial measurement units registers images in a common reference frame using inertial data to align images, triggering an alert when a zone exceeds a predetermined intensity threshold, enhancing detection precision by eliminating noise and aligning thermal traces across images.
The system improves detection accuracy by aligning images in a common reference frame, reducing noise, and efficiently detecting thermal traces of interest, such as missile launches, by summing image signals to enhance detection sensitivity and reduce false alarms.
Smart Images

Figure 00000013_0000 
Figure 00000013_0001 
Figure 00000013_0002
Abstract
Description
Title of the invention: Infrared surveillance system for military aircraft and military aircraft, in particular a missile, equipped with such a system
[0001] The present invention relates to the field of optical surveillance of the environment of a military aircraft and more particularly of a missile or a military aircraft such as a fighter.
[0002] BACKGROUND OF THE INVENTION
[0003] It is known to equip missiles and combat aircraft with optronic surveillance devices.
[0004] The most sophisticated devices, often located at the front of the aircraft, include, for example, a day vision sensor for long-range identification (40 km) and an infrared sensor with sensitivity in the wavelength bands 3 to 5 pm and 8 to 12 pm for detecting targets at long distances. These front surveillance devices are used for self-protection of the aircraft (detection of enemy missiles, for example) or for attack (target detection).
[0005] For self-protection, the images provided by the sensor are processed to return look for example for the thermal trace of the hot gases ejected by the missile at least at the time of its departure (we speak of the "feathers" of the missile) or of hot parts of the missile, more particularly on the fuselage. In the image, this thermal trace takes the form of an area or spot, which can be the size of a single pixel, having an intensity greater than an alert threshold. The alert threshold is relatively low to be able to detect a missile departure as far away as possible but must be significantly higher than the image noise to avoid the multiplication of false alerts. Image processing is therefore relatively delicate.
[0006] SUBJECT OF THE INVENTION
[0007] The invention aims in particular to provide an optronic system for improved surveillance of the environment of a missile or a military aircraft. Summary of the invention
[0008] To this end, according to the invention, there is provided an optronic system for monitoring a space surrounding a military aircraft, comprising an electronic processing unit, optronic sensors connected to the electronic processing unit and at least one inertial measurement unit connected to the electronic processing unit. The optronic sensors have a predetermined individual field and are arranged to be fixed on the aircraft so as to have an overall field corresponding substantially to the space to be monitored. The electronic processing unit incorporates a processor and a memory containing a program having instructions arranged to: - receive from each optronic sensor a plurality of image signals corresponding to images captured successively over a predetermined duration; - re-align the images together in relation to each other, in a reference frame common to all the sensors, from inertial positioning data provided by the inertial measurement unit; - trigger an alert when several of the images include, after registration, the same area with an intensity greater than a predetermined threshold.
[0009] The military aircraft may be a combat aircraft, a missile or other and the space to be monitored may be for example 360° or 180°. Thus, when the aircraft is moving, the images in the series of images provided by each optronic sensor are offset from each other due to the fact that each optronic sensor is fixed relative to the aircraft and that the aircraft has moved in space. This offset depends on the time interval between each image, the speed of the aircraft and the movements it has made. Therefore, the same element of the landscape scanned by a sensor, such as for example a thermal trace of a missile launch in the case of infrared sensors, will not be positioned exactly in the same place in the images of the series, even if the movement of the aircraft is minimal between two images, which complicates the detection of this element of the landscape.Inertial data is used to determine the aircraft speed and movements between images to be able to align the captured images with a common reference point for all sensors and thus make the same landscape elements (including missile thermal traces) visible on several captured images coincide. This makes it possible to highlight the thermal traces of interest by eliminating noise, which is present randomly from one image to another. Detection is therefore more precise.
[0010] The invention also relates to an aircraft equipped with such an optronic system, the aircraft comprising a structure on which the optronic sensors are rigidly fixed and distributed.
[0011] By "rigidly fixed" is meant that the optronic sensors are not orientable relative to the structure of the aircraft, which simplifies the arrangement of the sensors and increases the acquisition speed since the images are not obtained by a scanning movement of the optronic sensors.
[0012] The invention also relates to a method for monitoring a space surrounding a military aircraft provided with optronic sensors and at least one inertial measurement unit which are connected to an electronic processing unit, the optronic sensors having a predetermined individual field and being fixed on the aircraft so as to have an overall field corresponding substantially to the space to be monitored, the method comprising the steps of: - receive from each optronic sensor a plurality of image signals corresponding to images captured successively over a predetermined duration; - re-align the images from the different sensors together in relation to each other, in a frame of reference common to all the sensors, using inertial positioning data provided by the inertial measurement unit; - trigger an alert when several of the images include, after re-alignment in the reference frame common to all sensors, the same area having an intensity greater than a predetermined threshold.
[0013] Other characteristics and advantages of the invention will emerge from reading the following description of a particular and non-limiting embodiment of the invention. Brief description of the drawings
[0014] Reference will be made to the accompanying drawings, among which:
[0015] [Fig-1] [Fig.l] is a schematic view of a location device according to the invention;
[0016] [Fig.2] [Fig.2] is a schematic top view of an aircraft equipped with a location device according to the invention;
[0017] [Fig.3] [Fig.3] is a schematic view from below of this aircraft;
[0018] [Fig.4] [Fig.4] is a schematic side view of this aircraft. DETAILED DESCRIPTION OF THE INVENTION
[0019] With reference to Figures 2 to 4, the invention is described here in application to the surveillance of the space surrounding an aircraft, generally designated 100, having a structure 101 (the cell or fuselage) and wings 102, the other parts of the aircraft 100 not being mentioned here. The aircraft 100 carries a surveillance device in accordance with the invention.
[0020] Also referring to [Fig.l], the monitoring device according to the invention comprises an electronic processing unit 1 connected to six optronic sensors 2 and to six inertial measurement units 3 each arranged in the vicinity of one of the optronic sensors 2. In the figures, the numerical reference 2 is associated with the letters: “fu” to designate an optronic sensor 2 facing forward and upward, “ru” to designate an optronic sensor 2 facing backward and upward, “fd” to designate an optronic sensor 2 facing forward and downward, “rd” to designate an optronic sensor 2 facing backward and downward, “I1” to designate an optronic sensor 2 positioned laterally and facing left, “Ir” for designate an optronic sensor 2 positioned laterally and oriented to the right. In the figures, the numerical reference 3 is associated with the same letters as the optronic sensor 2 in the vicinity of which the inertial measurement unit 3 is fixed.
[0021] Each optronic sensor 2 is here a sensor made of a material sensitive to wavelengths between approximately 3 and 5 pm (MWIR band) and between approximately 8 and 12 pm (LWIR band). Indeed, the sensitivity to the MWIR band makes it possible to detect combustion gases (from a missile or aircraft and more generally from any aircraft) which include a lot of carbon dioxide emitting at the wavelength of 4.2 pm. The sensitivity to the LWIR band makes it possible to more effectively detect missiles whose engine has been stopped and missiles and aircraft coming from the front. The sensor is for example made of mercury-cadmium telluride (HgCdTe) or of a material having a periodic structure of “superlattice” type layers having sensitivity in the bands concerned.Each optronic sensor 2 is here a focal plane array or FPA detector (from the English "focal plane array"), has a predetermined individual field here of 105° by 84°, and has a resolution of 2560 pixels by 2048 pixels (it is recalled that an optronic sensor is composed of a plurality of elementary detectors adjacent to each other and that a pixel, from the English "picture element", designates such an elementary detector). Each pixel is formed by a photodiode and a well accumulating electric charges according to the photons that it receives. These electric charges are extracted periodically in the form of an elementary signal. All of the elementary signals extracted at a given instant from the elementary detectors of an optronic sensor 2 form an image signal. The frame frequency of the optronic sensors 2 is preferably between 50 and 100 Hz.
[0022] The optronic sensors 2 are rigidly fixed to the cell 101 of the aircraft 100 as follows: - a first optronic sensor 2fu opens onto an upper surface of the structure 101, here in front of the cockpit, and is oriented towards the front of the aircraft 100; - a second optronic sensor 2ru opens onto the upper surface of the structure 101, here behind the cockpit, and is oriented towards the rear of the aircraft 100; - a third optronic sensor 2fd opens onto a lower surface of the structure 101 and is oriented towards the front of the aircraft 100; - a fourth optronic sensor 2nd opens onto the lower surface of the structure 101 and is oriented towards the rear of the aircraft 100; - a fifth sensor 211 and a sixth sensor 21r open onto opposite lateral surfaces of the structure 101 and are oriented on a transverse axis of the structure 101 respectively towards the left and towards the right.
[0023] This positioning makes it possible to combine the individual fields of the op-tronic sensors 2 to obtain a global field of 4ir steradians covering the entire space surrounding the aircraft 100.
[0024] Each inertial measurement unit 3 (commonly referred to as an IMU) comprises, in a manner known per se, accelerometers and gyroscopes or gyrometers for determining linear and angular movements relative to the axes of a reference frame. Each inertial measurement unit 3 is fixed to the same part of the structure 101 as one of the optronic sensors 2, in the immediate vicinity thereof, to detect the movements of the part of structure 101 to which the optronic sensor 2 is fixed. The fact of fixing an inertial measurement unit 3 on the same part of structure 101 as the associated optronic sensor 2 and in the immediate vicinity thereof makes it possible to limit the possible relative movements of the inertial measurement unit 3 of the associated optronic sensor 2 which could result for example from an elastic deformation of the structure 101.Each inertial measurement unit 3 is arranged to provide electrical signals, here called positioning signals, representative of inertial attitude data (orientation of the support in space around the axes of the reference frame) and inertial movement data of the cell part 101 to which the inertial measurement unit 3 and the associated optronic sensor 2 are fixed. More precisely, the inertial data have the form of quaternions.
[0025] The electronic processing unit 1 comprises a calculation component, such as a processor, and at least one memory containing at least one program executable by the calculation component. The electronic unit may be produced in the form of an integrated circuit, an FPGA circuit, or an electronic card comprising a substrate provided with conductive tracks and electronic components connected to the conductive tracks. The program contained in the electronic processing unit 1 comprises instructions arranged to:
[0026] - control the acquisition of image signals by each optronic sensor 2 and more generally control each optronic sensor 2;
[0027] - recover the inertial data contained in the positioning signals from each inertial measurement unit 3 and perform calculations on this inertial data;
[0028] - associate the inertial data of each inertial measurement unit 3 with the signals image of the optronic sensor associated with it so that at each capture moment there corresponds an image and inertial positioning data,
[0029] - carry out processing of the image signals provided by each optronic sensor 2.
[0030] More precisely, the processing carried out comprises, for each optronic sensor 2 and the associated inertial measurement unit 3, the steps of:
[0031] - control the acquisition of image signals by each optronic sensor 2 for a predetermined period;
[0032] - realign the images with respect to each other, in a single, common reference frame to all the optronic sensors, from the inertial data provided by the inertial measurement unit 3 associated with the optronic sensor 2 having provided said images to be readjusted.
[0033] Finally, for all the optronic sensors and on the images assembled in the single reference frame common to all the optronic sensors, the processing comprises the step of triggering an alert when several of the images comprise, after registration, the same zone having an intensity greater than a predetermined threshold.
[0034] As an illustration, with a number N of optronic sensors 2: at each instant t each optronic sensor n provides an image noted image_sensor(n)_instant(t); the recalibration of these images at each instant t in a reference frame common to all the sensors provides a “macro-image” such that
[0035] macro_im.age_ins^ - image_sensor(n)_instant(l); the processing is carried out on a series of macro-images, corresponding to successive moments, realigned with respect to each other in the common reference frame.
[0036] The reference point common to the optronic sensors 2 is obtained for example by designating one of the inertial measurement units 3 as “master” and by choosing the reference point of this master inertial measurement unit as the common reference point. The other inertial measurement units 3 are aligned, in a manner known per se, with the reference point of the “master” inertial measurement unit.
[0037] It is recalled that the registration makes it possible to make the same elements of the landscape (including the thermal traces of missiles) which would be visible on several captured images but at different locations of the images coincide, taking into account the movements of the aircraft 100 between the captures. This registration is possible because the inertial data makes it possible to determine the difference in position in space and the difference in orientation in space of each optronic sensor 2 between two successive images. The predetermined threshold is of the order of magnitude of the noise: it is the use of several images which makes it possible to reveal that the area in question is an area of interest. The registration thus makes it possible to replace the images in the same fixed reference frame which is not that of the carrier.
[0038] In practice, the program of the electronic processing unit 1 implements the following steps:
[0039] - sum the image signals obtained during the predetermined duration by resetting them to from the inertial positioning data so as to obtain a result signal;
[0040] - search in the result signal for one or more portions of interest having a amplitude greater than the predetermined threshold.
[0041] Thus, during operation of the device of the invention, the signals corresponding to the same scene visible in the images of the series are accumulated to form a result signal. Even if the portion of interest of each image signal, representative of the presence of a thermal trace, is of low amplitude, the result signal will comprise a corresponding portion of interest whose amplitude will be the sum of the amplitudes of the portions of interest of all the image signals. This temporal accumulation amounts to making the optronic sensor comparable to a fictitious sensor stabilized in space and having an increased well depth. In addition, since the noise is random, it is attenuated relative to the portion of interest in the result signal. The accumulation of the image signals thus makes it possible to highlight that several images included, after registration, the same zone having an intensity greater than the fixed threshold.
[0042] It will be noted that in the case of a missile targeting the aircraft 100, the missile is on a collision course so that it appears for most of its course at the same heading relative to the aircraft. This is the reason why it is possible to use a relatively low detection threshold and several images to detect it efficiently. We are in substantially the same situation with a distant aircraft.
[0043] Of course, the invention is not limited to the embodiment described but encompasses any variant falling within the scope of the invention as defined by the claims.
[0044] In particular, the monitoring device according to the invention may have a structure different from that described.
[0045] It is possible not to have an inertial measurement unit per sensor. Preferably, it will be ensured that no sensor is more than one meter away from an inertial measurement unit.
[0046] In a less efficient embodiment than that described, it is possible to use the aircraft's inertial unit to recalibrate the images with each other, assuming that the cell does not deform or does so within acceptable limits relative to the desired detection sensitivity.
[0047] The optronic sensors may be sensitive to only one of the two wavelength bands mentioned or to other wavelength bands than those mentioned.
[0048] Other materials than those mentioned are conceivable, notably indium antimonide (InSb).
[0049] It is also possible to use quantum well infrared photodetectors.
[0050] Quaternions can be used in inertial data but also Euler angles and rotation matrices.
[0051] It is further possible to combine sensors from the infrared range and sensors from the visible range.
[0052] Other sensor sizes are usable and for example (in pixels): 640 by 640, 1280 by 1280, 1600 by 1600, 2000 by 2000, 2560 by 2048, 4000 by 4000, 5120 by 5120, or others.
[0053] The number and position of the optronic sensors may be different from those indicated above. Preferably, the optronic sensors are n in number and each have a field substantially equal to 1 / n of the space to be monitored.
[0054] To remove the signal from the noise, it is possible to use the summation method that has been described but other known methods are conceivable. Thus, another method comprises the steps of: - set a relatively low threshold on each image, which generates a high number of plots, a large part of which corresponds to noise; - filter the "noise" plots from the "target" plots using statistics. According to the statistical approach, if in a restricted area of a few pixels around a plot of interest in an image, there are, in a first predetermined number of subsequent images, a second predetermined number of new plots that exceed the threshold, then the plot of interest is likely to be a "target" plot. For example, we consider that if in a restricted area of a few pixels around a plot, there are, in the next 5 images, at least 3 new plots that exceed the threshold, then the plot initially considered is likely to be a "target" plot. It is commonly said that all the plots are tracked here in 3 / 5 (3 plots detected on 5 acquisitions).
[0055] The invention is applicable to any type of aircraft, piloted or not.
Claims
1. Claims Optronic system for monitoring a space surrounding a military aircraft (100), comprising an electronic processing unit (1), a plurality N of optronic sensors (2) connected to the electronic processing unit (1) and at least one inertial measurement unit (3) connected to the electronic processing unit (1), the optronic sensors (2) having a predetermined individual field and being arranged to be fixed on the aircraft so as to have an overall field corresponding substantially to the space to be monitored, the electronic processing unit (1) incorporating a processor and a memory containing a program having instructions arranged to: - receive from each optronic sensor (2) a plurality of image signals corresponding to images captured successively over a predetermined duration; - re-align the images from the different sensors together in relation to each other, in a reference frame common to all the sensors, from inertial positioning data provided by the inertial measurement unit (3); - trigger an alert when several of the images include, after registration in the reference frame common to all sensors, the same zone having an intensity greater than a predetermined threshold; in such a way that: - at each instant t each optronic sensor n provides an image image_sensor(n)_instant(t); - these images are realigned at each instant t in a reference frame common to all the sensors to provide a macro-image such that / \ ,y macro _image_instanttt\ - image _sensor(n)_instant(t) ' - the macro-images of a series of macro-images, corresponding to successive instants, are recalibrated relative to each other in the common reference frame to detect the area having an intensity greater than a predetermined threshold.
2. System according to claim 1, in which the program comprises the steps of: - summing the image signals obtained during the predetermined duration by realigning them from the inertial positioning data so as to obtain a result signal; - searching in the result signal for a portion of interest having an amplitude greater than the predetermined threshold.
3. The system of claim 1, wherein the program comprises the steps of: - setting a predetermined threshold on each image to generate one or more plots; - filtering plots corresponding to noise from plots corresponding to a target using a statistical approach.
4. The system of claim 3, wherein, according to the statistical approach, if in a restricted area around a plot of interest in an image, there are, in a first predetermined number of subsequent images, a second predetermined number of new plots corresponding to pixels exceeding the predetermined threshold, then the plot of interest is likely a plot corresponding to a target.
5. System according to claim 1, in which the optronic sensors (2) are n in number and each have a field substantially equal to 1 / n of the space to be monitored.
6. A system according to any preceding claim, wherein the optronic sensors (2) have one of the following approximate resolutions (pixels): - 640 by 640, 1280 by 1280, - 2000 by 2000, 2560 by 2048, - 4000 by 4000.
7. System according to any one of the preceding claims, wherein the optronic sensors (2) are sensitive to wavelengths of the MWIR band.
8. System according to claim 7, wherein the optronic sensors (2) are sensitive to wavelengths of the LWIR band as well.
9. Military aircraft (100) equipped with an optronic system according to any one of the preceding claims, the aircraft comprising a structure (101) on which the optronic sensors (2) are rigidly fixed and distributed.
10. Military aircraft (100) according to claim 9, wherein the optronic sensors (2) are positioned as follows: - a first optronic sensor (2fu) opening onto an upper surface of the structure (101) and oriented towards the front of the aircraft (100); - a second optronic sensor (2ru) opening onto the upper surface of the structure (101) and oriented towards the rear of the aircraft (100); - a third optronic sensor (2fd) opening onto a lower surface of the structure (101) and oriented towards the front of the aircraft (100); - a fourth optronic sensor (2rd) opening onto the lower surface of the structure (101) and oriented towards the rear of the aircraft (100); - a fifth sensor (211) and a sixth sensor (21r) which open onto opposite lateral surfaces of the structure (101) and which are oriented on a transverse axis of the fuselage.
11. Military aircraft (100) according to claim 10, wherein the monitoring device comprises as many inertial measurement units (3) as optronic sensors (2) and each inertial measurement unit (3) is fixed to the structure (101) in the vicinity of each optronic sensor (2).
12. Method for monitoring a space surrounding a military aircraft (100) provided with a plurality N of optronic sensors (2) and at least one inertial measurement unit (3) which are connected to an electronic processing unit (1), the optronic sensors (2) having a predetermined individual field and being fixed on the aircraft so as to have an overall field corresponding substantially to the space to be monitored, the method comprising the steps of: - receiving from each optronic sensor (2) a plurality of image signals corresponding to images captured successively over a predetermined duration; - re-align the images from the different sensors together in relation to each other, in a reference frame common to all the sensors, from inertial positioning data provided by the inertial measurement unit (3); - trigger an alert when several of the images include, after registration in the reference frame common to all sensors, the same zone having an intensity greater than a predetermined threshold; in such a way that: - at each instant t each optronic sensor n provides an image image_sensor(n)_instant(t); - these images are realigned at each instant t in a reference frame common to all the sensors to provide a macro-image such that macro _image _instan^ = image_sensor(n)_instant(t) ' - the macro-images of a series of macro-images, corresponding to successive instants, are recalibrated relative to each other in the common reference frame to detect the area having an intensity greater than a predetermined threshold.
