METHOD FOR DETECTING A TARGET OBJECT USING A CAMERA

Description

[0001] The invention relates to a method for detecting a target object by means of a camera, in which the camera takes several images and an object is determined as a potential target object from at least one of the images.

[0002] Vehicles on water, land, and in the air are usually reliably detected by infrared cameras based on their heat signature during operation. While this is generally easy in cold and low-reflection environments, surrounding land areas with low temperature contrasts or even sunlight reflections on ice surfaces, such as a glacier, can create heat signatures in an IR image that can be misinterpreted as target objects. Finding a vehicle in a signature-rich IR image using image processing methods can be a challenging task.

[0003] From "Francisco Bonin-Font, et al., 'A Visual Navigation Strategy Based on Inverse Perspective Transformation', in 'Robot Vision', March 1, 2010" obstacle detection based on an inverse perspective transformation is known.

[0004] It is an object of the present invention to provide a reliable method for detecting a target object by means of a camera.

[0005] This object is achieved by a method according to claim 1, in which, according to the invention, line-of-sight information of the potential target object is determined from the images in relation to a topography assumed to be known, and the target object is recognized as such on the basis of the line-of-sight information.

[0006] The invention is based on the idea that most noise signatures in infrared images originate from a topography, i.e., from a surface such as a landscape, a cloud surface, or an ice surface, particularly from stationary objects belonging to a topography. A target object moving relative to this topography or to one or more stationary objects in the topography causes a heat signature that moves relative to the noise signatures in several images.

[0007] The camera can be stationary or moved relative to the topography. If the camera is stationary, a target object is relatively easy to detect, as it is the only object moving through the images. The line-of-sight information of an object in the topography can be set to zero if it is moving, such as a line-of-sight angular rate or acceleration.

[0008] However, things become more difficult when the camera is moved relative to the topography. The location of the potential target object moves through the images, regardless of whether it is a stationary clutter object or a moving vehicle. If the topography or the movement of the camera relative to the topography is complex, the movements of the clutter signatures are also complex and not identical to one another, making it difficult to separate a target object from a clutter signature. For example, if a missile with the camera moves in complex trajectories over a hot, mountainous landscape, the entire landscape as topography moves relative to the missile, and thus the clutter signatures also move in unpredictable trajectories – if the topography is unknown – through the images.

[0009] However, if the topography is known, the trajectories of the interference signatures can be related to each other, since the topography as a whole moves relative to the missile or camera. Only by relating, for example, the movement of a vehicle relative to the topography can the target object be identified as such. The movement of the topography can be subtracted or used as a comparison movement. If a vehicle moves relative to this topography, its line-of-sight information deviates from the line-of-sight information that lies within this topographical entity, and the vehicle or target object can be identified as such.

[0010] In this respect, one or more target objects are first identified as potential target objects from at least one of the images. For the sake of simplicity, we will assume only one potential target object below. The potential target object can be identified using image processing methods, although it is not yet necessary to know at this point whether it is a desired target object, in this case a vehicle, or an interference signature, i.e. an undesirable topographical object, such as a very heated rock or a reflection on ice. Line-of-sight information of the potential target object is then determined from one or more of the images. The line-of-sight information can be related to the topography, which is assumed to be known. Depending on the line-of-sight information in relation to the topography, the potential target object can now be recognized as the true target object, i.e. as the target object as such.

[0011] The line-of-sight information can be a line-of-sight direction from the camera to the potential target object, a location of the line-of-sight on or within the topography, a line-of-sight rotation rate, and / or a line-of-sight rotation acceleration. The line-of-sight information can be determined from one or more of the captured images. If the line-of-sight information is a crossing point or intersection of the line-of-sight with the topography, the line-of-sight information can be determined from a single image. However, if the line-of-sight information is a line-of-sight rotation rate or a line-of-sight rotation acceleration, it is expediently determined from two or more images.

[0012] The target object is a vehicle on land, on water, or in the air, which can also include missiles of all kinds. The potential target object is conveniently identified from one or more images using image processing techniques. The images are conveniently infrared images, and the camera is an infrared camera. The relationship of the potential target object to the topography can be a movement of the potential target object relative to the topography, the intersection of a line of sight with the topography, and / or the movement of an intersection of the line of sight through the topography.

[0013] The camera can be stationary relative to the topography. Alternatively, it can be moved relative to the topography, for example, in a flying aircraft. It can also be pivotable, conveniently both relative to the topography and relative to a vehicle axis, such as a missile axis.

[0014] If the potential target object is identified as such, it can be tracked, for example, using a camera or the flight of a missile or aircraft. If it is determined that the potential target object is not actually a target object, it can be discarded as a target object.

[0015] According to the invention, the line-of-sight information is the intersection of the camera's line of sight to the potential target object with the topography. Although the distance between the camera and the potential target object is typically unknown, the potential target object can be assigned a fictitious location on the topography using the intersection point, or a fictitious object can be defined as another object located at the intersection point. In the following, the intersection point is considered a fictitious object, and its movement is tracked. For clarity, the intersection point will be referred to as an object instead of an object.

[0016] Depending on the movement of the intersection point from one image to another, a decision is made as to whether the potential target object is discarded or recognized as a target object and, for example, tracked. To do this, the movement of the intersection point can be compared with other line-of-sight information. This can be done by comparing the movements of the intersection point and the potential target object, or by comparing the line-of-sight information of the intersection point and the potential target object, for example, their line-of-sight rotation rates.

[0017] The topography can be the earth's surface, a glacier's surface, a water surface, or another topography. Even a cloud surface is considered if clouds are flown over and their position and shape are known. In principle, it doesn't matter which topography is used, as long as the topography is at least roughly known. If, for example, the topography is a cloud surface over which an aircraft with the camera flies, an aircraft flying between the camera and the cloud surface can be reliably detected by the method according to the invention, even if a large number of solar reflections create interference signatures in the images.

[0018] Advantageously, the topography is a terrain model. The topography is thus a landscape topography, preferably that of the landscape imaged by the camera. The topography is advantageously known from geographical or cartographic data stored in a unit that is connected to the camera via signal technology or can be retrieved by the camera.

[0019] It is also possible to determine the topography from sensor data even if it is not known in advance, for example, when the topography is calculated from the movements of a large number of objects through the images. If the camera's own motion is known, for example, its movement in geodetic coordinates, the topography itself can be calculated from the movement of a large number of objects defined as lying on the topography.

[0020] To facilitate the calculations, it is useful to determine the line-of-sight information in geostationary coordinates, for example, geodetic coordinates. If the camera is a moving camera, the camera's own coordinate system can be converted into a geostationary coordinate system.

[0021] In one embodiment variant that does not fall within the claimed scope of protection, the line of sight information is the intersection point of the camera's line of sight to the potential target object with the topography. If the intersection point of the line of sight to the same target object is determined from two images, it can be calculated whether the two intersection points are identical or differ. If they differ, the intersection point of the line of sight with the topography moves through the topography from one image to the next. The target object therefore moves through or over the topography and can therefore be recognized as such. Generally speaking, depending on the two intersection points, a decision can be made as to whether the potential target object is rejected or recognized as a target object as such.

[0022] If the camera is moving, a known relative movement of the camera across the topography can be used to determine the line-of-sight information. The known relative movement of the camera across the topography allows the movement of the topographic image through the images to be calculated. If the movement of the potential target object deviates from the topographic movement, it can be identified as such. Or, more generally, the line-of-sight information to the target object can be determined from the movement of the target object image through multiple images.

[0023] In one embodiment of the invention, the line-of-sight information is a line-of-sight rotation rate. The line-of-sight rotation rate is determined from a calculated movement of an intersection point of the camera's line of sight with the potential target object, and the calculation is performed using a known relative movement of the camera across the topography. From the known relative movement, the line-of-sight rotation rate of a fictitious object located in the topography can be determined. If the line-of-sight rotation rate of a potential target object deviates from this in the images, it can be identified as such.

[0024] The line of sight rotation rate of a point in the topography or fictitious object is determined from the position of the potential target object and the camera's own movement relative to the topography from a single image. If the potential target object visible in the image is determined to be located in the topography, or if the location in the topography is selected that lies at the intersection of the line of sight with the topography, the line of sight rotation rate of this location can be determined from the known relative movement of the camera over the topography. To reliably detect the target object, two pieces of line of sight information are determined. One piece of line of sight information can be determined from the position of the potential target object in two images, for example by determining the line of sight rotation rate of the potential target object from one image to the other.The other line-of-sight information can be determined from a calculated movement of an intersection point between the camera's line of sight to the potential target object and the topography. For example, the line-of-sight rotation rate of the landscape point behind the potential target object is determined from the camera's own motion. The target object can now be recognized as such if the two pieces of line-of-sight information—in the above example, the two line-of-sight rotation rates—deviate from each other by more than specified.

[0025] In terms of line-of-sight rotation rates, it is advantageous to determine the line-of-sight rotation rate of the potential target object from two images, determine the line-of-sight rotation rate of an intersection point of the line of sight with the topography, and compare the two line-of-sight rotation rates. If these differ by more than specified, the potential target object can be identified or categorized as a target object.

[0026] In a further embodiment of the invention, the line-of-sight information is a line-of-sight rotational acceleration of the line of sight from the camera to the potential target object from one image to, in particular, at least two subsequently acquired images. Analogous to the above description, the target object can be recognized as such if the line-of-sight rotational accelerations differ from each other by more than specified.

[0027] The invention is also directed to a guided missile according to claim 5, having a seeker containing a camera and a control unit configured to control the camera's recordings, evaluate the recorded images using an image processing program, and identify an object as a potential target object from at least one image. In order to detect the target object even in images with numerous interference signatures, the control unit is configured according to the invention to determine line-of-sight information of the potential target object from the images in relation to a topography assumed to be known, and to identify the target object as such based on the line-of-sight information.

[0028] The description of advantageous embodiments of the invention given so far contains numerous features, some of which are summarized in several dependent claims. However, the features can also be expediently considered individually and combined into useful further combinations, particularly in the case of claims that refer back to one another, so that an individual feature of a dependent claim can be combined with an individual, several, or all features of another dependent claim. Furthermore, these features can each be combined individually and in any suitable combination both with the method according to the invention and with the device according to the invention according to the independent claims. Thus, method features can also be viewed as objectively formulated properties of the corresponding device unit, and functional device features can also be viewed as corresponding method features.

[0029] The above-described properties, features, and advantages of this invention, as well as the manner in which they are achieved, will become clearer and more readily understood in conjunction with the following description of the embodiments, which are explained in more detail in conjunction with the drawings. The embodiments serve to illustrate the invention and do not limit the invention to the combination of features specified therein, including with regard to functional features. Furthermore, suitable features of each embodiment can also be explicitly considered in isolation, removed from one embodiment, incorporated into another embodiment to supplement it, and / or combined with any of the claims.

[0030] They show: FIG 1 shows a guided missile with a camera that records multiple objects, FIG 2 shows the guided missile at a later time when a later image of the objects is recorded, and FIG 3 shows the guided missile with the camera at a later time looking at an object.

[0031] FIG 1 shows a guided missile 2 with a seeker 4 comprising a camera 6 with infrared optics 8 and an infrared detector 10. Connected to the camera 6 is a control unit 12 for evaluating the infrared images captured by the camera 6 and for deriving guidance commands from the evaluation results, which are used to control guidance vanes 14. This allows the guided missile 2 to be guided toward a target and deploy an active element 16 there.

[0032] The guided missile 2 flies over a landscape 18, the topography 20 of which is stored in a database 22 connected to the control unit 12 via signal transmission. The topography 20 is a model of the landscape 18. Via a navigation system for determining position and attitude 24, the control unit 12 regularly receives information about its own location, its own flight speed, and the flight direction and can thus determine the current position of the guided missile 2 relative to the stored topography 20. The navigation system 24 can include a GNSS receiver.

[0033] During the flight of the guided missile 2 over the landscape 18, the camera 6 takes several infrared images of the landscape 18. The control unit 12 evaluates these images using image processing software and identifies, for example, from one of the images, several potential target objects 26a-26f, each of which could represent a vehicle. However, it is initially unclear whether the target objects 26a-26f actually represent vehicles or, for example, areas of terrain that radiate strong heat, such as a heated rock or a glacier point reflecting sunlight. To determine whether one or more of the potential target objects 26a-f represents an image of a vehicle, the guided missile 2 or the control unit 12 can now perform one or more of the following processes.

[0034] In FIG 1 Three lines of sight 28a-c from camera 6 to three of the potential target objects 26a-c are shown as examples. The control unit 12 can now select one or more of these target objects 26a-c to check whether it is an image of a vehicle and thus a target object 26a-c as such. The control unit 12 then selects the potential target object 26b for examination as an example.

[0035] The control unit 12 determines line-of-sight information to the potential target object 26b. This line-of-sight information is in the embodiment of FIG 1 the intersection point 30 of the line of sight 28b with the topography 20. To determine this intersection point 30, the control unit 12 determines the direction of the line of sight 28b from the camera 6 to the target object 26b in geodetic coordinates. To achieve this, the position of the object 26b in the recorded image is first determined. With this and the known orientation of the camera 6 relative to the missile axis, the direction of the line of sight 28b relative to the missile axis can be determined. Since the flight direction and position of the guided missile 2 are also known, the direction of the line of sight 28b can be converted from the missile's own coordinate system into the geodetic coordinate system. By forming the intersection point of the line of sight 28b with the topography 20, the intersection point 30 can now be determined. During the flight of the guided missile 2, the camera 6 takes several images of the potential target object 26b.The recording times are chronologically successive, so that they are recorded from different positions of the moving camera 6 relative to the topography 20. This is exemplified by the illustration in . FIG 2 shown.

[0036] FIG 2 shows the guided missile 2 at the same time as in FIG 1 and additionally at a later point in time, whereby only the front part of the guided missile 2 is indicated. From the previous point in time to the later point in time, the guided missile 2 moves from a previous to a later position, which is indicated by the arrow on the guided missile 2 in FIG 2 During this time, the potential target object 26b has also moved relative to the landscape 18, as indicated by the arrow between the representations of the target object 26b in FIG 2 In the later image, the target object 26b appears at a new location in the image of the landscape 18, as indicated by the intersection point 32 in FIG 2 is indicated.

[0037] It could now be assumed that the potential target object 26b could already be identified as such from this relative movement through the image of the landscape 18. However, this is not very reliable, since the images of the potential target objects 26a-f can change significantly relative to one another during the flight of the guided missile 2, especially when the guided missile 2 flies very close to a mountainous landscape 18 or toward the landscape 18. All potential target objects 26a-f move unpredictably relative to one another in the images, so that it cannot be reliably determined from their relative movement which of the potential target objects 26a-f could represent a vehicle.

[0038] To circumvent this uncertainty, the control unit 12 calculates the intersection point 32 of the line of sight 28b with the topography 20 from the known line of sight 28b and the known topography 20. With the two intersection points 30, 32, two different lines of sight information of the potential target object 26b are now available from two consecutively acquired infrared images. There are now several possibilities for recognizing the potential target object 26b as a target object 26b as such. One of these, which does not fall within the claimed scope of protection, is described below using FIG 3 explained.

[0039] FIG 3 shows the missile 2 at the same two times as at FIG 2 From the two intersection points 30, 32, the control unit 12 can now calculate whether the intersection point 30 has moved through the topography 20. As can be seen from FIG 3 visible, this is certainly the case, since the intersection point 32 is a significant distance from the intersection point 30 within the topography 20. If this distance is greater than a predetermined maximum distance value, the target object 26b is considered to be a target object 26b as such. If a target object 26b is recognized as such, the control unit 12 can guide the guided missile 2 such that the guided missile 2 approaches the target object 26b, for example, to hit it or to detonate its active element 16 in its vicinity.

[0040] Another procedure is shown in FIG 2 explained. For this purpose, the calculation of the second intersection point 32 is not necessary. The sight rotation rate 34 of the sight line 28b between the two time periods is determined. For clarification, FIG 2 the direction of the original line of sight 28b, as shown in FIG 3 with the right line of sight 28b is shown, in FIG 2 shown dashed. At a later time, this line of sight 28b has changed from the dashed line of sight, as shown in FIG 2 This change in direction as a function of the time between the two recording times of the two images is the line of sight rotation rate 34, which is FIG 2 is shown by the small double arrow.

[0041] This line-of-sight rotation rate 34 is now compared with the line-of-sight rotation rate 36 of the line of sight 28b from the camera 6 to the intersection point 30. In this case, the direction of the line of sight 28b from the camera 6 to the intersection point 30 at the later time is not determined from the later image, but rather calculated from the proper motion of the guided missile 2 or the camera 6 over the topography 20. Since this proper motion or relative motion is known, the control unit 12 calculates from the known later time the direction in which the intersection point 30, which was determined from the image at the earlier time, lies at the later time. This direction 38 is in FIG 2 and does not represent a line of sight in the strict sense, but is a calculated direction 38 starting from the camera 6. The angle between the line of sight 28b of the first time, which is in FIG 2shown in dashed lines, to the direction 38 can now be divided by the time period between the two recording times and then forms the fictitious line of sight rotation rate 36 of the intersection point 30. If the two line of sight rotation rates 34, 36 differ from each other by more than a predetermined value, the potential target object 26b can be recognized as a target object 26b as such.

[0042] Analogous to the line-of-sight rotation rate method, a line-of-sight acceleration can also be determined, i.e., the second derivative of the line-of-sight direction over time. Based on these two acceleration values, the target object 26 can be identified as such, namely if the two acceleration values ​​differ by more than a specified value. Like the line-of-sight rotation rates 34, 36, the corresponding line-of-sight accelerations can each be used as line-of-sight information to identify the target object 26b as such.

[0043] It is also possible to use the line-of-sight directions analogously to the above line-of-sight rotation rates or line-of-sight rotation acceleration rates. The two line-of-sight directions of the lines of sight 28b, 38 are compared, and the potential target object 26b is recognized as a target object if the two directions 28b, 38 deviate from each other by more than a predetermined value. As previously stated, the predetermined value does not have to be an absolute value, but can depend on one or more parameters, such as the movement speed of the camera 6 relative to the topography 20 and / or the temporal recording interval between the images. List of reference symbols

[0044] 2 Guided missile 4 Seeker 6 Camera 8 Infrared optics 10 Infrared detector 12 Control unit 14 Guided vane 16 Active body 18 Landscape 20 Topography 22 Database 24 Navigation system for determining position and attitude 26a-f Target object 28a-c Line of sight 30 Intersection point 32 Intersection point 34 Line of sight rate 36 Line of sight rate 38 Direction

Claims

1. Method for identifying a target object (26b), in the form of a vehicle on land, water or in the air, that is moving relative to a topography (20) by means of a camera (6), which is at rest or moving in relation to the topography (20), in which the camera (6) captures multiple images and an object from at least one of the images is determined as a potential target object (26a-f), characterized in that the images are used to determine line of sight information relating to the potential target object (26a-f) with respect to the topography (20), which is assumed to be known, and the line of sight information is taken as a basis for - either identifying the potential target object (26a-f) as the target object (26b) that is moving relative to the topography (20) - or rejecting the potential target object (26a-f) as an interference object that is stationary with respect to the topography (20) by capturing two images in succession, and: a) using a first image to determine line of sight information as the intersection (30) of the line of sight from the camera (6) to the potential target object (26a-f) with the topography (20) and determining line of sight information as a first line of sight rotation rate (34) of the line of sight (28b) between the two instants of the photographs as a change in the direction of the line of sight (28b) as a function of the time between the two instants of the photographs, the line of sight (28b) in each case leading from the camera (6) to the potential target object (26a-f), and accordingly ascertaining line of sight information as a second line of sight rotation rate (36) of the line of sight (28b, 38) from the camera (6) to the intersection (30), the direction of the line of sight (38) from the camera (6) to the intersection (30) at the later time not being ascertained from the later image, but rather being determined mathematically from the intrinsic movement of the camera (6) over the topography (20) and - the potential target object (26a-f) being identified as the target object (26b) that is moving relative to the topography (20) if the two line of sight rotation rates (34, 36) deviate from each other by more than a predetermined value, - and otherwise the potential target object (26a-f) being rejected as the interference object that is stationary with respect to the topography (20), or b) in the variant according to a), instead of the line of sight rotation rates (36, 38), ascertaining respective line of sight information as a line of sight acceleration as the second derivative of the line of sight direction over time and taking these two acceleration values as a basis for - identifying the potential target object (26a-f) as the target object (26b) that is moving relative to the topography (20) if the two acceleration values deviate from each other by more than a predetermined value, - and otherwise rejecting the potential target object (26a-f) as the interference object that is stationary with respect to the topography (20).

2. Method according to Claim 1, characterized in that the topography (20) is a terrain model.

3. Method according to either of the preceding claims, characterized in that the line of sight information is determined in geodetic coordinates.

4. Method according to either of the preceding claims, characterized in that in variant a) a known relative movement of the camera (6) over the topography (20) is used to determine the line of sight information.

5. Guided missile (2) comprising a seeker (4) containing a camera (6) and a control unit (12) that is prepared to control the capture by the camera (6), to evaluate the captured images by means of an image processing program and to determine from at least one image an object as a potential target object (26b), characterized in that the control unit (12) is configured to carry out the method according to one of the preceding claims.