Methods, and apparatus for configuration of a projectile launching device based on a ballistic model and passive sensing

Abstract

According to one aspect of the present disclosure, there is provided a computer-implemented method of determining a configuration of a projectile launching device, the method includes receiving first data from a passive sensor; determining a first location of a detected object based on the first data; determining a second location based on the first location; and determining a configuration of the projectile launching device to propel a projectile to collide with the detected object in the second location.

Description

Technical Field

[0001] The present disclosure relates to systems, methods, and computer program products for controlling projectiles launching devices based on a Ballistic Model of the projectile and information provided by a passive sensor.Background

[0002] A problem in Counter Unmanned-Aircraft Systems (C-UAS), is the emergence of new types of weapons, such as small and agile drones, which have a "pop-up threat" character, in other words they can fly "under the radar." These new weapons have a small size, low radar cross section, and low signal-to-noise ratio. Additionally, they can be surprisingly fast, reaching up to hundreds of km / h, and they can perform sudden evasive manoeuvres. The combination of these features renders them hard to detect and hard to hit.

[0003] A known solution to counter the threat posed by new weapons includes utilizing high-cost defence systems, such as air defence missiles, which may cost multiple hundreds of thousands of Euros. Since the cost of drones and other new weapons is at most in the tens of thousands of Euros, they put defenders in an economically asymmetric position.

[0004] Therefore, there is a need for low-cost systems and methods that enable the detection of these new weapons and that disable these weapons with low-cost counter ammunitions.Summary

[0005] The present invention is defined in the independent claims. The dependent claims recite selected optional features. In the following, each of the described methods, systems, apparatuses, examples, and aspects, which do not fully correspond to the invention as defined in the claims is thus not according to the invention and is, as well as the whole following description, present for illustration purposes only or to highlight specific aspects or features of the claims.

[0006] A first aspect of the present disclosure relates to a computer-implemented method of determining a configuration of a projectile launching device. An exemplary projectile launching device may be a weapon, for example a machine gun, which may be included in a weapon station. The method comprises receiving first data from a passive sensor and determining a first location of a detected object based on the first data. The detected object may be a drone or a type of weapon or any other threat to the weapon station. Further, the method comprises determining a second location of the detected object based on the first location. The second location may be a future location of the detected object. The determined first and second locations may be locations of the detected object relative to the projectile launching device, respectively. Alternatively, the first and second locations may be locations of the detected object relative to another coordinate origin, respectively. Further, the method comprises determining a configuration of the projectile launching device to propel a projectile to collide with the detected object in the second location. The passive sensor may be an electrooptical sensor, such as camera, in particular including a two-dimensional sensor (array).

[0007] The method is directed to the determination of a suitable configuration of a projectile launching device to hit an object detected by the passive sensor where the object may be potentially moving. Examples of the object may include a drone or other types of weapons.

[0008] The method may be preferably executed by a computing device that may be coupled with the passive sensor and the projectile launching device. The coupling above may enable the computing device to determine the configuration of the projectile launching device that may enable the configured projectile launching device to hit the detected object at the second location.

[0009] The first data may be the result of a single observation performed by the passive sensor, or the first data may be derived by a detection and / or tracking process, such as a single object tracking (SOT) process that may receive data or observations from the passive sensor and derive the location of the detected object and possible other information such as velocity and direction of the detected object. The detection and / or tracking process may be performed by computing resources included in the passive sensor or by the computing device.

[0010] In some examples, the second location may be determined based on an expected trajectory of the detected object, in particular a direction of movement and / or velocity of the detected object.

[0011] The second location may be determined by dead reckoning based on the direction and velocity of movement of the detected object at the first location in combination with a determined time interval to the second location. The determination of the second location may also include a model of behaviour of the detected object to estimate changes of the direction of movement and / or velocity of the detected object during the time interval. Behavioural models may be based on knowledge of behaviour of the detected object or based on pre-trained AI models or learned based on AI based models and experience with the detected object.

[0012] In some examples, determining the configuration of the projectile launching device comprises determining a trajectory of the projectile to the second location.

[0013] In some examples, the trajectory of the projectile may be based on at least one of: an aerodynamic drag of the projectile, a drop of the projectile, and atmospheric conditions.

[0014] The atmospheric conditions may include one or more of a humidity, an air pressure, and / or a wind.

[0015] The trajectory of the projectile may also be based on a ballistic model of the projectile and the projectile launching device. The ballistic model may leverage on the detected object trajectory and additional projectile launching device and projectile statistics to provide a configuration of the projectile launching device with an aim correction which compensates the drag and drop of the projectiles and the motion of the detected object to reliably hit the detected obj ect.

[0016] In some examples, determining the configuration of the projectile launching device comprises determining target pan and tilt angles of the projectile launching device based on the trajectory of the projectile.

[0017] The configuration of the projectile launching device determines where the projectile launching device points to, consequently it will affect the trajectory of the projectiles. Thus, the determined configuration may need to match the expected trajectory of the projectile.

[0018] The projectile launching device may include a launcher that directs the projectiles during propulsion. If the projectile launching device is an exemplary machine gun, the launcher may be the barrel of the machine gun. The target pan and tilt angles comprised in the configuration of the projectile launching device may also be considered to be the tilt and pan angles of the launcher included in the projectile launching device, where tilt angle may relate to an angle or an inclination of the launcher in the vertical direction, and pan angle to an angle or a deviation in the horizontal direction.

[0019] In some examples, determining the configuration further includes an estimate of a tilt rotation time and / or a pan rotation time of the projectile launching device from first tilt and pan angles to the target tilt and pan angles.

[0020] Configuring the projectile launching device may further include rotating the projectile launching device from a first initial position to a second target position. Planning this rotation may include making estimates of the rotation time of the projectile launching device from the first position to a second target position. The rotation time may be decomposed in a tilt rotation time that is the time required to rotate the projectile launching device from a first tilt angle to the target tilt angle; and the pan rotation time that is the time required to rotate the projectile launching device from a first pan angle to the target pan angle. The projectile launching device rotation time may depend on the physical construction of the projectile launching device; for example, the projectile launching device may include a gimbal that control the position of the launcher. Thus, the pan and tilt rotation times may depend on the velocity rates for a two-axis gimbal that may control the position of the projectile launching device.

[0021] In some examples, the configuration further includes an estimate of a total execution time, wherein: the total execution time may be based on the tilt rotation time, the pan rotation time; and / or an estimate of a flight time of the projectile from the projectile launching device to the second location and / or an estimate of the lock time that is the time that the propelling mechanism of the projectile launching device requires to propel the projectile.

[0022] The total execution time may further be based on processing latencies of any involved computing process, such as neural network computations (e.g. for object detection, tracking and / or classification) and / or sensor data capture processes, such as for example the electrooptical sensor image capture process.

[0023] The total execution time is relevant because it may indicate whether the projectile arrives in time to hit the target. The total execution time may indicate the time required to rotate the projectile launching device from a first position to a target position that may correspond to the position from which to propel the projectile, or multiple projectiles, to the detected object. The total rotation time of the projectile launching device may depend on how the projectile launching device rotates from a first position to a second position. In some cases, the pan rotation may precede the tilt rotation, or vice versa, thus the total rotation time may be computed as a function of the sum of the tilt rotation time and the pan rotation time; in other examples, the total rotation time of the projectile launching device may be computed as a function of the maximum of the tilt rotation time and the pan rotation time, in particular when the pan rotation and the tilt rotation occur simultaneously and / or concurrently.

[0024] In some examples, the computer-implemented method may further include: determining a time interval needed for the detected object to reach the second location; wherein the second location is determined such that the time interval exceeds the estimate of the total execution time.

[0025] Upon estimating the trajectory of the detected object, the computing device may determine the second location along the trajectory. The determination of the second location may depend on the total execution time to enable the projectile launching device 114 to reconfigure before the detected object 140 reaches the second location. In other words, the total execution time may be equal or shorter than the time required to the detected object 140 to reach the second location.

[0026] The determination of the second location may depend on the velocity of rotation of the projectile launching device. The faster the rotation velocity of the projectile launching device, the smaller distance between the first and the second location may be.

[0027] In some examples, the computer-implemented method may further include receiving second data from the passive sensor; determining the location of the detected object in the second data and determining whether the second data may be indicative of any damage to the detected object.

[0028] Once determined, the configuration of the projectile launching device may be transferred to the projectile launching device that may then propel the projectile, or multiple projectiles, towards the detected object. The second data may indicate of the status of the detected object after the propulsion of the projectile. Based on the second data, computing device may determine whether the detected object has been damaged or destroyed or whether the projectile launching device may need to be reconfigured to attempt to hit the detected object again.

[0029] In some examples, the computer-implemented method may further include determining a third location based on the second location and / or the second data.

[0030] In some examples, the computer-implemented method may further include determining a second configuration of the projectile launching device according to any of the previous examples using the second location as the first location; the third location as the second location. In other words: any of the previous examples may be performed iteratively.

[0031] The projectile launching device may identify a third location temporally following the second location of the detected object. The third location may be a location in which the detected object may be at the third time. The determination of the third location may be analogous to the determination of the second location with the only difference that the third location may be determined based on the second location or on the second data, or the second data. In some examples, the third location may be determined based on an expected direction and velocity of movement of the detected object at the second location or at a location derived from an indication of the position of the detected object based on the second data.

[0032] Further, the third location may be associated with a second configuration of the projectile launching device so that the projectile launching device may aim at the detected object in that third location.

[0033] Consequently, a plan may be constructed that may indicate a sequence of configurations directed to multiple points in which the detected object may be. The execution of the plan may require continuous relative adjustments of the configurations of the projectile launching device in view of the changes of location, direction, and velocity of the detected object. The computing device may perform these continuous configuration adjustments based on a Model Predictive Control (MPC) algorithm to minimize the rotations of the launcher of the projectile launching device while maintaining the projectile launching device pointing toward the detected object.

[0034] One advantage of MPC is that it may leverage predictions of the detected object motion and adapt the position of the projectile launching device to these changes. Furthermore, to adapt to the irregular and possibly low frequency of sensor information, the MPC algorithm may be combined with a continuous-time Kalman filter that processes sensor data at irregular time intervals to estimate the gun position considering rotation inertia even when no new sensor data provides updates on the position of the detected object.

[0035] In some examples, the passive sensor includes an electrooptical sensor, preferably an infrared camera or any other camera; and the first data and / or second data includes one or more images.

[0036] The passive sensor may be an electrooptical sensor and the data includes one or more images from the electrooptical sensor. Electrooptical sensors may enable efficient image processing to detect and recognize objects in the image and locate the objects in space.

[0037] In some examples, determining the first location includes determining a type of the detected object; calculating a distance between the passive sensor and the detected object based on the type of the detected object. In some examples, the type, in particular the physical size, of the object is determined by a (machine learning) object classification tool, e.g., including an object classification AI model,

[0038] In some examples, determining the first location may further include determining a physical size of the detected object based on the type of the detected object and / or the size of the detected object in the image. In some examples, the trajectory and / or a distance between the passive sensor and the detected object may be determined based on the physical size of the detected object.

[0039] In some examples, determining the first location may be further based on the passive sensor parameters.

[0040] The determination of the type of the detected object may enable a determination of the action to perform in reaction to the detection of the object. The determination of the type of the object may also enable the determination of the distance between the sensor and the detected object.

[0041] The determination of the type of the detected object may also enable the determination of the of the physical size of the detected object based on some list of the possible known objects, where the list may be a database of objects, or the list may be implicit in a classification system that given the image of the detected object reports the detected object type.

[0042] The determination of the size of the detected object may enable the determination of the location of the detected object in a 3D space based on the physical size of the detected object, the size of the detected object in the image and the intrinsic and extrinsic properties of the electrooptical sensor. Intrinsic properties may include parameters such as focal length, the field of view, aperture, resolution. Extrinsic properties of the electrooptical sensor may include camera location and orientation.

[0043] It is preferred to determine a bounding box enclosing the detected object based on the first data. This allows the determination of a size the detected object in the image. It is further preferred to determine the first location information based on the size the detected object in the image and the physical size of the detected object. The first location information is preferably indicative of a distance between the passive sensor and the detected object.

[0044] Determining the first location preferably includes processing the first data by a single object tracking algorithm. The single object tracking algorithm may determine the bounding box.

[0045] In some examples, the detected object may be a drone.

[0046] The detected objects may be drones, but other detected objects may also be possible, such as other weapons, stationary or non-stationary sensors or other types of manned or unmanned devices.

[0047] A second aspect of the present disclosure relates to a system including one or more processors and one or more storage devices, wherein the system may be configured to perform the computer-implemented method of any preceding examples.

[0048] A third aspect of the present disclosure relates to a computer program product for loading into a memory of a computer, including instructions, that, when executed by a processor of the computer, cause the computer to execute a computer-implemented method of any preceding examples.Brief description of the drawings

[0049] The features, objects, and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings in which like reference numerals refer to similar elements. Fig. 1 is a schematic drawing illustrating an exemplary weapon station; Fig. 2 is a schematic drawing illustrating the detection of an object in a first location and the prediction of a second location of the object; Fig. 3 is a schematic top view of an exemplary configuration of the projectile launching device; Fig. 4 is a schematic side view of the exemplary configuration of the projectile launching device; and Fig. 5 is a flow chart of an example of a method of determining a configuration of a projectile launching device. Detailed description of the preferred embodiments

[0050] Fig. 1 schematically illustrates an exemplary weapon station 110.

[0051] The weapon station 110 may include at least one passive sensor 112, a computing device 114 and a projectile launching device 116. The passive sensor 112 may be communicatively coupled with the computing device 114 and in turn the computing device 114 may be communicatively coupled with the projectile launching device.

[0052] The passive sensor 112 may perform one or more observations of the environment within a range of view 130 that may depend on the sensor. Further, the passive sensor 112 may produce data codifying the observations performed and transmit the data to the computing device 114.

[0053] When an exemplary object 140 is present in said range of view 130, then passive sensor 112 may observe and detect object 140 and produce first data about the detected object 140. In particular, the first data may be indicative of the detected object 140.

[0054] Passive sensors, such as sensor 112, are sensors that may detect signals or energy emitted or reflected directly or indirectly by the objects in the range of view of the sensor. Passive sensors differ from active sensors, such as radars that actively transmit energy to detect objects. Since passive sensors do not transmit any signals or any energy, but rather passively detect signals produced by other objects, enemy objects may not detect any signal revealing the existence of the passive sensor, thus passive sensors tend to be invisible to enemy objects.

[0055] Examples of passive sensors may include electro-optical sensors, such as a camera for visible or infrared light. Other examples of passive sensors may include inter alia microphones for noise sensing, thermal sensors, sensors detecting changes in the electric field, sensors detecting chemical compounds, or seismic sensors. Additionally, passive sensor110 may be a sensor network of passive sensors.

[0056] The data produced by the passive sensor 112 may depend on the type of sensor. In the specific case of electro-optical sensors, the data produced by passive sensor 112, such as the first data, may include one or more images. Passive sensor data, such as the first data, may also include meta-data about the data. Examples of meta-data may include extrinsic parameters such as temporal or location information indicating when and where the data has been collected. Other types of meta-data may include intrinsic parameters of the sensor. In the exemplary case of electro-optical sensors, intrinsic sensors parameters may include inter alia focal length, aperture, field-of-view, resolution.

[0057] The computing device 114 may be configured to receive data about object 140 from the passive sensor 112 and process the received data. The data transmission from the passive sensor 112 to the computing device 114 may be based on different protocols. In some cases, the passive sensor 112 may stream data to the computing device 114, enabling the passive sensor 112 to perform an observation, and immediately transmit the data related to the observation to the computing device; thus, tightly coupling data sensing and processing. However, the data transfer may also be based on other protocols that may enable data transfer in batches to be processed serially, or other transfer processes or policies may be utilized depending on the specific application requirements.

[0058] Upon receiving data from the passive sensor, the computing device 114 may process the data. Such processing may include determining whether the detected object 140 may pose a threat, estimating the location and movement of the detected object 140 and estimating how to configure the projectile launching device 116 to hit and disable the detected object 140.

[0059] The computing device 114 may include one or more physical and / or virtual computing platforms that may preferably be in the proximity of the passive sensor 112; but it may also be in a cloud geographically distant from the passive sensor 112. The computing device may also be a hybrid combination of local computing platforms, such as a computer positioned in the proximity of the passive sensor 112, and virtual computers distributed in geographically distinct clouds.

[0060] The computing device 114 may also include any type of physical computing platforms including single board computers, mobile devices, and / or tablets. The computing device may be a computing platform embedded in the passive sensor 112 or in the projectile launching device, or embedded in other components that may be included in the weapon station. The computing device 114 may also be embedded in one or more devices of the weapon station 110. For example, functionalities related to the sensor data processing may be embedded in the passive sensor 112, while the functionalities related to the configuration of the projectile launching device may be embedded in the projectile launching device itself.

[0061] The projectile launching device may include a launcher emitting, for example propelling, projectiles. More generally, a projectile launching device may be a weapon and the launcher may be the barrel of the weapon. Examples of projectile launching devices may include any device configured or configurable to launch visible, in particular traceable, ammunition, such as a machine gun, a cannon, or a grenade launcher for unpropelled grenades. Since the projectiles are propelled by the launcher in these cases, the disclosed projectile launching device may reduce the cost and complexity of the weapon station, and in particular the cost and complexity of the projectiles may be reduced because the projectiles may not require complex hardware or software to pursue their target as well as hardware and software to be self-propelled. The projectile launching device may also comprise a launcher for an unguided propelled projectile, such as a rocket-propelled grenade, or an unguided rocket.

[0062] The coupling between the passive sensor 112, the computing device 114, and the projectile launching device 116 may be based on any network configuration that may enable data transmission. Thus, the coupling may be based on wired or wireless networking; it may further be based any Local Area Network infrastructure, Wide Area Network infrastructure, or Satellite communication infrastructure.

[0063] The passive sensor 112, the computing device 114 and the projectile launching device 116 may be tightly connected and be partially or entirely embedded systems. Thus, for example, the passive sensor may be placed on the projectile launching device or embedded in the projectile launching device. Similarly, the computing device may be an embedded system. Thus, the coupling between the components of the weapon station 110 may be at least partially based on a network on chip communication system as well as any on-chip or peripheral communication infrastructure.

[0064] Fig. 2 illustrates the detection of object 140 in a first location and the prediction of a second location of the object 140. Fig. 2 analogously to Fig. 1 includes the weapon station 110 which in turn includes the passive sensor 112, the computing device 114 and the projectile launching device 116 as discussed above. The passive sensor 112 may detect the object 140 and may transmit first data about the detected object 140 to the computing device 114 that is coupled with the passive sensor 112.

[0065] The transmission of the first data from the passive sensor 112 to the computing device 114 may be part of a single object tracking (SOT) process to track objects detected by the passive sensor 112, such as detected object 140. The SOT may be performed by the passive sensor 112 or by the computing device 114 that may provide to the passive sensor 112 indications to track the detected object 140 and indications on how to modify the range of view to track the detected object 140.

[0066] Upon receiving the first data from the passive sensor 112, the computing device 116 may attempt to recognize the detected object 140 and determine a type of detected object 140. This determination may be part of SOT, and it may be based on a pretrained AI recognition / classification model. Further, based on the recognition of the detected object 140 and of its type, the computing device 114 may determine whether the detected object 140 may poses a threat.

[0067] The determination of the type of the detected object may also enable the determination of the properties of the detected object, such as its physical size, and other properties. This determination may be based on a list of known objects, such as a list of known drones and their properties. This list may be a lookup table or a database including information about drones, or the list may be implicit in the AI recognition / classification model.

[0068] The determination of the physical size of object 140 may enable the determination of the location 232 of the detected object 140 and the determination of the direction and velocity of movement of the detected object 140. The determination of the location and movement of the detected object may be based on the determined physical size of object 140, the features of the object 140 in the data and on the properties of the passive sensor.

[0069] If for example, the passive sensor is an electrooptical sensor, preferably an infrared camera; the data includes one or more images. The determination of the type of the detected object may be based on the application of image recognition, which may be based on a pretrained AI recognition / classification model. The AI model may also provide information such as the physical size of the object. The calculation of the size of the object in the image may be performed by computing the bounding box of the object in the image. The location of the detected object may be estimated based on the extrinsic parameters of the electrooptical sensor such as location and orientation and intrinsic parameters of the electrooptical sensor such as focal length, the field of view, aperture, resolution.

[0070] The computing device 114 may determine that object 140 has been detected at location 232. Location 232 may be considered a first location of the object 140. This location may correspond to the location of the object at a first time, which may be the instant in which the object has been detected by the passive sensor. The computing device may also determine a second location 234, that may correspond to a location that the detected object 140 may occupy at a second time, which may temporally follow the first time, in other words the second time is in the future with respect to the first time.

[0071] The second location 234 of the detected object 140 may be determined based on the current location 232 of the detected object 140 and knowledge about the detected object 140 such as current velocity and direction of motion 242 of the detected object 140 and an expected motion of the detected object. In case the expected motion of the detected object may be assumed to be constant in direction and velocity, thus the second location may be estimated based on the physical laws of motion. However, other estimation methods may be used such as method referring to behavioural models that may attempt to predict unexpected changes of direction and velocity of the detected object. The second location may also be extracted from a Kalman filter that may also be used to improve the reliability of the passive sensor data.

[0072] The estimate of the second location may also depend on a determination of the second time that may specify when to aim at the detected object. The second time may need to be sufficiently far in the future to take into account delays due to the reconfiguration of the projectile launching device, thus, enabling the projectile launching device to reconfigure and to aim to the detected object.

[0073] Upon determining the second location, the computing device may determine one or more configurations of the projectile launching device so that the projectile launching device may point to the detected object and propel projectiles to collide with and disable the detected obj ect.

[0074] Fig. 3 is a schematic top view of an exemplary configuration 310 of the projectile launching device 116. The direction of view is indicated by coordinate system 312. Fig. 3 relates to the pointing of the launcher included in the projectile launching device 116 to the detected object with the objective to hit the object and disable it.

[0075] Fig. 3 relates to the horizontal rotation of the launcher of the projectile launching device 116. In Fig. 3, line 311 may represent a first position of the launcher, while line 313 may represent a target position of the launcher. Angle 316 may be considered to be the pan angle of the launcher indicating the deviation between two positions of the launcher in a horizontal direction.

[0076] Fig. 4 is a schematic side view of the exemplary configuration 310 of the projectile launching device 116 that illustrates vertical rotation of the launcher of the projectile launching device 116. The direction of view is indicated by coordinate system 322. Line 321 may represent a possible first position of the launcher of projectile launching device 116. Furthermore, line 323 may represent the target position of the launcher of projectile launching device 116. Angle 326 may represent the tilt angle of the launcher of the projectile launching device 116, where the tilt angle may indicate the deviation between the first and the second position of the launcher of device 116 in a vertical direction.

[0077] The determination of the configuration of the projectile launching device 116 may include the determination of the pan and tilt angles that match the expected trajectory of a trajectory of the propelled projectiles from the projectile launching device towards one of the second locations where the detected object 140 is expected to be.

[0078] The determination of the trajectory of the projectile may be based on the second location 234 and on at least one of: an aerodynamic drag of the projectile, a drop of the projectile, and atmospheric conditions, where the atmospheric conditions may include one or more of a humidity, an air pressure, and / or a wind.

[0079] The determination of the trajectory of the projectiles may be based on a ballistic model of the projectiles that may leverage the trajectory of the detected object and on additional gun and bullet statistics to provide pan and tilt angles with aim correction which compensates drag, bullet drop and target motion to reliably hit the target.

[0080] Configuring the projectile launching device may further include estimating an execution time. The execution time may be a function of the sum of: the time required to execute the rotation from the current position to the target position of the projectile launching device, an estimate of the lock time that is the time that the propelling mechanism of the projectile launching device requires to propel the projectile. The lock time may also be defined as the time required to propel the projectile, e.g., the time of rotation of a firing pin or a bolt of a gun, and the projectile's time of flight from the projectile launching device 116 to the second location 234 of the detected object 140.

[0081] Configuring the projectile launching device may include an abort condition in case the execution time may be shorter than the available time until the detected object 140 may be expected to be at the second position 234. The available time may be determined as the difference between the second time and the first time as described with reference to Figure 2.

[0082] An estimate of the rotation time of the projectile launching device may be further decomposed in estimating a tilt rotation time that may be the time required to perform the determined tilt rotation; and the pan rotation time that may be the time required to perform a pan rotation. The total rotation time of the projectile launching device may depend on the process of rotation of the device. For example, if the device rotates in a first direction, which may be the horizontally, or pan direction, and then in a second direction, which may be a vertically, or tilt direction, then the total rotation time may be a function of the sum of the pan and tilt rotation times. If instead the device rotates in both directions concurrently, then the time of rotation may be a function of the maximum of the pan or tilt rotation times. Additionally, other rotation policies may be used that may lead to other rotation time estimation functions to be used.

[0083] The total execution time may further include time related to processing latencies of any involved computing process, such as neural network computations (e.g. for object detection, tracking and / or classification) and / or sensor data capture processes, such as for example the electrooptical sensor image capture process.

[0084] The total execution time may be equal or shorter than the time required to the detected object 140 to reach the second location to enable to projectile launching device 114 to propel the projectiles to hit the detected object 140.

[0085] Once completed, the configuration parameters may be transmitted to the gun that could proceed to execute the received configuration and propel the projectiles towards the detected object.

[0086] The passive sensor may be further configured to perform at least one second observation of the detected object and thereby generate second data. The second data may have a format similar to the first data. The computing device may then receive the second data from the passive sensor. Based on the second data, the computing device may determine a new location of the detected object according to the second data and determine whether the second data is indicative of any damage to the detected object. Further, the second data may provide the basis for a decision as to whether to further damage the detected object or whether the detected object was already damaged or destroyed.

[0087] To further damage the detected object, the computer-implemented method may further include determining at least one third location based on the second location and / or the new location of the detected object. The third location may be the next location where to attempt to hit and damage the detected object. The third location may be further associated to a third time, where the third time temporally follows the second time.

[0088] The third location may then be considered to be analogous to the second location in the sense that the computing device may project a trajectory of the propelled projectiles to the third location and specify a new configuration of the projectiles launching device to hit the detected object at the third location according to the methods disclosed in relation to the second location. Thus, the third location may implicitly trigger an iterative process in which the passive sensor detects the object, the computing device projects the location of the detected object in the future and determines a configuration of the projectile launching device to disable the detected in the third location.

[0089] Since, the computing device may determine multiple second and / or third locations that may be distributed to be progressively further in the future. The computing device may be further configured to optimize the rotations of the projectile propelling device to reduce the delays between locations in which to damage the detected object thus improving the likelihood of damaging the detected object.

[0090] The optimization may be achieved by estimating a plurality of possible locations of the detected object 140 at multiple times, that may be considered to be multiple second and third times, and plan the rotations of the projectile launching device to propel projectiles to the detected object in as many of the estimated plurality of possible locations of the detected object 140 as possible. Furthermore, the optimization may require adapting the plan to react to unexpected changes of location of the detected object.

[0091] The computation device may construct and adapt the plan based on the MPC algorithm to minimize the rotations of the projectile launching device while maintaining the projectile launching device pointing toward the detected object. The updates of the positions of the projectile launching device 140 may depend on sensor data indicating the location of the detected object and thus the adjustments to be made to the projectile launching device.

[0092] Since sensor data may be received irregularly depending on the ability of the sensor to detect and track the detected object, the MPC algorithm may be combined with a continuous-time Kalman filter that processes sensor data at irregular time intervals to estimate the location of the detected object based on the sensor data and the expected location of the detected object. Further, the estimated locations of the detected object are transformed into configurations of the projectile launching device while considering rotation inertia of the projectile launching device.

[0093] Fig. 5 illustrates an example of a method 500 of determining a configuration of a projectile launching device. Method 500 may be executed by the computing device 114 of the weapon station.

[0094] In step 510, the computing device receives first data from a passive sensor that may be sensor 112. The first data may indicate the detection of a detected object 140. In particular, the first data may be indicative of a position and / or type of detected object 140.

[0095] Upon receiving the first data, the computing device 114 determines, at step 520, a first location 232 of the detected object 140 based on the first data. The first location may be indicated as coordinates of the detected object 140 in a coordinate system, such as geographic coordinates.

[0096] The computing device may further determine, at step 530, a second location 234 based on the first location 232. The second location 234 may indicate a location where the detected object 140 may be directed to. The second location 234 may be indicated in a format similar to that of first location 232.

[0097] The computing device 114 may further determine, at step 540, a configuration of the projectile launching device to propel a projectile to collide with the detected object in the second location 234. The configuration preferably includes the target tilt and pan angles and a propelling time indicative of the time when the projectile should leave the projectile launching device.List of reference signs

[0098] 110Weapon station 112Passive sensor 114Computing device 116Projectile launching device 130Range of view 140Detected object 232First location 234Second location 242Direction of motion 310Configuration 311Lateral position at first position 312Coordinate system 313Lateral position at target position 316Pan angle 321Elevation at first position 322Coordinate system 323Elevation at target position 326Tilt angle 500Method 510-540Steps of method 500

Examples

Embodiment Construction

[0050]Fig. 1 schematically illustrates an exemplary weapon station 110.

[0051]The weapon station 110 may include at least one passive sensor 112, a computing device 114 and a projectile launching device 116. The passive sensor 112 may be communicatively coupled with the computing device 114 and in turn the computing device 114 may be communicatively coupled with the projectile launching device.

[0052]The passive sensor 112 may perform one or more observations of the environment within a range of view 130 that may depend on the sensor. Further, the passive sensor 112 may produce data codifying the observations performed and transmit the data to the computing device 114.

[0053]When an exemplary object 140 is present in said range of view 130, then passive sensor 112 may observe and detect object 140 and produce first data about the detected object 140. In particular, the first data may be indicative of the detected object 140.

[0054]Passive sensors, such as sensor 112, are sensors that ma...

Technical Field

[0001] The present disclosure relates to systems, methods, and computer program products for controlling projectiles launching devices based on a Ballistic Model of the projectile and information provided by a passive sensor.Background

[0002] A problem in Counter Unmanned-Aircraft Systems (C-UAS), is the emergence of new types of weapons, such as small and agile drones, which have a "pop-up threat" character, in other words they can fly "under the radar." These new weapons have a small size, low radar cross section, and low signal-to-noise ratio. Additionally, they can be surprisingly fast, reaching up to hundreds of km / h, and they can perform sudden evasive manoeuvres. The combination of these features renders them hard to detect and hard to hit.

[0003] A known solution to counter the threat posed by new weapons includes utilizing high-cost defence systems, such as air defence missiles, which may cost multiple hundreds of thousands of Euros. Since the cost of drones and other new weapons is at most in the tens of thousands of Euros, they put defenders in an economically asymmetric position.

[0004] Therefore, there is a need for low-cost systems and methods that enable the detection of these new weapons and that disable these weapons with low-cost counter ammunitions.Summary

[0005] The present invention is defined in the independent claims. The dependent claims recite selected optional features. In the following, each of the described methods, systems, apparatuses, examples, and aspects, which do not fully correspond to the invention as defined in the claims is thus not according to the invention and is, as well as the whole following description, present for illustration purposes only or to highlight specific aspects or features of the claims.

[0006] A first aspect of the present disclosure relates to a computer-implemented method of determining a configuration of a projectile launching device. An exemplary projectile launching device may be a weapon, for example a machine gun, which may be included in a weapon station. The method comprises receiving first data from a passive sensor and determining a first location of a detected object based on the first data. The detected object may be a drone or a type of weapon or any other threat to the weapon station. Further, the method comprises determining a second location of the detected object based on the first location. The second location may be a future location of the detected object. The determined first and second locations may be locations of the detected object relative to the projectile launching device, respectively. Alternatively, the first and second locations may be locations of the detected object relative to another coordinate origin, respectively. Further, the method comprises determining a configuration of the projectile launching device to propel a projectile to collide with the detected object in the second location. The passive sensor may be an electrooptical sensor, such as camera, in particular including a two-dimensional sensor (array).

[0007] The method is directed to the determination of a suitable configuration of a projectile launching device to hit an object detected by the passive sensor where the object may be potentially moving. Examples of the object may include a drone or other types of weapons.

[0008] The method may be preferably executed by a computing device that may be coupled with the passive sensor and the projectile launching device. The coupling above may enable the computing device to determine the configuration of the projectile launching device that may enable the configured projectile launching device to hit the detected object at the second location.

[0009] The first data may be the result of a single observation performed by the passive sensor, or the first data may be derived by a detection and / or tracking process, such as a single object tracking (SOT) process that may receive data or observations from the passive sensor and derive the location of the detected object and possible other information such as velocity and direction of the detected object. The detection and / or tracking process may be performed by computing resources included in the passive sensor or by the computing device.

[0010] In some examples, the second location may be determined based on an expected trajectory of the detected object, in particular a direction of movement and / or velocity of the detected object.

[0011] The second location may be determined by dead reckoning based on the direction and velocity of movement of the detected object at the first location in combination with a determined time interval to the second location. The determination of the second location may also include a model of behaviour of the detected object to estimate changes of the direction of movement and / or velocity of the detected object during the time interval. Behavioural models may be based on knowledge of behaviour of the detected object or based on pre-trained AI models or learned based on AI based models and experience with the detected object.

[0012] In some examples, determining the configuration of the projectile launching device comprises determining a trajectory of the projectile to the second location.

[0013] In some examples, the trajectory of the projectile may be based on at least one of: an aerodynamic drag of the projectile, a drop of the projectile, and atmospheric conditions.

[0014] The atmospheric conditions may include one or more of a humidity, an air pressure, and / or a wind.

[0015] The trajectory of the projectile may also be based on a ballistic model of the projectile and the projectile launching device. The ballistic model may leverage on the detected object trajectory and additional projectile launching device and projectile statistics to provide a configuration of the projectile launching device with an aim correction which compensates the drag and drop of the projectiles and the motion of the detected object to reliably hit the detected obj ect.

[0016] In some examples, determining the configuration of the projectile launching device comprises determining target pan and tilt angles of the projectile launching device based on the trajectory of the projectile.

[0017] The configuration of the projectile launching device determines where the projectile launching device points to, consequently it will affect the trajectory of the projectiles. Thus, the determined configuration may need to match the expected trajectory of the projectile.

[0018] The projectile launching device may include a launcher that directs the projectiles during propulsion. If the projectile launching device is an exemplary machine gun, the launcher may be the barrel of the machine gun. The target pan and tilt angles comprised in the configuration of the projectile launching device may also be considered to be the tilt and pan angles of the launcher included in the projectile launching device, where tilt angle may relate to an angle or an inclination of the launcher in the vertical direction, and pan angle to an angle or a deviation in the horizontal direction.

[0019] In some examples, determining the configuration further includes an estimate of a tilt rotation time and / or a pan rotation time of the projectile launching device from first tilt and pan angles to the target tilt and pan angles.

[0020] Configuring the projectile launching device may further include rotating the projectile launching device from a first initial position to a second target position. Planning this rotation may include making estimates of the rotation time of the projectile launching device from the first position to a second target position. The rotation time may be decomposed in a tilt rotation time that is the time required to rotate the projectile launching device from a first tilt angle to the target tilt angle; and the pan rotation time that is the time required to rotate the projectile launching device from a first pan angle to the target pan angle. The projectile launching device rotation time may depend on the physical construction of the projectile launching device; for example, the projectile launching device may include a gimbal that control the position of the launcher. Thus, the pan and tilt rotation times may depend on the velocity rates for a two-axis gimbal that may control the position of the projectile launching device.

[0021] In some examples, the configuration further includes an estimate of a total execution time, wherein: the total execution time may be based on the tilt rotation time, the pan rotation time; and / or an estimate of a flight time of the projectile from the projectile launching device to the second location and / or an estimate of the lock time that is the time that the propelling mechanism of the projectile launching device requires to propel the projectile.

[0022] The total execution time may further be based on processing latencies of any involved computing process, such as neural network computations (e.g. for object detection, tracking and / or classification) and / or sensor data capture processes, such as for example the electrooptical sensor image capture process.

[0023] The total execution time is relevant because it may indicate whether the projectile arrives in time to hit the target. The total execution time may indicate the time required to rotate the projectile launching device from a first position to a target position that may correspond to the position from which to propel the projectile, or multiple projectiles, to the detected object. The total rotation time of the projectile launching device may depend on how the projectile launching device rotates from a first position to a second position. In some cases, the pan rotation may precede the tilt rotation, or vice versa, thus the total rotation time may be computed as a function of the sum of the tilt rotation time and the pan rotation time; in other examples, the total rotation time of the projectile launching device may be computed as a function of the maximum of the tilt rotation time and the pan rotation time, in particular when the pan rotation and the tilt rotation occur simultaneously and / or concurrently.

[0024] In some examples, the computer-implemented method may further include: determining a time interval needed for the detected object to reach the second location; wherein the second location is determined such that the time interval exceeds the estimate of the total execution time.

[0025] Upon estimating the trajectory of the detected object, the computing device may determine the second location along the trajectory. The determination of the second location may depend on the total execution time to enable the projectile launching device 114 to reconfigure before the detected object 140 reaches the second location. In other words, the total execution time may be equal or shorter than the time required to the detected object 140 to reach the second location.

[0026] The determination of the second location may depend on the velocity of rotation of the projectile launching device. The faster the rotation velocity of the projectile launching device, the smaller distance between the first and the second location may be.

[0027] In some examples, the computer-implemented method may further include receiving second data from the passive sensor; determining the location of the detected object in the second data and determining whether the second data may be indicative of any damage to the detected object.

[0028] Once determined, the configuration of the projectile launching device may be transferred to the projectile launching device that may then propel the projectile, or multiple projectiles, towards the detected object. The second data may indicate of the status of the detected object after the propulsion of the projectile. Based on the second data, computing device may determine whether the detected object has been damaged or destroyed or whether the projectile launching device may need to be reconfigured to attempt to hit the detected object again.

[0029] In some examples, the computer-implemented method may further include determining a third location based on the second location and / or the second data.

[0030] In some examples, the computer-implemented method may further include determining a second configuration of the projectile launching device according to any of the previous examples using the second location as the first location; the third location as the second location. In other words: any of the previous examples may be performed iteratively.

[0031] The projectile launching device may identify a third location temporally following the second location of the detected object. The third location may be a location in which the detected object may be at the third time. The determination of the third location may be analogous to the determination of the second location with the only difference that the third location may be determined based on the second location or on the second data, or the second data. In some examples, the third location may be determined based on an expected direction and velocity of movement of the detected object at the second location or at a location derived from an indication of the position of the detected object based on the second data.

[0032] Further, the third location may be associated with a second configuration of the projectile launching device so that the projectile launching device may aim at the detected object in that third location.

[0033] Consequently, a plan may be constructed that may indicate a sequence of configurations directed to multiple points in which the detected object may be. The execution of the plan may require continuous relative adjustments of the configurations of the projectile launching device in view of the changes of location, direction, and velocity of the detected object. The computing device may perform these continuous configuration adjustments based on a Model Predictive Control (MPC) algorithm to minimize the rotations of the launcher of the projectile launching device while maintaining the projectile launching device pointing toward the detected object.

[0034] One advantage of MPC is that it may leverage predictions of the detected object motion and adapt the position of the projectile launching device to these changes. Furthermore, to adapt to the irregular and possibly low frequency of sensor information, the MPC algorithm may be combined with a continuous-time Kalman filter that processes sensor data at irregular time intervals to estimate the gun position considering rotation inertia even when no new sensor data provides updates on the position of the detected object.

[0035] In some examples, the passive sensor includes an electrooptical sensor, preferably an infrared camera or any other camera; and the first data and / or second data includes one or more images.

[0036] The passive sensor may be an electrooptical sensor and the data includes one or more images from the electrooptical sensor. Electrooptical sensors may enable efficient image processing to detect and recognize objects in the image and locate the objects in space.

[0037] In some examples, determining the first location includes determining a type of the detected object; calculating a distance between the passive sensor and the detected object based on the type of the detected object. In some examples, the type, in particular the physical size, of the object is determined by a (machine learning) object classification tool, e.g., including an object classification AI model,

[0038] In some examples, determining the first location may further include determining a physical size of the detected object based on the type of the detected object and / or the size of the detected object in the image. In some examples, the trajectory and / or a distance between the passive sensor and the detected object may be determined based on the physical size of the detected object.

[0039] In some examples, determining the first location may be further based on the passive sensor parameters.

[0040] The determination of the type of the detected object may enable a determination of the action to perform in reaction to the detection of the object. The determination of the type of the object may also enable the determination of the distance between the sensor and the detected object.

[0041] The determination of the type of the detected object may also enable the determination of the of the physical size of the detected object based on some list of the possible known objects, where the list may be a database of objects, or the list may be implicit in a classification system that given the image of the detected object reports the detected object type.

[0042] The determination of the size of the detected object may enable the determination of the location of the detected object in a 3D space based on the physical size of the detected object, the size of the detected object in the image and the intrinsic and extrinsic properties of the electrooptical sensor. Intrinsic properties may include parameters such as focal length, the field of view, aperture, resolution. Extrinsic properties of the electrooptical sensor may include camera location and orientation.

[0043] It is preferred to determine a bounding box enclosing the detected object based on the first data. This allows the determination of a size the detected object in the image. It is further preferred to determine the first location information based on the size the detected object in the image and the physical size of the detected object. The first location information is preferably indicative of a distance between the passive sensor and the detected object.

[0044] Determining the first location preferably includes processing the first data by a single object tracking algorithm. The single object tracking algorithm may determine the bounding box.

[0045] In some examples, the detected object may be a drone.

[0046] The detected objects may be drones, but other detected objects may also be possible, such as other weapons, stationary or non-stationary sensors or other types of manned or unmanned devices.

[0047] A second aspect of the present disclosure relates to a system including one or more processors and one or more storage devices, wherein the system may be configured to perform the computer-implemented method of any preceding examples.

[0048] A third aspect of the present disclosure relates to a computer program product for loading into a memory of a computer, including instructions, that, when executed by a processor of the computer, cause the computer to execute a computer-implemented method of any preceding examples.Brief description of the drawings

[0049] The features, objects, and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings in which like reference numerals refer to similar elements. Fig. 1 is a schematic drawing illustrating an exemplary weapon station; Fig. 2 is a schematic drawing illustrating the detection of an object in a first location and the prediction of a second location of the object; Fig. 3 is a schematic top view of an exemplary configuration of the projectile launching device; Fig. 4 is a schematic side view of the exemplary configuration of the projectile launching device; and Fig. 5 is a flow chart of an example of a method of determining a configuration of a projectile launching device. Detailed description of the preferred embodiments

[0050] Fig. 1 schematically illustrates an exemplary weapon station 110.

[0051] The weapon station 110 may include at least one passive sensor 112, a computing device 114 and a projectile launching device 116. The passive sensor 112 may be communicatively coupled with the computing device 114 and in turn the computing device 114 may be communicatively coupled with the projectile launching device.

[0052] The passive sensor 112 may perform one or more observations of the environment within a range of view 130 that may depend on the sensor. Further, the passive sensor 112 may produce data codifying the observations performed and transmit the data to the computing device 114.

[0053] When an exemplary object 140 is present in said range of view 130, then passive sensor 112 may observe and detect object 140 and produce first data about the detected object 140. In particular, the first data may be indicative of the detected object 140.

[0054] Passive sensors, such as sensor 112, are sensors that may detect signals or energy emitted or reflected directly or indirectly by the objects in the range of view of the sensor. Passive sensors differ from active sensors, such as radars that actively transmit energy to detect objects. Since passive sensors do not transmit any signals or any energy, but rather passively detect signals produced by other objects, enemy objects may not detect any signal revealing the existence of the passive sensor, thus passive sensors tend to be invisible to enemy objects.

[0055] Examples of passive sensors may include electro-optical sensors, such as a camera for visible or infrared light. Other examples of passive sensors may include inter alia microphones for noise sensing, thermal sensors, sensors detecting changes in the electric field, sensors detecting chemical compounds, or seismic sensors. Additionally, passive sensor110 may be a sensor network of passive sensors.

[0056] The data produced by the passive sensor 112 may depend on the type of sensor. In the specific case of electro-optical sensors, the data produced by passive sensor 112, such as the first data, may include one or more images. Passive sensor data, such as the first data, may also include meta-data about the data. Examples of meta-data may include extrinsic parameters such as temporal or location information indicating when and where the data has been collected. Other types of meta-data may include intrinsic parameters of the sensor. In the exemplary case of electro-optical sensors, intrinsic sensors parameters may include inter alia focal length, aperture, field-of-view, resolution.

[0057] The computing device 114 may be configured to receive data about object 140 from the passive sensor 112 and process the received data. The data transmission from the passive sensor 112 to the computing device 114 may be based on different protocols. In some cases, the passive sensor 112 may stream data to the computing device 114, enabling the passive sensor 112 to perform an observation, and immediately transmit the data related to the observation to the computing device; thus, tightly coupling data sensing and processing. However, the data transfer may also be based on other protocols that may enable data transfer in batches to be processed serially, or other transfer processes or policies may be utilized depending on the specific application requirements.

[0058] Upon receiving data from the passive sensor, the computing device 114 may process the data. Such processing may include determining whether the detected object 140 may pose a threat, estimating the location and movement of the detected object 140 and estimating how to configure the projectile launching device 116 to hit and disable the detected object 140.

[0059] The computing device 114 may include one or more physical and / or virtual computing platforms that may preferably be in the proximity of the passive sensor 112; but it may also be in a cloud geographically distant from the passive sensor 112. The computing device may also be a hybrid combination of local computing platforms, such as a computer positioned in the proximity of the passive sensor 112, and virtual computers distributed in geographically distinct clouds.

[0060] The computing device 114 may also include any type of physical computing platforms including single board computers, mobile devices, and / or tablets. The computing device may be a computing platform embedded in the passive sensor 112 or in the projectile launching device, or embedded in other components that may be included in the weapon station. The computing device 114 may also be embedded in one or more devices of the weapon station 110. For example, functionalities related to the sensor data processing may be embedded in the passive sensor 112, while the functionalities related to the configuration of the projectile launching device may be embedded in the projectile launching device itself.

[0061] The projectile launching device may include a launcher emitting, for example propelling, projectiles. More generally, a projectile launching device may be a weapon and the launcher may be the barrel of the weapon. Examples of projectile launching devices may include any device configured or configurable to launch visible, in particular traceable, ammunition, such as a machine gun, a cannon, or a grenade launcher for unpropelled grenades. Since the projectiles are propelled by the launcher in these cases, the disclosed projectile launching device may reduce the cost and complexity of the weapon station, and in particular the cost and complexity of the projectiles may be reduced because the projectiles may not require complex hardware or software to pursue their target as well as hardware and software to be self-propelled. The projectile launching device may also comprise a launcher for an unguided propelled projectile, such as a rocket-propelled grenade, or an unguided rocket.

[0062] The coupling between the passive sensor 112, the computing device 114, and the projectile launching device 116 may be based on any network configuration that may enable data transmission. Thus, the coupling may be based on wired or wireless networking; it may further be based any Local Area Network infrastructure, Wide Area Network infrastructure, or Satellite communication infrastructure.

[0063] The passive sensor 112, the computing device 114 and the projectile launching device 116 may be tightly connected and be partially or entirely embedded systems. Thus, for example, the passive sensor may be placed on the projectile launching device or embedded in the projectile launching device. Similarly, the computing device may be an embedded system. Thus, the coupling between the components of the weapon station 110 may be at least partially based on a network on chip communication system as well as any on-chip or peripheral communication infrastructure.

[0064] Fig. 2 illustrates the detection of object 140 in a first location and the prediction of a second location of the object 140. Fig. 2 analogously to Fig. 1 includes the weapon station 110 which in turn includes the passive sensor 112, the computing device 114 and the projectile launching device 116 as discussed above. The passive sensor 112 may detect the object 140 and may transmit first data about the detected object 140 to the computing device 114 that is coupled with the passive sensor 112.

[0065] The transmission of the first data from the passive sensor 112 to the computing device 114 may be part of a single object tracking (SOT) process to track objects detected by the passive sensor 112, such as detected object 140. The SOT may be performed by the passive sensor 112 or by the computing device 114 that may provide to the passive sensor 112 indications to track the detected object 140 and indications on how to modify the range of view to track the detected object 140.

[0066] Upon receiving the first data from the passive sensor 112, the computing device 116 may attempt to recognize the detected object 140 and determine a type of detected object 140. This determination may be part of SOT, and it may be based on a pretrained AI recognition / classification model. Further, based on the recognition of the detected object 140 and of its type, the computing device 114 may determine whether the detected object 140 may poses a threat.

[0067] The determination of the type of the detected object may also enable the determination of the properties of the detected object, such as its physical size, and other properties. This determination may be based on a list of known objects, such as a list of known drones and their properties. This list may be a lookup table or a database including information about drones, or the list may be implicit in the AI recognition / classification model.

[0068] The determination of the physical size of object 140 may enable the determination of the location 232 of the detected object 140 and the determination of the direction and velocity of movement of the detected object 140. The determination of the location and movement of the detected object may be based on the determined physical size of object 140, the features of the object 140 in the data and on the properties of the passive sensor.

[0069] If for example, the passive sensor is an electrooptical sensor, preferably an infrared camera; the data includes one or more images. The determination of the type of the detected object may be based on the application of image recognition, which may be based on a pretrained AI recognition / classification model. The AI model may also provide information such as the physical size of the object. The calculation of the size of the object in the image may be performed by computing the bounding box of the object in the image. The location of the detected object may be estimated based on the extrinsic parameters of the electrooptical sensor such as location and orientation and intrinsic parameters of the electrooptical sensor such as focal length, the field of view, aperture, resolution.

[0070] The computing device 114 may determine that object 140 has been detected at location 232. Location 232 may be considered a first location of the object 140. This location may correspond to the location of the object at a first time, which may be the instant in which the object has been detected by the passive sensor. The computing device may also determine a second location 234, that may correspond to a location that the detected object 140 may occupy at a second time, which may temporally follow the first time, in other words the second time is in the future with respect to the first time.

[0071] The second location 234 of the detected object 140 may be determined based on the current location 232 of the detected object 140 and knowledge about the detected object 140 such as current velocity and direction of motion 242 of the detected object 140 and an expected motion of the detected object. In case the expected motion of the detected object may be assumed to be constant in direction and velocity, thus the second location may be estimated based on the physical laws of motion. However, other estimation methods may be used such as method referring to behavioural models that may attempt to predict unexpected changes of direction and velocity of the detected object. The second location may also be extracted from a Kalman filter that may also be used to improve the reliability of the passive sensor data.

[0072] The estimate of the second location may also depend on a determination of the second time that may specify when to aim at the detected object. The second time may need to be sufficiently far in the future to take into account delays due to the reconfiguration of the projectile launching device, thus, enabling the projectile launching device to reconfigure and to aim to the detected object.

[0073] Upon determining the second location, the computing device may determine one or more configurations of the projectile launching device so that the projectile launching device may point to the detected object and propel projectiles to collide with and disable the detected obj ect.

[0074] Fig. 3 is a schematic top view of an exemplary configuration 310 of the projectile launching device 116. The direction of view is indicated by coordinate system 312. Fig. 3 relates to the pointing of the launcher included in the projectile launching device 116 to the detected object with the objective to hit the object and disable it.

[0075] Fig. 3 relates to the horizontal rotation of the launcher of the projectile launching device 116. In Fig. 3, line 311 may represent a first position of the launcher, while line 313 may represent a target position of the launcher. Angle 316 may be considered to be the pan angle of the launcher indicating the deviation between two positions of the launcher in a horizontal direction.

[0076] Fig. 4 is a schematic side view of the exemplary configuration 310 of the projectile launching device 116 that illustrates vertical rotation of the launcher of the projectile launching device 116. The direction of view is indicated by coordinate system 322. Line 321 may represent a possible first position of the launcher of projectile launching device 116. Furthermore, line 323 may represent the target position of the launcher of projectile launching device 116. Angle 326 may represent the tilt angle of the launcher of the projectile launching device 116, where the tilt angle may indicate the deviation between the first and the second position of the launcher of device 116 in a vertical direction.

[0077] The determination of the configuration of the projectile launching device 116 may include the determination of the pan and tilt angles that match the expected trajectory of a trajectory of the propelled projectiles from the projectile launching device towards one of the second locations where the detected object 140 is expected to be.

[0078] The determination of the trajectory of the projectile may be based on the second location 234 and on at least one of: an aerodynamic drag of the projectile, a drop of the projectile, and atmospheric conditions, where the atmospheric conditions may include one or more of a humidity, an air pressure, and / or a wind.

[0079] The determination of the trajectory of the projectiles may be based on a ballistic model of the projectiles that may leverage the trajectory of the detected object and on additional gun and bullet statistics to provide pan and tilt angles with aim correction which compensates drag, bullet drop and target motion to reliably hit the target.

[0080] Configuring the projectile launching device may further include estimating an execution time. The execution time may be a function of the sum of: the time required to execute the rotation from the current position to the target position of the projectile launching device, an estimate of the lock time that is the time that the propelling mechanism of the projectile launching device requires to propel the projectile. The lock time may also be defined as the time required to propel the projectile, e.g., the time of rotation of a firing pin or a bolt of a gun, and the projectile's time of flight from the projectile launching device 116 to the second location 234 of the detected object 140.

[0081] Configuring the projectile launching device may include an abort condition in case the execution time may be shorter than the available time until the detected object 140 may be expected to be at the second position 234. The available time may be determined as the difference between the second time and the first time as described with reference to Figure 2.

[0082] An estimate of the rotation time of the projectile launching device may be further decomposed in estimating a tilt rotation time that may be the time required to perform the determined tilt rotation; and the pan rotation time that may be the time required to perform a pan rotation. The total rotation time of the projectile launching device may depend on the process of rotation of the device. For example, if the device rotates in a first direction, which may be the horizontally, or pan direction, and then in a second direction, which may be a vertically, or tilt direction, then the total rotation time may be a function of the sum of the pan and tilt rotation times. If instead the device rotates in both directions concurrently, then the time of rotation may be a function of the maximum of the pan or tilt rotation times. Additionally, other rotation policies may be used that may lead to other rotation time estimation functions to be used.

[0083] The total execution time may further include time related to processing latencies of any involved computing process, such as neural network computations (e.g. for object detection, tracking and / or classification) and / or sensor data capture processes, such as for example the electrooptical sensor image capture process.

[0084] The total execution time may be equal or shorter than the time required to the detected object 140 to reach the second location to enable to projectile launching device 114 to propel the projectiles to hit the detected object 140.

[0085] Once completed, the configuration parameters may be transmitted to the gun that could proceed to execute the received configuration and propel the projectiles towards the detected object.

[0086] The passive sensor may be further configured to perform at least one second observation of the detected object and thereby generate second data. The second data may have a format similar to the first data. The computing device may then receive the second data from the passive sensor. Based on the second data, the computing device may determine a new location of the detected object according to the second data and determine whether the second data is indicative of any damage to the detected object. Further, the second data may provide the basis for a decision as to whether to further damage the detected object or whether the detected object was already damaged or destroyed.

[0087] To further damage the detected object, the computer-implemented method may further include determining at least one third location based on the second location and / or the new location of the detected object. The third location may be the next location where to attempt to hit and damage the detected object. The third location may be further associated to a third time, where the third time temporally follows the second time.

[0088] The third location may then be considered to be analogous to the second location in the sense that the computing device may project a trajectory of the propelled projectiles to the third location and specify a new configuration of the projectiles launching device to hit the detected object at the third location according to the methods disclosed in relation to the second location. Thus, the third location may implicitly trigger an iterative process in which the passive sensor detects the object, the computing device projects the location of the detected object in the future and determines a configuration of the projectile launching device to disable the detected in the third location.

[0089] Since, the computing device may determine multiple second and / or third locations that may be distributed to be progressively further in the future. The computing device may be further configured to optimize the rotations of the projectile propelling device to reduce the delays between locations in which to damage the detected object thus improving the likelihood of damaging the detected object.

[0090] The optimization may be achieved by estimating a plurality of possible locations of the detected object 140 at multiple times, that may be considered to be multiple second and third times, and plan the rotations of the projectile launching device to propel projectiles to the detected object in as many of the estimated plurality of possible locations of the detected object 140 as possible. Furthermore, the optimization may require adapting the plan to react to unexpected changes of location of the detected object.

[0091] The computation device may construct and adapt the plan based on the MPC algorithm to minimize the rotations of the projectile launching device while maintaining the projectile launching device pointing toward the detected object. The updates of the positions of the projectile launching device 140 may depend on sensor data indicating the location of the detected object and thus the adjustments to be made to the projectile launching device.

[0092] Since sensor data may be received irregularly depending on the ability of the sensor to detect and track the detected object, the MPC algorithm may be combined with a continuous-time Kalman filter that processes sensor data at irregular time intervals to estimate the location of the detected object based on the sensor data and the expected location of the detected object. Further, the estimated locations of the detected object are transformed into configurations of the projectile launching device while considering rotation inertia of the projectile launching device.

[0093] Fig. 5 illustrates an example of a method 500 of determining a configuration of a projectile launching device. Method 500 may be executed by the computing device 114 of the weapon station.

[0094] In step 510, the computing device receives first data from a passive sensor that may be sensor 112. The first data may indicate the detection of a detected object 140. In particular, the first data may be indicative of a position and / or type of detected object 140.

[0095] Upon receiving the first data, the computing device 114 determines, at step 520, a first location 232 of the detected object 140 based on the first data. The first location may be indicated as coordinates of the detected object 140 in a coordinate system, such as geographic coordinates.

[0096] The computing device may further determine, at step 530, a second location 234 based on the first location 232. The second location 234 may indicate a location where the detected object 140 may be directed to. The second location 234 may be indicated in a format similar to that of first location 232.

[0097] The computing device 114 may further determine, at step 540, a configuration of the projectile launching device to propel a projectile to collide with the detected object in the second location 234. The configuration preferably includes the target tilt and pan angles and a propelling time indicative of the time when the projectile should leave the projectile launching device.List of reference signs

[0098] 110Weapon station 112Passive sensor 114Computing device 116Projectile launching device 130Range of view 140Detected object 232First location 234Second location 242Direction of motion 310Configuration 311Lateral position at first position 312Coordinate system 313Lateral position at target position 316Pan angle 321Elevation at first position 322Coordinate system 323Elevation at target position 326Tilt angle 500Method 510-540Steps of method 500

Examples

Embodiment Construction

[0050]Fig. 1 schematically illustrates an exemplary weapon station 110.

[0051]The weapon station 110 may include at least one passive sensor 112, a computing device 114 and a projectile launching device 116. The passive sensor 112 may be communicatively coupled with the computing device 114 and in turn the computing device 114 may be communicatively coupled with the projectile launching device.

[0052]The passive sensor 112 may perform one or more observations of the environment within a range of view 130 that may depend on the sensor. Further, the passive sensor 112 may produce data codifying the observations performed and transmit the data to the computing device 114.

[0053]When an exemplary object 140 is present in said range of view 130, then passive sensor 112 may observe and detect object 140 and produce first data about the detected object 140. In particular, the first data may be indicative of the detected object 140.

[0054]Passive sensors, such as sensor 112, are sensors that ma...

Claims

1. A computer-implemented method of determining a configuration of a projectile launching device, the method comprising: receiving first data from a passive sensor; determining a first location of a detected object based on the first data; determining a second location based on the first location; and determining a configuration of the projectile launching device to propel a projectile to collide with the detected object in the second location.

2. The computer-implemented method of the previous claim, wherein the second location is a location determined based on an expected trajectory of the detected object, in particular a direction of movement and / or velocity of the detected object.

3. The computer-implemented method of any preceding claim, wherein determining the configuration of the projectile launching device further comprises determining a trajectory of the projectile to the second location.

4. The computer-implemented method of the preceding claim wherein the trajectory of the projectile is based on at least one of: an aerodynamic drag of the projectile, a drop of the projectile, and atmospheric conditions.

5. The computer-implemented method of any preceding claim, wherein determining the configuration of the projectile launching device further comprises determining target pan and tilt angles of the projectile launching device based on the trajectory of the projectile.

6. The computer-implemented method of the preceding claim, wherein determining the configuration further comprises an estimate of a tilt rotation time and a pan rotation time of the projectile launching device from first tilt and pan angles to the target tilt and pan angles.

7. The computer-implemented method of the preceding claim, wherein the configuration further comprises an estimate of a total execution time, wherein: the estimate of the total execution time is based on the tilt and pan rotation time and an estimate of a flight time of the projectile.

8. The computer-implemented method of the preceding claim, further comprising: determining a time interval needed for the detected object to reach the second location; wherein the second location is determined such that the time interval exceeds the estimate of the total execution time.

9. The computer-implemented method of any preceding claim, further comprising: receiving second data from the passive sensor; determining the location of the detected object in the second data; and determining whether the second data is indicative of any damage to the detected obj ect.

10. The computer-implemented method of the preceding claim, further comprising: determining a third location based on the second location and / or the second data; determining a second configuration of the projectile launching device by performing the method of any of the claims from 2 to 9 using: the second location as the first location; and the third location as the second location.

11. The computer-implemented method of any preceding claim, wherein: the passive sensor comprises an electrooptical sensor, preferably an infrared camera; and the first data and / or second data includes one or more images.

12. The computer-implemented method of any preceding claim, wherein determining the first location comprises: determining a type of the detected object; in particular a physical size of the detected object; and determining the trajectory and / or a distance between the passive sensor and the detected object based on the type, in particular the physical size, of the detected obj ect.

13. The computer-implemented method of any preceding claim, wherein the detected object is a drone.

14. A system comprising one or more processors and one or more storage devices, wherein the system is configured to perform the computer-implemented method of any preceding claim.

15. A computer program product for loading into a memory of a computer, comprising instructions, that, when executed by a processor of the computer, cause the computer to execute a computer-implemented method of any of claims 1-13.

Images