AIRCRAFT, SYSTEM AND PROCEDURES
A decentralized computer architecture with off-the-shelf components addresses the Von Neumann bottleneck in aircraft cluster cameras, enhancing scalability and flexibility for real-time image processing and transmission.
Patent Information
- Application Number
- DE102023117932
- Authority / Receiving Office
- DE · DE
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-07-07
- Publication Date
- 2025-01-09
AI Technical Summary
Existing aircraft with cluster cameras face a Von Neumann bottleneck, limiting their ability to handle resource-intensive tasks simultaneously due to constrained data exchange between components, which hinders scalability and requires trade-offs between computing power, space, weight, and data transmission.
Implementing a decentralized or distributed computer architecture with each camera module having its own processing unit, connected via a network, and a transmission unit for real-time data processing and transmission, using off-the-shelf products to form a scalable and flexible camera group.
This architecture eliminates the Von Neumann bottleneck, enabling efficient real-time processing and transmission of image data, allowing for cost-effective, scalable, and flexible aircraft systems suitable for continuous area-wide monitoring.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The present invention relates to an aircraft, a system comprising such an aircraft and a method for operating such an aircraft.
[0002] For comprehensive, continuous monitoring of the Earth's surface, particularly for so-called WAPS (Wide-Area Persistent Surveillance) applications, manned or unmanned aerial vehicles (UAVs) equipped with cluster cameras can be deployed. A cluster camera comprises multiple cameras for capturing image data. This image data is transmitted to a ground-based base station, which is then used to analyze and process the image data.
[0003] According to internal findings, the processing units of such cluster cameras exhibit a classic von Neumann computer architecture. This involves linking individual or multiple computer components via a central bus system, enabling data exchange. The well-known problem with this architecture is that, due to the so-called "von Neumann bottleneck," a computer with this architecture is only capable of handling multiple resource-intensive tasks simultaneously to a limited extent. In the case of a cluster camera with numerous cameras, however, very large amounts of data from various sources must be transferred and processed using simple arithmetic operations.
[0004] This quickly turns the bus system into a bottleneck, especially when several essential functions, such as processing, storage, analysis, or the like, are used simultaneously. This problem severely limits the scalability of cluster cameras, as while the computing power of individual components can be very high, the data exchange between these components is limited.
[0005] Furthermore, such cluster cameras for aircraft present a trade-off or conflict of objectives that generally underlies airborne WAPS applications. Very high resolutions and computing power are required to perform WAPS. At the same time, space, weight, and data transmission in airplanes, helicopters, and drones vary considerably and are severely limited. This means, for example, that increasing computing power can lead to a reduction in the range and / or operational time of such an aircraft.
[0006] Against this background, one object of the present invention is to provide an improved aircraft.
[0007] Accordingly, an aircraft equipped with a camera array for continuous, area-wide surveillance of the Earth's surface is proposed. The camera array comprises several cameras, each with its line of sight inclined relative to the others, N camera modules with N ≥ 2, each camera module comprising M cameras with M ≥ 1, and each of the N camera modules comprising a processing unit for processing image data provided by the M cameras, a network connecting the processing units, and a transmission unit connected to the network for data communication between the camera array and a base station of the aircraft.The transmission unit is designed to transmit the processed image data from the processing units to the base station, where the network data transmission rate is too low to transmit the image data provided by the M cameras and / or the processed image data of the processing units in real time.
[0008] Because each camera module has its own processing unit, a decentralized or distributed computing architecture can be implemented. In particular, by exploiting the physical limitations of the cameras, the image processing task becomes mathematically separable during live operation of the camera group. The previously described problem of the von Neumann bottleneck is eliminated. The camera group can be assembled from so-called OTS (Off the Shelf) products, especially COTS (Commercial Off the Shelf) products. This enables cost-effective manufacturing of the camera group. Furthermore, it allows for unlimited scalability, high flexibility, and rapid further development, for example, with regard to extended functions. All cameras preferably form a rigid mechanical unit within the camera group.Due to the rigid mechanics, it is advantageous that only a one-time calibration of the camera group is required.
[0009] The aircraft can be manned or unmanned. Preferably, the aircraft is a drone, in particular a reconnaissance drone. The terms "aircraft" and "drone" can therefore be used interchangeably throughout this document. The aircraft can be used for both civilian and military purposes. The aircraft can be an airplane, in particular a fixed-wing aircraft. However, the aircraft can also be a rotorcraft or helicopter. In particular, the aircraft is suitable for WAPS or AWAPS (Airborne Wide-Area Persistent Surveillance) applications.
[0010] The Earth's surface can be a body of water. Accordingly, the terms "Earth's surface" and "body of water" can be used interchangeably in this context. In particular, the Earth's surface can be the surface of a sea or an inland body of water. The Earth's surface can also be another, distant object that appears comparatively flat and / or planar. Specifically, the aircraft serves to support reconnaissance in maritime disaster situations by means of automated data processing in the air.
[0011] The aircraft can also be used for reconnaissance in matters of civil and military security, including disaster relief, airborne Earth and environmental observation, monitoring of major events, escort of convoys or troops, border patrol, cartography, guidance of ground forces, monitoring of strategically important places and objects, detection of large-scale events and their interrelationships in terms of location and time, and / or tracking and subsequent analysis of movements in an entire surveillance area, or the like.
[0012] The camera group comprises at least two camera modules, each camera module containing at least one camera. The cameras can be digital cameras. In addition to a multitude of cameras, cameras with different sensors, particularly for detecting different wavelengths, can also be used. The camera group can comprise any number of camera modules, as long as N ≥ 2. For example, the camera group can comprise two, three, four, five, six, or more than six camera modules. Accordingly, each camera module can comprise any number of cameras. For example, each camera module can comprise one, two, three, four, five, six, or more than six cameras. The cameras are preferably arranged in a grid or matrix configuration. "Grid" or "matrix" in this context means that the cameras are arranged side by side and / or one above the other in rows and columns.
[0013] The cameras are arranged, in particular, such that they are inclined towards each other or tilted inwards. However, the cameras can also be inclined outwards or away from each other. Preferably, the cameras are arranged such that the distance between lenses, especially the intersection points of lens planes and associated optical axes, of adjacent cameras is minimal. The line of sight of each camera can coincide with a central axis of a lens of the corresponding camera. The cameras are preferably arranged such that all lines of sight are inclined relative to each other. "Inclined" or "tilted" in this context means, in particular, that the lines of sight are neither parallel nor perpendicular to each other. Each line of sight can be inclined relative to an x-direction, a y-direction, and a z-direction.Preferably, the cameras have a fixed arrangement relative to each other.
[0014] The lines of sight, or at least some of them, can intersect. Specifically, the lines of sight intersect in front of the camera array. However, they can also intersect behind the camera array. Because the camera array consists of multiple cameras, it can also be called a cluster camera. Since the lines of sight are arranged at an angle to each other, the camera array can also be called an oblique cluster camera.
[0015] Each camera module has a processing unit. These processing units are video processing and media servers and can therefore be referred to as such. The processing units, the cameras, and / or the transmission unit are preferably OTS products, particularly COTS products. Together with the transmission unit, the processing units form a decentralized or distributed computer architecture. For this purpose, the processing units and the transmission unit are interconnected via a network.
[0016] The processing units and the transmission unit are primarily embedded PCs (personal computers). Cameras can also be assigned to the transmission unit. The processing units can, for example, process the image data from the cameras by performing a preliminary analysis. This allows them to identify regions of interest (ROIs) on the Earth's surface and forward image data depicting only these regions to the transmission unit. The transmission unit then transmits only this image data depicting the regions of interest to the base station. The base station can then perform an analysis of the regions of interest.
[0017] The transmission unit can be, or be referred to as, an Application and Data Relay Server. Specifically, the transmission unit establishes the data connection between the camera array and the base station. The transmission unit can be a dedicated master module, which may also include cameras as mentioned previously. The transmission unit primarily facilitates data exchange with the base station. Specifically, the transmission unit is connected to the base station via the aforementioned data link. This data link can also be referred to as a data link.
[0018] With the aid of the data connection, the transmission unit is thus preferably configured to transmit the processed image data from the processing units to the base station. The transmission unit is also preferably suitable for processing the image data from the cameras. The transmission unit is particularly preferably configured to transmit the processed image data from the processing units to the base station via the data connection.
[0019] In this context, a "data connection" or "data link" refers to a connection, particularly a radio connection, between the transmission unit and the base station for data transmission. The data connection also has a data transmission rate. The data transmission rate of the network is preferably greater than or equal to, or preferably similar to, the data transmission rate of the data connection. A live stream of a video mosaic from the cameras can be transmitted to the base station via the data connection. It is possible for multiple base stations to be connected to the data connection.
[0020] The base station can be located on the ground. The base station can be stationary. Alternatively, the base station can also be mobile. In the latter case, the base station can be transported by a vehicle. The vehicle can be a land vehicle, a watercraft, or an aircraft, in particular a manned aircraft. If the vehicle is an aircraft, the base station is not located on the ground, but in the air.
[0021] Since the network's data transmission rate is too low to transmit the image data provided by the cameras and / or the processed image data by the processing units in real time, the processing and / or pre-evaluation of the image data preferably takes place in the processing units. Therefore, it is not necessary to transmit all image data from all cameras over the network.
[0022] According to one embodiment, each processing unit has sufficient computing power to process the image data provided by the M cameras in real time.
[0023] Thus, it is possible for each camera module to independently process and / or evaluate the image data provided by the cameras assigned to the respective camera module.
[0024] According to another embodiment, the data transmission rate of the network is greater than or equal to, preferably similar to, the data transmission rate between the transmission unit and the base station.
[0025] The data transmission rate of the network is greater than or equal to, preferably similar to, the data transmission rate of the data connection between the transmission unit and the base station.
[0026] According to another embodiment, the processing unit of each of the N camera modules is configured to identify an area or areas of interest of the Earth's surface based on the image data and to pass on only a part of the image data depicting the area or areas of interest to the transmission unit.
[0027] This reduces the amount of data that needs to be transmitted to the base station via the data connection. Objects can be located on the Earth's surface. These objects can be, for example, boats or people in the water. Each object can be assigned or have an area of interest assigned to it. The respective area of interest preferably encompasses the object to which it is assigned. The areas of interest can each have any geometry and / or size. The geometry and / or size of the areas of interest can depend on the geometry and / or size of the objects. The areas of interest can be congruent with the objects.The processing unit of each camera module is preferably configured to identify the relevant area of interest on the Earth's surface based on the image data captured by the cameras of the respective camera module and to transmit only the portion of the image data depicting that area of interest to the transmission unit. In addition to automatically identified areas of interest, a section of the image can also be requested from the base station. Other objects or areas of interest can include military or civilian forces and vehicles, such as motor vehicles, trucks, aircraft, especially UAVs, individuals, buildings, explosions, changes in the environment, tire tracks, clearings, excavations, debris, blockages, infrastructure, and / or natural obstacles such as rivers, cliffs, gorges, or the like.
[0028] According to another embodiment, the transmission unit is configured to request the part of the image data that depicts the area or areas of interest from the processing units of the N camera modules.
[0029] This portion of the image data can be transmitted to the base station via the data connection. The base station can then analyze the area(s) of interest. For example, the base station can identify and / or classify the object(s).
[0030] According to another embodiment, the processing unit of each of the N camera modules is configured to assemble the image data of the M cameras into a video mosaic.
[0031] The video mosaic can be a wide-angle image of the Earth's surface. The video mosaic can have any geometry. If the video mosaic is square, it can have a side length of several kilometers. A live stream of the video mosaic can be transmitted to the base station via the data connection. The cameras can capture different sections of the video mosaic. The sections overlap in overlapping areas. Together, the sections form the video mosaic. Each camera can be assigned one section. The sections are assembled by stitching. In photography, "stitching" refers to creating a large photograph from several smaller individual shots, which usually show overlapping sections of the subject.The size of the overlap areas depends primarily on the calibration, its configuration, and the image quality. For a successful calibration, it is essential that meaningful information can be extracted from each overlap area to establish a correlation between the image data. As a rule of thumb, 5 to 10% of the image width, or at least 200 pixels, has proven to be a reasonable value for the width of the overlap areas.
[0032] According to another embodiment, the transmission unit is designed to combine the image data from the M cameras and / or image data from the N camera modules into a video mosaic.
[0033] Accordingly, not only the processing units but also the transmission unit are designed to assemble the aforementioned sections of the video mosaic into the video mosaic.
[0034] According to another embodiment, the processing unit of each of the N camera modules has a camera interface, a video and data processor, and a memory.
[0035] The camera interface can be referred to as the Camera Interface, the video and data processor as the Video and Data Processor, and the storage as the Storage. Each camera is preferably coupled to the respective camera interface of the processing unit assigned to that camera via a bus. Each camera preferably has its own bus.
[0036] According to another embodiment, the network has several switches, with each of the N camera modules being assigned a switch, and the transmission unit being connected to one of the switches.
[0037] The switches can also be called network switches. The switches are connected to the processing units via connections. The switches are also interconnected via connections. These connections are part of the network. The connections can be LAN cables (Local Area Network). The connections can form a ring topology. Depending on the size of the network, a combination of ring and star topology may be used to reduce latency. Thanks to the constant data transfer rate limited by the data connection, the network can be of any size, provided the aircraft is capable of carrying the required number of cameras and camera modules.
[0038] According to another embodiment, each of the N camera modules has one of the switches.
[0039] This means, in particular, that N switches are provided. The transmission unit is connected to one of the switches via a connection, for example, a LAN cable. Subgroups K of the N camera modules with N ≥ K ≥ 1 can each share one switch.
[0040] According to another embodiment, the aircraft comprises a fuselage, a wing attached to the fuselage, and a propulsion system attached to the fuselage or the wing, wherein the camera assembly is attached to the fuselage and / or the wing.
[0041] A camera array can be provided on both the fuselage and the wing. The propulsion system can drive a propeller, which can be a pusher propeller. The propulsion system can, for example, comprise a two-cylinder, two-stroke engine or a Wankel engine. Alternatively, it can be a turboprop engine. The aircraft can, for example, have a wingspan of approximately 5 m, a length of approximately 3 m, and a height of approximately 1 m. Preferably, the aircraft has a takeoff mass of less than 90 kg. The aircraft can travel at a maximum speed of 90 km / h at an altitude of over 5,000 m in one direction. The maximum flight duration of the aircraft can be 12 hours. However, the values mentioned above are only examples.
[0042] Furthermore, a system with X such aircraft with X ≥ 1 and the base station is proposed, wherein the transmission unit transmits the processed image data of the processing units to the base station.
[0043] Specifically, the system is a Wide-Area Persistent Surveillance System (WAPSS) or an Airborne Wide-Area Persistent Surveillance System (AWAPSS) and can therefore also be described as such. In particular, the system serves to support reconnaissance in maritime disaster situations through automated data processing in the air. The system can be used for both civilian and military purposes. The system includes at least one aircraft. The system can include any number of aircraft. For example, the system can comprise a swarm of aircraft.
[0044] Furthermore, a method for operating such an aircraft is proposed. The aircraft comprises a camera array for continuous, area-wide surveillance of the Earth's surface. The camera array includes several cameras, all of whose lines of sight are inclined relative to each other, and N camera modules with N ≥ 2, each camera module having M cameras with M ≥ 1, and each of the N camera modules having a processing unit. The camera array also includes a network connecting the processing units and a transmission unit connected to the network for data communication between the camera array and a base station of the aircraft.The procedure comprises the following steps: a) processing image data provided by the M cameras using the processing units of the N camera modules and b) transmitting the processed image data from the processing units to the base station using the transmission unit, where the network data transmission rate is too low to transmit the image data provided by the M cameras and / or the processed image data from the processing units in real time.
[0045] Steps a) and b) are preferably performed simultaneously. During step b), the processed image data is preferably transmitted to the base station using the aforementioned data connection. The image data from the M cameras can be assembled into a video mosaic by the processing unit of each of the N camera modules. It is also possible for the transmission unit to assemble the image data from the M cameras and / or image data from the N camera modules into a video mosaic.
[0046] According to one embodiment, the processing unit of each of the N camera modules identifies one or more areas of interest on the Earth's surface based on the image data.
[0047] As mentioned previously, the area of interest can include an object, such as a boat or the like. Multiple areas of interest can be identified.
[0048] According to another embodiment, only a part of the image data depicting the area or areas of interest is passed on to the transmission unit.
[0049] The image data depicting the area of interest can be requested by the transmission unit or by the base station via the transmission unit. During step b), preferably only the portion of the image data depicting the area of interest is forwarded to the transmission unit.
[0050] The embodiments and features described for the proposed aircraft apply accordingly to the proposed system and / or the proposed method, and vice versa.
[0051] The term "one" here should not necessarily be understood as restricting the number to exactly one element. Rather, it can also refer to multiple elements, such as two, three, or more. Similarly, every other counter used here should not be interpreted as restricting the number to the exact number stated. Instead, numerical deviations, both higher and lower, are possible unless otherwise specified.
[0052] Other possible implementations of the aircraft, system, and / or method also include combinations of features or embodiments described previously or subsequently with regard to the exemplary embodiments, even if not explicitly mentioned. In such cases, the person skilled in the art will also add individual aspects as improvements or additions to the respective basic form of the aircraft, system, and / or method.
[0053] Further advantageous embodiments and aspects of the aircraft, system, and / or method are the subject of the dependent claims and the exemplary embodiments of the aircraft, system, and / or method described below. The aircraft, system, and / or method are further explained below with reference to preferred embodiments and the accompanying figures. Fig. Figure 1 shows a schematic view of an embodiment of a system for the comprehensive continuous monitoring of the Earth's surface; Fig. Figure 2 shows a schematic view of an embodiment of a camera group in view II according to the Fig. 1; Fig. Figure 3 shows a schematic sectional view of the camera group according to section line III-III of the Fig. 2; Fig. Figure 4 shows another schematic sectional view of the camera group according to section line IV-IV of the Fig. 2; Fig. 5 another schematic view of the camera group according to Fig. 2; Fig. Figure 6 shows a schematic top view of the Earth's surface; and Fig. Figure 7 shows a schematic block diagram of an embodiment of a method for operating an aircraft with a camera group according to Fig. 2.
[0054] In the figures, identical or functionally equivalent elements have been given the same reference symbols, unless otherwise indicated.
[0055] The Fig. Figure 1 shows a schematic view of an embodiment of a system 1 for the comprehensive continuous monitoring of the Earth's surface 2.
[0056] In particular, System 1 is a Wide-Area Persistent Surveillance System (WAPSS) or an Airborne Wide-Area Persistent Surveillance System (AWAPSS). Specifically, System 1 serves to support reconnaissance in maritime disaster situations by means of automated data processing in the air. System 1 can be used for both civilian and military purposes. The Earth's surface 2 can be a water surface. Accordingly, the terms "Earth's surface" and "water surface" can be interchanged as appropriate. In particular, the Earth's surface 2 can be the water surface of a sea or an inland body of water.
[0057] System 1 comprises at least one aircraft 3. System 1 can comprise any number of aircraft 3. For example, System 1 can include a swarm of aircraft 3. However, it is assumed below that System 1 comprises exactly one aircraft 3. The aircraft 3 can be manned or unmanned (UAV). For the purposes of this text, however, it is assumed that the aircraft 3 is unmanned. Preferably, the aircraft 3 is a drone, in particular a reconnaissance drone. The terms "aircraft" and "drone" can therefore be used interchangeably throughout this text.
[0058] Aircraft 3 can be an airplane, in particular a fixed-wing aircraft. However, aircraft 3 can also be a rotorcraft or a helicopter. For the purposes of this discussion, aircraft 3 is assumed to be a fixed-wing aircraft. Aircraft 3 can have a fuselage 4, a wing 5 attached to the fuselage 4, and a propulsion system 6 for driving aircraft 3. The propulsion system 6 can drive a propeller 7. The propeller 7 can be a pusher propeller. The propulsion system 6 can, for example, be a two-cylinder two-stroke engine or a Wankel engine. The propulsion system 6 can also be a turboprop engine.
[0059] Aircraft 3, for example, can have a wingspan of approximately 5 m, a length of approximately 3 m, and a height of approximately 1 m. Preferably, aircraft 3 has a takeoff mass of less than 90 kg. Aircraft 3 can travel at a maximum speed of 90 km / h at an altitude of over 5,000 m in a flight direction F. The maximum flight duration of aircraft 3 can be 12 hours. However, the values mentioned above are only examples.
[0060] The aircraft 3 carries a camera array 8 for continuous, area-wide monitoring of the Earth's surface 2. The camera array 8 is at least partially integrated into the fuselage 4. However, the camera array 8 can also protrude at least partially from the fuselage 4. The camera array 8 faces the Earth's surface 2. This means that the camera array 8 is mounted on the underside of the fuselage 4. The camera array 8 serves to capture and / or monitor the Earth's surface 2. Image data depicting the Earth's surface 2 can be generated using the camera array 8. The camera array 8 can be rotatable and / or tiltable.
[0061] System 1 further comprises a base station 9 assigned to the aircraft 3. The base station 9 can be located on the Earth's surface 2. The base station 9 can be stationary. Alternatively, the base station 9 can also be mobile. In the latter case, the base station 9 can be transported by a vehicle. The vehicle can be a land vehicle, a watercraft, or an aircraft, in particular a manned aircraft. If the vehicle is an aircraft, the base station 9 is not located on the Earth's surface 2, but in the air.
[0062] A data link or data connection 10 exists between base station 9 and aircraft 3, specifically between the aircraft's camera group 8. Data can be exchanged between base station 9 and aircraft 3 via this data connection 10. For example, base station 9 can transmit data to aircraft 3 in the form of control signals. The control signals sent by base station 9 to aircraft 3 and / or camera group 8 can include commands to rotate and / or tilt camera group 8. Furthermore, commands to transmit and record specific areas of interest and / or camera settings can also be transmitted. Additionally, commands can be sent to switch between different cameras within camera group 8, for example, for recording in different spectral ranges.
[0063] Aircraft 3 can, for example, transmit image data to base station 9. Base station 9 can, for example, have a processing unit, an output unit (such as a monitor or screen), and an input unit (such as a touchscreen). Furthermore, base station 9 can also have a transmitting unit and / or a receiving unit.
[0064] The Fig. Figure 2 shows an embodiment of the camera group 8 in view II according to the Fig. 1. The Fig. Figure 3 shows a sectional view of camera group 8 according to section line III-III of the Fig. 2. The Fig. Figure 4 shows another sectional view of camera group 8 according to section line IV-IV of the Fig. 2. The following refers to the Fig. 2 to 4 referenced simultaneously.
[0065] In particular, the Fig. 2 A preferred embodiment of a camera group 8 as previously mentioned. The camera group 8 comprises N camera modules 11, 12, 13, 14, where N ≥ 2. In other words, at least two camera modules 11, 12, 13, 14 are provided. However, more than two camera modules 11, 12, 13, 14 may also be provided. In the present case, the camera group 8 comprises exactly four camera modules 11, 12, 13, 14. However, two, three, five, six, seven, eight, or more than eight camera modules 11, 12, 13, 14 may also be provided.
[0066] Each of the camera modules 11, 12, 13, 14 has M cameras 15, 16, 17, 18, where M ≥ 1. In other words, each camera module 11, 12, 13, 14 has at least exactly one camera 15, 16, 17, 18. However, there can be more than one camera 15, 16, 17, 18 per camera module 11, 12, 13, 14. In this case, each camera module 11, 12, 13, 14 has exactly one camera 15, 16, 17, 18. However, there can also be two, three, four, five, six, seven, eight, or more than eight cameras 15, 16, 17, 18 per camera module 11, 12, 13, 14.
[0067] Cameras 15, 16, 17, and 18 are arranged in a grid or matrix configuration. "Grid" or "matrix" in this context means that cameras 15, 16, 17, and 18 are arranged side-by-side and / or one above the other in rows and columns. In the example case where camera group 8 comprises four cameras 15, 16, 17, and 18, cameras 15 and 16, as well as cameras 17 and 18, are each placed in the same row. Similarly, cameras 15 and 17, as well as cameras 16 and 18, are each placed in the same column.
[0068] Camera group 8 is assigned a coordinate system with a longitudinal direction (x-direction), a vertical direction (y-direction), and a horizontal direction (z-direction). The x, y, and z directions are perpendicular to each other. The flight direction (F) can be oriented opposite to the x-direction (x). The y-direction (y) can be oriented from the aircraft 3 towards the Earth's surface (2).
[0069] As the Fig. As shown in Figures 2 to 4, each camera 15, 16, 17, 18 is assigned a viewing axis 19, 20, 21, 22. The cameras 15, 16, 17, 18 are arranged such that all viewing axes 19, 20, 21, 22 are inclined relative to each other. "Inclined" in this context means, in particular, that the viewing axes 19, 20, 21, 22 are neither parallel nor perpendicular to each other. For this purpose, each viewing axis 19, 20, 21, 22 is inclined or tilted relative to the x-direction x, relative to the y-direction y, and / or relative to the z-direction z. The sight lines 19, 20, 21, 22 or at least part of the sight lines 19, 20, 21, 22 can intersect.
[0070] Since camera group 8 comprises several cameras 15, 16, 17, 18, camera group 8 can also be referred to as a cluster camera. Since the sight lines 19, 20, 21, 22 are arranged at an angle to each other, camera group 8 can also be referred to as an oblique cluster camera.
[0071] The Fig. Figure 5 shows a highly simplified schematic view of camera group 8.
[0072] As mentioned previously, the camera group 8 N comprises camera modules 11, 12, 13, 14 with N ≥ 2. Of these camera modules 11, 12, 13, 14, the Fig. Figure 5 shows only camera modules 11, 12, 13, but any number of additional camera modules 14, for example the previously mentioned camera module 14, can be provided in addition to camera modules 11, 12, 13. This is shown in the Fig. 5, containing a dashed box with three dots indicated. Since the camera module 14 is in the Fig. Since section 5 is not shown, only camera modules 11, 12, and 13 will be discussed below.
[0073] Each camera module 11, 12, 13 in turn has M cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m, where M ≥ 1. For example, if M = 5, then each camera module 11, 12, 13 is assigned five cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m. All cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m together form a mechanism 23 of camera group 8. The mechanism 23 is rigid, so that the cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m or the camera modules 11, 12, 13 cannot move relative to each other.
[0074] Camera modules 11, 12, and 13 can all have the same number of cameras: 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, and 17-m. However, this is not mandatory. Accordingly, camera modules 11, 12, and 13 can also each have a different number of cameras: 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, and 17-m.
[0075] Each camera module 11, 12, 13 has a processing unit 24, 25, 26 for processing image data provided by cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m. Thus, N processing units 24, 25, 26 are provided, with N ≥ 2. Processing unit 24 is assigned to camera module 11, processing unit 25 is assigned to camera module 12, and processing unit 26 is assigned to camera module 13. Accordingly, cameras 15-1, 15-2, 15-3, 15-4, 15-m are assigned to processing unit 24, cameras 16-1, 16-2, 16-3, 16-4, 16-m are assigned to processing unit 25, and cameras 17-1, 17-2, 17-3, 17-4, 17-m are assigned to processing unit 26.
[0076] The processing units 24, 25, 26 can also be referred to as video processing and media servers. Each processing unit 24, 25, 26 has a camera interface 27, a video and data processor 28, and storage 29. The processing units 24, 25, 26 are embedded PCs (personal computers) and can therefore also be referred to as such. Each processing unit 24, 25, 26 has sufficient computing power to process the image data provided by the respective cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m in real time.
[0077] Each camera 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m is coupled to the respective camera interface 27 of the processing unit 24, 25, 26 assigned to the respective camera 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m by means of a bus 30, 31, 32, 33, 34. Each camera 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m has its own bus 30, 31, 32, 33, 34.
[0078] The camera modules 11, 12, 13, in particular the processing units 24, 25, 26, are interconnected by means of a network 35. The network 35 has a data transfer rate C1. However, the data transfer rate C1 of the network 35 is preferably too low to transmit the image data provided by the cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m and / or the processed image data of the processing units 24, 25, 26 in real time.
[0079] In this context, "data transfer rate" refers to the speed at which a specific amount of data can be transmitted over a transmission channel within a given time interval. Common units of measurement for data transfer rate C1 are bits per second (bps), kilobits per second (kbps), megabits per second (Mbps), or gigabits per second (Gbps).
[0080] Network 35 can be a local Ethernet network. Network 35 comprises several network switches 36, 37, and 38. Each camera module 11, 12, and 13 is assigned to one of the switches 36, 37, and 38. Thus, switch 36 is assigned to camera module 11, switch 37 to camera module 12, and switch 38 to camera module 13. N switches 36, 37, and 38 are provided.
[0081] Switch 36 is connected to processing unit 24 via connection 39, switch 37 is connected to processing unit 25 via connection 40, and switch 38 is connected to processing unit 26 via connection 41. Switches 36, 37, and 38 are interconnected via connections 42, 43, and 44, respectively. Connections 42, 43, and 44 form a ring topology. Depending on the size of the network 35, a combination of ring and star topology may be used to reduce latency. Connections 39, 40, 41, 42, 43, and 44 can be LAN cables (Local Area Network).
[0082] Camera group 8 also includes a transmission unit 45 connected to network 35 for establishing the data connection 10 between camera group 8 and base station 9. The transmission unit 45 is configured to transmit the processed image data from processing units 24, 25, and 26 to base station 9. The data connection 10 can also be referred to as a data link. The transmission unit 45 is connected to network 35, for example to switch 36, by means of a connection 46. The connection 46 can also be a LAN cable.
[0083] The transmission unit 45 can also be an embedded PC. The transmission unit 45 can be a dedicated master module, which can also have cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m as previously mentioned. The transmission unit 45 is used for data exchange with the base station 9. The transmission unit 45 is connected to the base station 9 via data link 10.
[0084] In this context, a "data connection" or "data link" refers to a connection, particularly a radio connection, between the transmission unit 45 and the base station 9 for data transmission. The data connection 10 has a data transmission rate C2. The data transmission rate C1 of the network 35 is greater than or equal to, preferably similar to, the data transmission rate C2 of the data connection 10. The data transmission rate C2 is a constant data transmission rate and, from the perspective of the camera group 8, is severely limited.
[0085] The Fig. Figure 6 shows a top view of the Earth's surface.
[0086] For the sake of simplicity, it will be assumed below that camera group 8, as in the Fig. Figures 2 to 4 show four camera modules 11, 12, 13, 14 with four cameras 15, 16, 17, 18. The camera group 8 can capture or create a video mosaic 47 of the Earth's surface 2. The video mosaic 47 can be a wide-angle image of the Earth's surface 2. The video mosaic 47 can have any geometry. For simplicity, the video mosaic 47 is shown as rectangular. If the video mosaic 47 is square, it can have a side length of several kilometers. A live stream of the video mosaic 47 can be transmitted to the base station 9 via the data connection 10. For simplicity, the Fig. 6. Perspective distortions and optical distortions caused by lenses of cameras 15, 16, 17, 18 are neglected.
[0087] Cameras 15, 16, 17, and 18 capture different sections 48, 49, 50, and 51 of the video mosaic 47. Sections 48, 49, 50, and 51 overlap in overlap areas 52 and 53. Sections 48, 49, 50, and 51 can have any geometry. Fig. Sections 48, 49, 50, and 51 are shown as rectangles for simplification. Together, these sections form the video mosaic 47. Section 48 can, for example, be captured by camera module 12. Section 49 can, for example, be captured by camera module 11. Section 50 can, for example, be captured by camera module 14. Section 51 can, for example, be captured by camera module 13.
[0088] In the event that the camera modules 11, 12, 13, 14 each have several cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m, sections 48, 49, 50, 51 are further subdivided into overlapping subsections. Each of the subsections is assigned at least one of the cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m.
[0089] The transmission unit 45 is configured to assemble the image data assigned to sections 48, 49, 50, 51, or to the image data assigned to sections 48, 49, 50, 51, into the video mosaic 47. The transmission unit 45 can also be configured to assemble the subsections of sections 48, 49, 50, 51, or image data of the subsections captured by cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m, into the video mosaic 47. In particular, the processing unit 24, 25, 26 of each of the camera modules 11, 12, 13, 14 is also configured to assemble the sections 48, 49, 50, 51 or the image data of the cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m into the video mosaic 47.
[0090] Sections 48, 49, 50, and 51 are stitched together. In photography, "stitching" refers to creating a large photograph from several smaller individual shots, which typically show overlapping sections of the subject. The size of the overlapping areas 52 and 53 depends on the calibration, its configuration, and the image quality. For a functioning calibration, it is essential that meaningful information can be extracted from the respective overlapping areas 52 and 53 to establish a relationship between the image data. As a rule of thumb, 5 to 10% of the image width, or at least 200 pixels, has proven to be a reasonable value for the width of the overlapping areas 52 and 53.
[0091] Objects 54 and 55 can be located on the Earth's surface 2. These objects can be, for example, boats or people in the water. Each object 54 or 55 is associated with a region of interest 56 or 57. The respective region of interest 56 or 57 encompasses the object 54 or 55 to which it is assigned. The regions of interest 56 and 57 can also be congruent with the objects 54 and 55. The regions of interest 56 and 57 can each have any geometry and / or size. The geometry and / or size of the regions of interest 56 and 57 can depend on the geometry and / or size of the objects 54 and 55. The objects 54 and 55 can also be located in the overlapping regions 52 and 53.
[0092] The following section explains the functionality of System 1 and / or Camera Group 8: Fig. 1 to 6 simultaneously received. The processing unit 24, 25, 26 of each of the camera modules 11, 12, 13, 14 is configured to identify the respective area of interest 56, 57 of the Earth's surface 2 based on the image data captured by the cameras 15, 15-1, 15-2, 15-3, 15-4, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18 and to forward only a portion of the image data depicting the respective area of interest 56, 57 to the transmission unit 45. The transmission unit 45 is specifically configured to request the portion of the image data depicting the respective area of interest 56, 57 only from the processing units 24, 25, 26 of those camera modules 11, 12, 13, 14 that capture the area of interest 56, 57. This reduces the amount of data to be transmitted over the network 35.
[0093] The primary task of System 1 is to support reconnaissance in maritime disaster situations by means of automated data processing in the air. Camera group 8 is responsible for capturing high-resolution wide-angle images in the form of video mosaic 47 and performing automated preliminary analysis in processing units 24, 25, and 26 to identify the areas of interest 56 and 57 within video mosaic 47.
[0094] Aerial recording of the areas of interest 56 and 57 is necessary because it is not possible to transmit the image data captured by cameras 15, 15-1, 15-2, 15-3, 15-4, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, and 18 in their entirety and at a usable refresh rate via data link 10. Therefore, only the areas of interest 56 and 57 are transmitted in high quality to allow for more precise evaluation and classification at base station 9.
[0095] The previously described setup of camera group 8 utilizes so-called OTS (Off the Shelf) products, particularly so-called COTS (Commercial Off the Shelf) products. In other words, cameras 15, 15-1, 15-2, 15-3, 15-4, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18, processing units 24, 25, 26, and transmission unit 45 are mass-produced products. This enables cost-effective manufacturing of camera group 8. Furthermore, it allows for unlimited scalability, high flexibility, and rapid further development, for example, with regard to extended functions.
[0096] Aircraft 3, or rather camera group 8, can resolve a general trade-off or conflict of objectives inherent in airborne WAPS (Wide-Area Persistent Surveillance) applications. Very high resolutions and computing power are required to perform WAPS. At the same time, space, weight, and data transmission are very different and severely limited in airplanes, helicopters, and drones.
[0097] This is particularly true when smaller aircraft are to be used for reasons of cost, operational readiness, and / or effort and personnel. Finished products therefore suffer from the fact that development costs and times are high for such specialized equipment, and at the same time, changes and upgrades, such as higher resolution, require the products to be completely redesigned, and the individual product's application is limited due to its weight class.
[0098] Internally known cluster cameras (not shown) also feature a classic von Neumann computer architecture. In this architecture, individual or multiple components of a computer are interconnected via a central bus system, enabling data exchange. The well-known problem with this architecture is that, due to the so-called "von Neumann bottleneck," a computer with this architecture is only capable of handling multiple resource-intensive tasks simultaneously to a limited extent. In the case of a cluster camera with a large number of cameras, however, very large amounts of data from different sources must be transferred and processed using simple arithmetic operations.
[0099] This quickly turns the bus system into a bottleneck, especially when several essential functions, such as processing, storage, analysis, or the like, are used simultaneously. This problem severely limits the scalability of cluster cameras, as while the computing power of individual components can be very high, the data exchange between these components is limited.
[0100] During the development of camera group 8, it was determined that the planned application of camera group 8 and its internal processing is one with a high degree of mathematical separability due to its spatial independence. Therefore, it is advisable to design the hardware of camera group 8 to be separable as well, in order to overcome the limitations described above. The use of OTS products is advantageous in this regard due to their interchangeability.
[0101] To solve the problem outlined above, an innovative approach is being pursued: combining the technical disciplines of "High Performance Parallel Computing" with "Embedded Computing in Airborne Systems." By using embedded systems, a data center approach can be miniaturized and applied to avionics. By considering the individual product camera module 11, 12, 13, 14 as a data center comprised of embedded OTS products, high scalability and versatile applicability are achieved, thus enabling the long-term and broadband fulfillment of the payload requirements for various carriers in the avionics sector.
[0102] As the Fig. As shown in Figure 5, a top-level system architecture can be implemented using exemplary OTS products in the form of cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m. The camera modules 11, 12, and 13 are connected to a decentralized or distributed computing system via network 35, which consists of switches 36, 37, and 38 and their interconnections 42, 43, and 44. The computing load is distributed entirely among the processing units 24, 25, 26, which perform the image processing, analysis and storage of the image data from their respective cameras 15-1, 15-2, 15-3, 15-4, 15-m, 16-1, 16-2, 16-3, 16-4, 16-m, 17-1, 17-2, 17-3, 17-4, 17-m.
[0103] The transmission unit 45 serves solely as a coordinator and relay. All images, for example, in the form of the areas of interest 56 and 57, which are to be transmitted via data connection 10, are requested by the transmission unit 45 from the relevant camera modules 11, 12, and 13, possibly combined additively, and sent to the base station 9. The special feature of this distributed system is that the previously described so-called Von Neumann bottleneck is eliminated.
[0104] The unavoidable bottleneck in avionics in general is the data link 10. Thanks to the aforementioned mathematical separability, the bandwidth of network 35 hardly needs to be greater than that of data link 10, because data that cannot be sent to base station 9 does not need to be transmitted internally within network 35. This results in the particular advantages in the scalability of the system. If a higher image resolution is required, the camera modules 11, 12, 13, including network 35, scale linearly. However, the data transmission rates C1 and C2 in network 35 and via data link 10, as well as the transmission unit 45, remain constant.
[0105] The data exchanged via network 35, transmission unit 45, and data connection 10, besides image data, consists only of data generated by C&C (Command and Control) and calibration processes. These are also constant and, due to their relatively small proportion, can be disregarded; moreover, they do not occur during the operation of camera group 8 anyway.
[0106] The Fig. Figure 7 shows a schematic block diagram of an embodiment of a method for operating the aircraft 3.
[0107] In the process, image data provided by cameras 15, 15-1, 15-2, 15-3, 15-4, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18 are processed in a step S1 using the processing units 24, 25, 26 of the camera modules 11, 12, 13, 14.
[0108] In step S2, which can be performed simultaneously with step S1, the processed image data from processing units 24, 25, and 26 are transmitted to base station 9 using transmission unit 45. The data transmission rate C1 of network 35 is too low to transmit the image data provided by cameras 15, 15-1, 15-2, 15-3, 15-4, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, and 18, and / or the processed image data from processing units 24, 25, and 26, in real time.
[0109] The processing unit 24, 25, 26 of each of the camera modules 11, 12, 13, 14 can identify a respective area of interest 56, 57 of the Earth's surface 2 based on the image data. In particular, only a portion of the image data depicting the area of interest 56, 57 is then passed on to the transmission unit 45 and from there to the base station 9.
[0110] Although the present invention has been described using exemplary embodiments, it can be modified in many ways. REFERENCE MARK LIST 1 system 2 Earth's surface 3 aircraft 4 Hull 5 Wing 6 Drive 7 propeller 8 camera group 9 Base station 10 Data connection 11 Camera module 12 camera modules 13 Camera module 14 camera modules 15 cameras 15-1 Camera 15-2 camera 15-3 camera 15-4 camera 15-meter camera 16 cameras 16-1 Camera 16-2 camera 16-3 camera 16-4 camera 16-meter camera 17 Camera 17-1 Camera 17-2 Camera 17-3 camera 17-4 camera 17-m camera 18 Camera 19 Line of sight 20 sightline 21 Line of sight 22 Line of sight 23 Mechanics 24 processing units 25 processing units 26 processing units 27 Camera interface 28 Video and data processor 29 storage 30 Bus 31 Bus 32 Bus 33 Bus 34 Bus 35 Network 36 Switch 37 Switch 38 Switch 39 connection 40 connection 41 connection 42 connection 43 connection 44 connection 45 transmission unit 46 connection 47 Video mosaic Section 48 Section 49 Section 50 Section 51 52 Overlap area 53 Overlap area 54 objects 55 objects 56 areas of interest 57 areas of interest C1 data transfer rate C2 data transfer rate F Flight direction S1 step S2 step x x-direction y y-direction z z-direction
Claims
[1] Aircraft (3) with a camera group (8) for continuous, comprehensive monitoring of the earth's surface (2), the camera group (8) comprising: several cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18), whose viewing axes (19, 20, 21, 22) are all arranged at an inclination to one another, N camera modules (11, 12, 13, 14) with N ≥ 2, wherein each camera module (11, 12, 13, 14) has M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) with M ≥ 1, and wherein each of the N camera modules (11, 12, 13, 14) has a processing unit (24, 25, 26) for processing the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18), a network (35) connecting the processing units (24, 25, 26) to one another, and a transmission unit (45) connected to the network (35) for data connection (10) of the camera group (8) to a base station (9) of the aircraft (3), wherein the transmission unit (45) is configured to transmit the processed image data of the processing units (24, 25, 26) to the base station (9), and wherein a data transmission rate (C1) of the network (35) is too small to transmit the image data provided by the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) and / or the processed image data of the processing units (24, 25, 26) in real time. [2] Aircraft according to claim 1, characterized bythat each processing unit (24, 25, 26) has sufficient computing power to process the image data provided by the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) in real time. [3] Aircraft according to claim 1 or 2, characterized by that the data transmission rate (C1) of the network (35) is greater than or equal to, preferably similar to, a data transmission rate (C2) between the transmission unit (45) and the base station (9). [4] Aircraft according to any one of claims 1-3, characterized bythat the processing unit (24, 25, 26) of each of the N camera modules (11, 12, 13, 14) is configured to identify an area of interest (56, 57) or a plurality of areas of interest (56, 57) of the earth's surface (2) on the basis of the image data and to forward only a part of the image data depicting the area of interest (56, 57) or the areas of interest (56, 57) to the transmission unit (45). [5] Aircraft according to claim 4, characterized by that the transmission unit (45) is configured to request the part of the image data that depicts (56, 57) the region of interest or regions of interest (56, 57) from the processing units (24, 25, 26) of the N camera modules (11, 12, 13, 14). [6] Aircraft according to any one of claims 1-5, characterized bythat the processing unit (24, 25, 26) of each of the N camera modules (11, 12, 13, 14) is configured to combine the image data of the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) into a video mosaic (47). [7] Aircraft according to any one of claims 1-6, characterized by that the transmission unit (45) is configured to combine the image data of the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) and / or image data of the N camera modules (11, 12, 13, 14) into a video mosaic (47). [8] Aircraft according to any one of claims 1-7, characterized by that the processing unit (24, 25, 26) of each of the N camera modules (11, 12, 13, 14) has a camera interface (27), a video and data processor (28) and a memory (29). [9] Aircraft according to any one of claims 1-8, characterized bythat the network (35) has a plurality of switches (36, 37, 38), wherein each of the N camera modules (11, 12, 13, 14) is assigned a switch (36, 37, 38), and wherein the transmission unit (45) is connected to one of the switches (36, 37, 38). [10] Aircraft according to claim 9, characterized by that each of the N camera modules (11, 12, 13, 14) has one of the switches (36, 37, 38). [11] Aircraft according to any one of claims 1-10, characterized by a fuselage (4), a wing (5) attached to the fuselage (4), and a drive (6) attached to the fuselage (4) or to the wing (5), wherein the camera group (8) is attached to the fuselage (4) and / or to the wing (5). [12] System (1) with X aircraft (3) according to one of claims 1-11 with X ≥ 1 and the base station (9), wherein the transmission unit (45) transmits the processed image data of the processing units (24, 25, 26) to the base station (9). [13] Method for operating an aircraft (3) with a camera group (8) for the continuous, comprehensive monitoring of the earth's surface (2), wherein the camera group (8) comprises a plurality of cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18), the lines of sight (19, 20, 21, 22) of which are all arranged inclined to one another, N camera modules (11, 12, 13, 14) with N ≥ 2, wherein each camera module (11, 12, 13, 14) M of the cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) with M ≥ 1, and wherein each of the N camera modules (11, 12, 13, 14) has a processing unit (24, 25, 26), a network (35) which connects the processing units (24, 25, 26) to one another, and a transmission unit (45) connected to the network (35) for data connection (10) of the camera group (8) to a base station (9) of the aircraft (3),the method comprising the following steps: a) processing (S1) image data provided by the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) using the processing units (24, 25, 26) of the N camera modules (11, 12, 13, 14), and b) transmitting (S2) the processed image data of the processing units (24, 25, 26) to the base station (9) with the aid of the transmission unit (45), wherein a data transmission rate (C1) of the network (35) is too low to transmit the image data provided by the M cameras (15, 15-1, 15-2, 15-3, 15-3, 15-m, 16, 16-1, 16-2, 16-3, 16-4, 16-m, 17, 17-1, 17-2, 17-3, 17-4, 17-m, 18) and / or the processed image data of the processing units (24, 25, 26) in real time. [14] Method according to claim 13, characterized bythat the processing unit (24, 25, 26) of each of the N camera modules (11, 12, 13, 14) identifies one or more regions of interest (56, 57) of the earth's surface (2) on the basis of the image data. [15] Method according to claim 14, characterized by that only a part of the image data depicting the region of interest (56, 57) or the regions of interest (56, 57) is passed on to the transmission unit (45).
Citation Information
Patent Citations
Image capturing
EP2673953B1
Imaging system for UAV
US20120200703A1
