An autonomous imaging and control system for managing unmanned vehicles

Abstract

The invention relates to an autonomous imaging and command system comprising at least one unmanned aerial vehicle (10) with a downward-looking camera (12) configured with a wide field of view and arranged to periodically capture a field image (1) and transmit it to a controller (14); a first controller (14) connected to the camera (12) to provide data communication and configured to process the field image to identify objects; and a first communication module (16) connected to the first controller (14) to provide signal transmission; a swarm of unmanned vehicles (20) comprising a plurality of unmanned surface or ground vehicles, each having a second communication module (26) connected to the UAV's (10) first communication module (16) to provide signal transmission; and a second controller (24) connected to the second communication module (26) to provide data communication, and a propulsion apparatus (22) driven by the second controller (24). According to the autonomous imaging and control system, the first controller (14) of the UAV (10) is configured to detect initial position parameters of each swarm member within the field image (1) and to determine a target vector for each swarm member and transmit it to the second controller (24), and the second controller (24) of the unmanned vehicle (20) is configured to drive the propulsion apparatus (22) according to the detected target parameters.

Description

[0001] AN AUTONOMOUS IMAGING AND CONTROL SYSTEM FOR MANAGING UNMANNED VEHICLES

[0002] TECHNICAL FIELD

[0003] The present invention relates to an imaging and control system, and more particularly to an imaging and control system used with an unmanned aerial vehicle.

[0004] BACKGROUND ART

[0005] In the technical field, sea-level monitoring is performed via coastal stations, patrol vessels, satellite remote sensing and optical imaging. This makes it difficult to obtain accurate dynamic data due to challenges in coverage and focus, thereby necessitating development of an integrated system. When satellites acquire higher-resolution images, the observation swath narrows and the satellite loses land or sea references. The sea surface in maritime areas and around islands and reefs exhibits large undulations, which makes stable operation of unmanned surface vehicles (USVs) difficult during surveillance. Monitoring systems such as satellites, unmanned aerial vehicles (UAVs), USVs, islandreef bases, and buoy / floating platforms need to process large-format, high-frame-rate imagery online to extract information.

[0006] High vessel speeds (>60 knots) and wave action subject the gyroscopes and accelerometers of the Inertial Measurement Unit (IMU) to continuous shocks, reducing navigation accuracy and, especially in the absence of GNSS, requiring the use of an expensive IMU. Beyond-line-of-sight missions also require high-cost IMUs due to drift errors that increase with time and distance. Alongside high- performance navigation requirements, processor costs for navigation algorithms, power sources, and other auxiliary equipment also increase. Furthermore, in environments where GNSS is denied there is a risk of hostile electronic jamming and cyber security threats. In such cases, communication and data transfer by integrated systems may not be possible. Even in peacetime, strong solar flares can cause problems for electronic communications using traditional radio waves. In addition, if a situation known as the Kessler Syndrome occurs, communication via satellites may be impossible for decades.

[0007] CN107132530A concerns obtaining accurate dynamic data through coverage and intensive monitoring. The document discloses an air / space / ship / buoy / island-reef-based multi-sensor integrated monitoring system. In this system, satellite SAR image passive positioning is used; the UAV adopts a special multi-sensor monitoring pod such as radar / imaging / ultrasound; the unmanned ship uses an integrated monitoring system of search radar / imaging / ultrasound / laser; the island-reef base and coastal base use an integrated monitoring system of ADS-B / AIS / radar / panoramic imaging / monitoring imaging / ultrasound / laser; the buoy / floating platform uses a hemispherical panoramic imaging / laser system; the underwater vehicle uses a multi-channel infrared CCD / ultrasound and sonar monitoring system; a comprehensive background analysis system is established; and accurate dynamic data are obtained by acquiring illegal, non-cooperative air and sea targets and comprehensive marine environmental information in island-reef and offshore structures, thereby solving technical problems of coverage and intensive monitoring.

[0008] SUMMARY OF THE INVENTION

[0009] An object of the invention is to eliminate the need for sophisticated navigation systems on unmanned vehicles.

[0010] Another object of the invention is to reduce the cost incurred by swarm elements operating on the surface (land or sea) during a given mission.

[0011] Another object of the invention is to increase the probability of mission success by means of an autonomous imaging system with collective intelligence.

[0012] Another object of the invention is to ensure that unmanned vehicles resist electromagnetic jamming during a mission.

[0013] In order to achieve the above objectives, the invention relates to an autonomous imaging and command system comprising at least one unmanned aerial vehicle configured with a downwardlooking camera having a wide field of view and arranged to periodically capture a field image within its field of view and transmit it to a controller; a first controller connected to the camera to provide data communication and configured to process the field image to identify objects; and a first communication module connected to the first controller to provide signal transmission. The system further comprises a swarm of unmanned surface or ground vehicles having, for each unmanned surface vehicle, a second communication module connected to the UAV's first communication module for signal transmission; and a second controller connected to the second communication module for data communication and driving a propulsion apparatus. The autonomous imaging and command system is configured such that the first controller of the UAV detects initial position parameters of each member of the swarm within the field image and determines a target vector for each swarm member and transmits it to the second controller; and the second controller of the unmanned surface vehicle drives the propulsion apparatus according to the detected target parameters. Thus, unmanned surface vehicles can be managed from the UAV without using navigation systems aboard the USVs. With this system, the high-performance navigation system is carried on the UAV, reducing the number of navigation systems required. Additionally, if the unmanned surface vehicle(s) perform a one-way attack ("kamikaze") mission during a military operation, because navigation is provided on the UAV, the UAV's navigation system can be reused in another military or civilian mission.

[0014] In a preferred embodiment, the first controller is configured to verify whether the swarm members comply with the target parameters. In this way, it can be checked by the UAV whether the swarm members are moving toward the target.

[0015] In a preferred embodiment, when the first controller finds that the target parameters of the swarm members are incorrect, it determines a corrected target parameter and transmits it to the second controller, and the second controller is configured to drive the propulsion apparatus according to the corrected parameters. Thus, the UAV prevents an unmanned surface vehicle from proceeding toward a wrong target by providing information and redirects it to the correct target.

[0016] In a preferred embodiment, upon receiving a cancel signal for a swarm member, the first controller is configured to generate and transmit to the second controller a target parameter that causes the vehicle to move away from the target. Thus, the UAV manages withdrawal of an unmanned surface vehicle for which a cancel command has been issued.

[0017] In a preferred embodiment, the first controller is configured to detect that swarm members have reached their target parameters. Thus, the UAV can detect whether the unmanned surface vehicles have reached the target.

[0018] In a preferred embodiment, the first controller is configured to trigger detonation of an explosive carried by the swarm members that have reached the target parameters. Thus, in military applications, detonation is triggered by the UAV. In a preferred embodiment, the first controller is configured to determine, based on the detonated swarm members, that the target has been neutralized. Thus, in military applications, the UAV detects whether the target has been neutralized.

[0019] In a preferred embodiment, the first controller detects a target that has not been neutralized and is configured to direct at least a second unmanned surface vehicle toward the target parameters, and the second controller is configured to drive the propulsion apparatus according to the target parameters. Thus, in military applications, the UAV determines whether the target has been neutralized and, if not, directs the most suitably positioned unmanned vehicle toward the target.

[0020] In a preferred embodiment, the first controller is configured to send the target parameters to the second controller provided on the nearest unmanned surface vehicle. Thus, the surface vehicle is managed by the UAV.

[0021] In a preferred embodiment, each unmanned vehicle includes a coded LED source that provides signal communication to identify itself to the first communication module connected to the first controller. Thus, the unmanned vehicles are recognized by the UAV.

[0022] In a preferred embodiment, the first and second communication modules each comprise an optical receiver and an optical transmitter configured to provide data communication between them. Thus, optical communication is established between the UAV and the unmanned vehicle.

[0023] In a preferred embodiment, the optical receivers and transmitters in the first and second communication modules are configured to communicate via a pulse-train laser emission. Thus, communication between the UAV and the unmanned vehicle is established without being affected by electromagnetic interference.

[0024] In a preferred embodiment, the UAV includes a satellite communication module providing satellite data communication. Thus, images received from satellites are processed by the UAV.

[0025] In a preferred embodiment, the UAV further comprises a forward-looking second camera, configured to provide a narrow field-of-view, zoomed image of an area within the field image, and in data communication with the first controller provided on the UAV. Thus, positional information of the target within the field image is provided. In a preferred embodiment, the UAV and the surface vehicle are connected by a cable providing power and data transmission. Thus, high-speed data and power transfer are provided. With this arrangement, for the period during which the UAV remains tethered, the electrical power source it needs to carry onboard is reduced, increasing endurance and payload capacity. It also enables unmanned surface vehicles to withstand electromagnetic jamming during the mission.

[0026] BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a front view of a representative embodiment of the imaging and control system according to the invention.

[0028] Figure 2 is a schematic satellite view of a representative embodiment of the imaging and control system according to the invention.

[0029] Figure 3 is a front view of a representative embodiment of the cable connection between the UAV and the surface vehicle in the imaging and control system according to the invention.

[0030] DETAILED DESCRIPTION OF THE INVENTION

[0031] In this detailed description, the development according to the invention is described without limitation and only to better explain the subject, with reference to examples.

[0032] In Figure 1, a representative embodiment of the imaging and control system according to the invention is shown in front view. The imaging and control system is used with at least one UAV (10) and a plurality of surface or ground vehicles (20) with which the UAV (10) communicates. A camera (12) with a wide field of view is provided on a body (11) of the UAV (10). The camera (12) is aligned with and mounted on the central axis of the UAV (10). The camera (12) periodically captures a field image (1) within its field of view and transmits it to a first controller (14) to which it is connected for signal transmission. The first controller (14) is configured to use an artificial-intelligence algorithm that processes the field image (1). Al algorithms, particularly trained with offline data for military applications in positioning, arming and neutralizing unmanned surface vehicles (20)— in particular USVs— enable the UAV (10) to autonomously command the unmanned surface vehicles (20). On the UAV (10) there is also a second camera (18) spaced from the camera (12) that detects the unmanned surface vehicles (20) present in the field image (1). The first controller (14) is configured to organize information about targets within the field image (1) provided by the second camera (18). The first controller (10) examines the images from the camera (12) and the second camera (18) using an Al algorithm to determine the position information of the unmanned surface vehicles (20) and the target information. The target information determined by the first controller (14), as target parameters, is transmitted optically from a first communication module (16) provided on the UAV (10) to a second communication module (26) provided on the unmanned surface vehicle (20). Communication is provided by an optical receiver (162) provided on the first communication module (16) and an optical transmitter (262) provided on the second communication module (26). The information received by the second communication module (26) is transmitted to a second controller (24) of the unmanned surface vehicle (20). The second controller (24) processes the received information and is configured to drive the unmanned surface vehicle (20) via a propulsion apparatus (22) toward the target parameters. The unmanned surface vehicle (20) includes a coded LED source (28) that identifies itself and enables detection by the UAV (10). Detection and designation (naming) of the unmanned surface vehicles (20) are thereby provided. After transmitting the target parameters, the UAV (10) tracks the movement of the unmanned surface vehicle (20) toward the target parameters and detects whether it has arrived. The UAV (10) includes a satellite communication module (19) providing communication with a command center via a satellite (2). Commands from the command center are transmitted via the satellite communication module (19).

[0033] Figure 2 schematically illustrates, from a satellite perspective, an application scenario of a representative embodiment of the imaging and control system according to the invention. The UAVs (10) shown in Figure 2 are referred to from left to right as a first, second, and third UAV (10). A military use case is selected as an application scenario. The first UAV (10) commands three unmanned surface vehicles (20), in particular three surface vessels, within its field of view. The second unmanned aerial vehicle (10) commands four surface vessels (20) within its field of view. The third UAV (10) commands three surface vessels (20) within its field of view. In the military scenario, an attack command from the command center is transmitted from the satellite (2) shown in Figure 1 to each UAV (10). The UAVs (10) analyze the field images (1) and determine the position and targetinformation parameters of the surface vessels (20) for the attack command. The first controller (14) verifies that the detected parameters for the swarm members comply with the target parameters. If found suitable, the first UAV (10) positions the swarm members according to the target information parameters. The positioned swarm members are directed toward the target parameters by the first, second and third UAVs (10). When the UAVs (10) detect that the swarm members have reached the target parameters, the explosives mounted on the surface vessels (20) are triggered and a command is issued to neutralize the target. After the attack, the UAVs (10) analyze the field images (1) to check whether the targets have been neutralized and the damage status of the surface vessels (20). If the target has not been neutralized, the UAV (10) identifies the surface vessel that is tactically best suited for the follow-up attack and directs it toward the target that has not been neutralized. If the target parameters are found to be incorrect or a cancellation command from the command center is received, a second set of target parameters is entered by the UAVs (10) to cause the surface vessels to move away from the original target.

[0034] In an alternative military application scenario, the second UAV (10) shown in Figure 2 is positioned as the leader, and the first and third UAVs (10) as assistants. Information from the command center is transmitted to the second UAV (10) via the satellite (2) and executed by the second UAV (10). The second UAV (10) manages the positions of the first and third UAVs, positioning the first UAV to attack a first flank of the target and the third UAV to attack a second flank of the target. The swarm members managed by the second UAV (10) are positioned to attack toward the center of the target. The first controller (14) verifies that the detected parameters for the swarm members comply with the target parameters. If found suitable, the second UAV (10) positions the swarm members according to the target information parameters. After positioning is complete, the attack begins toward the target parameters upon issuance of the attack command by the second UAV (10). After the attack, the UAVs (10) analyze the field images (1) to check whether the targets have been neutralized and the damage status of the surface vessels (20). If the target has not been neutralized, the second UAV (10) identifies the surface vessel that is tactically best suited for the follow-up attack and directs it toward the target that has not been neutralized. If the target parameters are found to be incorrect or a cancellation command from the command center is received, a second set of target parameters is entered by the second UAV (10) to cause the surface vessels to move away from the original target.

[0035] Figure 3 is a front view of a representative embodiment of the cable connection between the UAV and the surface vessel in the imaging and control system according to the invention. A cable (3) is provided to enable data and power transmission between the UAV (10) and the surface vessel (20). Because cabled connections are made using physical cables, they are more secure than wireless connections and help protect data from unauthorized access. In addition, they experience fewer interruptions than wireless links during data transfer and provide higher-speed data transmission.

[0036] REFERENCE NUMERALS

[0037] 1 Field image

[0038] 2 Satellite 3 Cable

[0039] 10 Unmanned Aerial Vehicle (UAV)

[0040] 11 Body

[0041] 12 Camera 14 First controller

[0042] 16 First communication module

[0043] 162 Optical receiver

[0044] 18 Second camera

[0045] 19 Satellite communication module 20 Unmanned surface vehicle (USV)

[0046] 22 Propulsion apparatus

[0047] 24 Second controller

[0048] 26 Second communication module

[0049] 262 Optical transmitter 28 LED source

Claims

CLAIMS1. An autonomous imaging and command system comprising: at least one unmanned aerial vehicle (10) with a downward-looking camera (12) configured with a wide field of view and arranged to periodically capture a field image (1) within its field of view and transmit it to a controller (14); a first controller (14) connected to the camera (12) to provide data communication and configured to process the field image to identify objects; and a first communication module (16) connected to the first controller (14) to provide signal transmission; a swarm of unmanned surface or ground vehicles (20) comprising a plurality of vehicles each having a second communication module (26) connected to the UAV's (10) first communication module (16) to provide signal transmission; and a second controller (24) connected to the second communication module (26) to provide data communication, and a propulsion apparatus (22) driven by the second controller (24); characterized in that the first controller (14) of the UAV (10) is configured to detect initial position parameters of each swarm member within the field image (1) and to determine a target vector for each swarm member and transmit it to the second controller (24), and the second controller (24) of the unmanned surface vehicle (20) is configured to drive the propulsion apparatus (22) according to the detected target parameters.

2. The imaging and command system according to claim 1, wherein the first controller (14) is configured to verify that the swarm members comply with the target parameters.

3. The imaging and command system according to claim 2, wherein the first controller (14) determines a corrected target parameter and transmits it to the second controller (24) when it finds that the target parameters of the swarm members are incorrect, and the second controller (24) is configured to drive the propulsion apparatus (22) according to the corrected parameters.

4. The imaging and command system according to any one of the preceding claims, wherein the first controller (14) is configured to generate and transmit to the second controller (24) a target parameter that causes the vehicle to move away from the target upon receiving a cancel signal for a swarm member.

5. The imaging and command system according to any one of the preceding claims, wherein the first controller (14) is configured to detect that the swarm members have reached the target parameters.

6. The imaging and command system according to claim 5, wherein the first controller (14) is configured to trigger detonation of an explosive carried by the swarm members that have reached the target parameters.

7. The imaging and command system according to any one of claims 5-6, wherein the first controller (14) is configured to determine, based on the detonated swarm members, that the target has been neutralized.

8. The imaging and command system according to claim 7, wherein the first controller (14) is configured to detect a target that has not been neutralized and to direct at least a second unmanned vehicle (20) toward the target parameters, and the second controller (24) is configured to drive the propulsion apparatus (22) according to the target parameters.

9. The imaging and command system according to claim 8, wherein the first controller (14) is configured to send the target parameters to the second controller (24) provided on the nearest unmanned surface vehicle (20).

10. The imaging and command system according to any one of the preceding claims, wherein each unmanned surface vehicle (20) comprises a coded LED source (28) that provides signal communication to identify itself to the first communication module (16) connected to the first controller (14).

11. The imaging and command system according to any one of the preceding claims, wherein the first and second communication modules (16, 26) comprise an optical receiver (162) and an optical transmitter (262) configured to provide data communication with each other.

12. The imaging and command system according to claim 11, wherein the optical receiver (162) and optical transmitter (262) in the first and second communication modules (16, 26) are configured to communicate via a pulse-train laser emission.

13. The imaging and command system according to any one of the preceding claims, wherein the UAV (10) comprises a satellite communication module (19) providing satellite (2) data communication.

14. The imaging and command system according to any one of the preceding claims, wherein the UAV (10) comprises a forward-looking second camera (18) configured to provide a narrowfield-of-view, zoomed image of an area within the field image in data communication with the first controller (14) provided on the UAV (10).

15. The imaging and command system according to any one of the preceding claims, wherein a cable (3) is provided between the UAV (10) and the unmanned surface vehicle (20) to provide power transmission.

Images