Information control system, control method, and program

Abstract

[Problem] To more accurately determine the position of an object or more accurately analyze the object in the monitoring or investigation of the object on the water using an unmanned vehicle. [Solution] The present invention provides an information control system comprising: a moving body group composed of a plurality of moving bodies each provided with a measurement sensor capable of acquiring measurement data obtained by measuring an object in a state where at least a part thereof is exposed above the water; a moving body operation control unit for controlling the operations of the moving bodies; a measurement data acquisition unit for acquiring first measurement data obtained by measuring the object from a first position by means of at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by means of the moving body or another moving body; an object position determination unit for determining position information of the object or determining the validity of the determined position information on the basis of the first measurement data and the second measurement data; and an information output unit for displaying and outputting the position information of the object determined by the object position determination unit or transmitting the position information to the outside.

Description

Information control system, control method and program

[0001] The present invention relates to an information control system, a control method, and a program.

[0002] Patent document 1 discloses a technology that improves the ease of identifying suspicious ships by using an external device that is connected via a network to an imaging device equipped with an imaging unit for capturing images of the subject ship, and that is equipped with: an acquisition means for acquiring image data including the ship captured by the imaging unit; a display means for displaying the image data; an analysis means for extracting estimated ship information of any ship contained in the image data based on the image data; a receiving means for receiving ship information based on wireless communication from the ship; and a comparison means for comparing the estimated ship information extracted by the analysis means with the ship information received by the receiving means, and the display means displays a warning in addition to the image data if the comparison means finds a difference between the estimated ship information and the ship information.

[0003] Marine surveillance using manned patrol vessels and research vessels has traditionally been conducted to prevent nuisance and illegal fishing by ships and divers navigating the ocean surface, as well as for the purpose of conducting ecological surveys of marine life such as whales and dolphins. However, because the oceanic areas subject to surveillance or survey are extremely vast, there are limits to the areas that can be monitored or surveyed by manned patrol vessels and research vessels, resulting in the problem of areas remaining uncovered. Furthermore, when multiple manned patrol vessels are coordinated for surveillance or survey, it is difficult to quickly control and manage them manually, making it difficult to properly coordinate and measure monitored objects. Furthermore, it is not easy to train personnel with the necessary skills. Against this background, the use of autonomously navigating vessels and other unmanned vessels has been considered in recent years, and it is expected that they will be used for the aforementioned surveillance of suspicious vessels and ecological surveys.

[0004] Japanese Patent Application Laid-Open No. 2018-19359

[0005] As in Patent Document 1, when determining whether a ship is suspicious by comparing the ship's characteristic information extracted from captured image data with the ship information received from the ship via wireless communication, if the ship's exact position cannot be determined, there is a possibility that the target ship to be compared with the ship information received from the ship will be incorrect, so it is necessary to detect the ship's position more accurately.

[0006] In addition, when requesting continuous measurement tasks to continuously measure ships, divers, or marine life such as whales and dolphins detected by unmanned vessels, etc., or tracking tasks to track them, it is necessary to provide more accurate location information of the detected objects to other unmanned vessels or external systems.

[0007] Therefore, the present invention has been made in consideration of at least one of the above problems, and one object of the present invention is to provide a system or control method, etc., that can more accurately determine the position of an object or more accurately analyze the object when monitoring or investigating an object on water using an unmanned aerial vehicle.

[0008] According to the present invention, an information control system is provided which includes: a group of mobile bodies consisting of multiple mobile bodies each equipped with a measurement sensor capable of acquiring measurement data measuring an object with at least a portion thereof above water; a mobile body operation control unit which controls the operation of the mobile bodies; a measurement data acquisition unit which acquires first measurement data measured of the object from a first position by at least one of the mobile bodies in the group of mobile bodies and second measurement data measured of the object from a second position by the mobile body or another of the mobile bodies; an object position determination unit which determines position information including at least one of relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; and an information output unit which displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside the analysis processing results of an object analysis unit which analyzes the object based on the position information.

[0009] According to the present invention, when an unmanned aerial vehicle is used to monitor or survey an object on water, the position of the object can be determined more accurately or the object can be analyzed more accurately.

[0010] 1 is an overall configuration diagram of an information control system 1 according to an embodiment of the present invention. FIG. 1 is a diagram showing an example of an implementation image when the information control system 1 is implemented in real space. FIG. 2 is a diagram showing stakeholders related to the information control system 1. FIG. 3 is a configuration diagram showing a platoon made up of unmanned boats 1000. FIG. 4 is a conceptual diagram showing an unmanned boat 1000 deployed on the sea monitoring or investigating an object 7000. FIG. 5 is a functional block diagram showing the functional configuration of the unmanned boat 1000. FIG. 6 is a functional block diagram showing the functional configuration of the overall control system 2000. FIG. 7 is a diagram showing items to be determined by a detection determination unit 2200. FIG. 8 is a conceptual diagram showing a position calculation method when independent surveying is used. FIG. 9 is a conceptual diagram showing a position calculation method when triangulation is used. FIG. 10 is a conceptual diagram showing a position calculation method when geometric calculation is used. FIG. 11 is a hardware configuration diagram of the overall control system 2000. FIG. 12 is a flowchart showing the processing flow of the information control system 1. FIG. 13 is a sequence diagram showing the exchange of signals between systems within the information control system 1. FIG. 14 is a flowchart showing an example of a processing flow for determining a position calculation method by a position calculation method determination unit 2300. 2 is a flowchart showing an example of a process flow for determining a detailed measurement operation by the detailed measurement operation control unit 2400. FIG. 3 is a conceptual diagram showing a process for determining detailed measurement by the detailed measurement operation control unit 2400. FIG. 4 is a diagram showing a first example of a combination of machines selected by the measurement machine selection unit 2420. FIG. 5 is a diagram showing a second example of a combination of machines selected by the measurement machine selection unit 2420. FIG. 6 is a diagram showing a third example of a combination of machines selected by the measurement machine selection unit 2420. FIG. 7 is a flowchart showing an example of a process flow for calculating a position using single surveying by the detailed position determination unit 2500. FIG. 8 is a flowchart showing an example of a process flow for calculating a position using triangulation by the detailed position determination unit 2500. FIG. 9 is a flowchart showing an example of a process flow for calculating a position using geometric calculation by the detailed position determination unit 2500. FIG. 10 is a flowchart showing an example of a process flow for calculating a position by combining a plurality of position calculation methods by the detailed position determination unit 2500. FIG. 11 is a flowchart showing an example of a process flow for determining an additional operation by the additional operation control unit 2600. FIG. 12 is a flowchart showing an example of a process flow for analyzing an object by the object analysis unit 2700.

[0011] The details of embodiments of the present invention are listed below. The present invention comprises the following configuration: [Item 1] An information control system comprising: a mobile body group consisting of multiple mobile bodies each equipped with a measurement sensor capable of acquiring measurement data measuring an object with at least a portion thereof above water; a mobile body operation control unit that controls the operation of the mobile bodies; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one mobile body in the mobile body group and second measurement data obtained by measuring the object from a second position by the mobile body or another mobile body; an object position determination unit that determines position information including at least one of relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; and an information output unit that displays or transmits to an external device the position information of the object determined by the object position determination unit, or that displays or transmits to an external device the analysis results of an object analysis unit that analyzes the object based on the position information. [Item 2] The information control system according to item 1, wherein the object position determination unit calculates the position information of the object based on information about an angle between the object and the second position as seen from the first position obtained from the first measurement data, an angle between the object and the first position as seen from the second position obtained from the second measurement data, and a relative distance between the first position and the second position. [Item 3] The information control system according to item 1 or 2, wherein the measurement data acquisition unit acquires third measurement data obtained by measuring the object from the third position, and the object position determination unit calculates the position information of the object based on information about the relative angle between the object and the first position as seen from the third position obtained from the third measurement data, the relative angle between the object and the third position as seen from the first position obtained from the first measurement data, and a relative distance between the first position and the third position, and compares the calculated plurality of pieces of position information of the object to determine the position information of the object.[Item 4] In the information control system described in Items 1 to 3, the measurement data acquisition unit acquires third measurement data obtained by measuring the object from the third position; the object position determination unit calculates the position information of the object based on information on the relative angle between the object and the first position as seen from the third position obtained from the third measurement data, the relative angle between the object and the third position as seen from the first position obtained from the first measurement data, and the relative distance between the first position and the third position; calculates the position information of the object based on information on the relative angle between the object and the second position as seen from the third position obtained from the third measurement data, the relative angle between the object and the third position as seen from the second position obtained from the second measurement data, and the relative distance between the second position and the third position; and determines the position information of the object by comparing the calculated multiple pieces of position information of the object. [Item 5] An information control system according to any one of items 1 to 4, wherein the measurement sensor includes a stereo camera, a monocular camera, a laser ranging sensor, a radar ranging sensor, an acoustic ranging sensor, an optical ranging sensor, or any other sensor capable of distance measurement, and the object position determination unit determines the position information of the object based on the first measurement data acquired by the measurement sensor from the first position, position coordinates of the first position, the second measurement data acquired by the measurement sensor from the second position, and position coordinates of the second position. [Item 6] An information control system according to any one of items 1 to 5, wherein the object position determination unit determines the position information of the object based on the first measurement data, position coordinates of the first position, the second measurement data, position coordinates of the second position, third measurement data obtained by measuring the object by the measurement sensor from the third position, and position coordinates of the third position.[Item 7] In the information control system described in items 1 to 6, the object position determination unit determines the position information of the object by calculating position coordinates of a geometric intersection of a line passing through the object and the first position and a line passing through the object and the second position, based on the position coordinates of the first position, the relative orientation or absolute orientation of the object seen from the first position obtained from the first measurement data, the position coordinates of the second position, and the relative orientation or absolute orientation of the object seen from the second position obtained from the second measurement data. [Item 8] In the information control system described in items 1 to 7, the measurement data acquisition unit acquires third measurement data obtained by measuring the object from the third position, and the object position determination unit determines the position information of the object by calculating position coordinates of a geometric intersection of a line passing through the object and the first position, a line passing through the object and the second position, and a line passing through the object and the third position, based on the position coordinates of the first position, the relative orientation or absolute orientation of the object as seen from the first position, the position coordinates of the second position, the relative orientation or absolute orientation of the object as seen from the second position, the position coordinates of the third position, and the relative orientation or absolute orientation of the object as seen from the third position obtained from the third measurement data. [Item 9] In the information control system described in items 1 to 8, the group of mobile objects is composed of a plurality of unmanned boats capable of navigating on water, and the first measurement data and the second measurement data are acquired by the measurement sensors provided on the unmanned boats. [Item 10] In the information control system described in items 1 to 9, the plurality of unmanned vessels include at least a first unmanned vessel and a second unmanned vessel, the first measurement data is acquired at the first position by the measurement sensor provided on the first unmanned vessel, and the second measurement data is acquired at the second position by the measurement sensor provided on the second unmanned vessel.[Item 11] An information control system according to any one of items 1 to 10, wherein the plurality of unmanned watercraft include at least a first unmanned watercraft, the first measurement data being acquired at the first position by the measurement sensor provided on the first unmanned watercraft, and the second measurement data being acquired at the second position by the measurement sensor provided on the first unmanned watercraft that has moved from the first position to the second position. [Item 12] An information control system according to any one of items 1 to 11, wherein the group of mobile objects includes an unmanned watercraft capable of navigating on water and an air vehicle capable of flying in the sky, the first measurement data being acquired by the measurement sensor provided on the unmanned watercraft, and the second measurement data being acquired by the measurement sensor provided on the air vehicle. [Item 13] An information control system according to any one of items 1 to 12, wherein the group of mobile objects includes an unmanned craft capable of navigating on the surface of water and a submersible craft capable of navigating underwater, the first measurement data being acquired by the measurement sensor provided on the unmanned craft, and the second measurement data being acquired by the measurement sensor provided on the submersible. [Item 14] An information control system according to any one of items 1 to 13, wherein the group of mobile objects includes an air vehicle capable of flying in the sky and a submersible craft capable of navigating underwater, the first measurement data being acquired by the measurement sensor provided on the air vehicle, and the second measurement data being acquired by the measurement sensor provided on the submersible. [Item 15] The information control system according to items 1 to 14, further comprising a position calculation method determination unit that determines a calculation method for the position information of the object from any of the following methods: triangulation that calculates the position of the object using the relative angle between the object seen from the first position and the second position and the relative angle between the object seen from the second position and the first position; standalone surveying that obtains the relative distance between the object and the moving body based on measurement data obtained from the measurement sensor provided on one of the moving bodies; geometric calculation that calculates the position coordinates of a geometric intersection of a line passing through the object and the first position and a line passing through the object and the second position; and a combination of these methods.[Item 16] In the information control system described in items 1 to 15, the position calculation method determination unit determines a calculation method for the position information of the object based on at least any of the measurement data acquired from the measurement sensor provided on the moving object, information on the calculation accuracy of the self-position of the moving object, information on the calculation accuracy of the relative distance between at least two of the moving objects, and information on the calculation accuracy of the relative orientation or absolute orientation of the moving objects. [Item 17] In the information control system described in items 1 to 16, when the measurement data acquired from the measurement sensor provided on the moving object is image data acquired by an optical camera, the position calculation method determination unit determines a calculation method for the position information of the object from either the triangulation or the geometric calculation when it detects that the exposure of the image data is higher than an upper tolerance value, the amount of light of the image data is lower than a lower tolerance value, or the field of view of the image data is shorter than a lower limit distance. [Item 18] In the information control system described in items 1 to 17, the mobile object operation control unit determines the first position and the second position so as to satisfy at least one of the following conditions: a relative distance from the object to the first position and a relative distance from the object to the second position are within a predetermined distance, a relative distance between the first position and the second position is within a predetermined distance, and an angle formed by the first position and the second position as seen from the object is within a predetermined angle range. [Item 19] In the information control system described in items 1 to 18, the mobile object operation control unit determines, based on the first position and the second position, a first mobile object to be moved to the first position and to be instructed to acquire the measurement data, and a second mobile object to be moved to the second position and to be instructed to acquire the measurement data.[Item 20] In the information control system described in items 1 to 19, the mobile object operation control unit commands a first mobile object and a second mobile object that are present at the first position and the second position, respectively, to acquire the measurement data when the first mobile object and the second mobile object satisfy at least one of the following conditions: a relative distance from the object to the first position and a relative distance from the object to the second position are within a predetermined distance; a relative distance between the first position and the second position is within a predetermined distance; and an angle between the first position and the second position as seen from the object is within a predetermined angle range. [Item 21] In the information control system described in items 1 to 20, the object analysis unit, when it is determined that the object is a ship, compares information about the object determined from the measurement data based on the position information of the object determined by the object position determination unit with information about the object obtained from the object by wireless communication by an external automatic identification system, and determines whether the object corresponds to a predetermined monitoring target based on the comparison result. [Item 22] In the information control system described in items 1 to 21, when the unmanned aircraft operation control unit causes the mobile body to communicate with the object by wireless communication or light emission, or to call or warn by voice or display device, the object analysis unit determines whether the object corresponds to a predetermined monitoring target, depending on the object's behavior in response to the communication, call, or warning. [Item 23] In the information control system described in items 1 to 22, the information control system further comprises a recording unit that records at least any of the position information of the object determined by the object position determination unit, the analysis processing result of the object analysis unit, the measurement data obtained by measuring the object, and the operation history of the mobile body, in association with identification information of the object.[Item 24] A control method for a system that measures an object using a group of mobile objects consisting of multiple mobile objects equipped with measurement sensors that can acquire measurement data of the object when at least a portion of the object is above water, wherein a computer executes the following steps: a mobile object operation control step that controls the operation of the mobile objects; a measurement data acquisition step that acquires first measurement data obtained by measuring the object from a first position by at least one mobile object of the group of mobile objects, and second measurement data obtained by measuring the object from a second position by the mobile object or another mobile object; an object position determination step that determines position information including at least any of relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object based on the first measurement data and the second measurement data; and an information output step that displays or transmits to the outside the position information of the object determined by the object position determination step, or displays or transmits to the outside the analysis processing results of an object analysis unit that analyzes the object based on the position information. [Item 25] A program usable in a system for measuring an object using a group of mobile objects consisting of multiple mobile objects equipped with measurement sensors capable of acquiring measurement data obtained by measuring the object when at least a portion of the object is above water, the program causing a computer to execute: a mobile object operation control command to control the operation of the mobile objects; a measurement data acquisition command to acquire first measurement data obtained by measuring the object from a first position by at least one mobile object of the group of mobile objects and second measurement data obtained by measuring the object from a second position by the mobile object or another mobile object; an object position determination command to determine position information including at least one of relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object based on the first measurement data and the second measurement data; and an information output command to display or output or transmit to the outside the position information of the object determined by the object position determination step, or to display or output or transmit to the outside the analysis processing results of an object analysis unit that analyzes the object based on the position information.

[0012] <A. First Embodiment> Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, the embodiment described below is merely an example, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc.

[0013] [A-1. Configuration] (A-1-1. System Configuration) First, the system configuration of an information control system 1 according to one embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG.

[0014] (A-1-1-1. Overview of System Configuration) FIG. 1 is an overall configuration diagram of an information control system 1 (hereinafter also referred to as "system 1") according to one embodiment of the present invention. As shown in FIG. 1, the information control system 1 includes an unmanned watercraft 1000 and a supervisory control system 2000. The supervisory control system 2000 is also configured to be able to communicate with an external cooperative system 5000 and an external system 6000 via an internet line or the like, and is capable of inputting and outputting information. The supervisory control system 2000 can send control commands to the unmanned watercraft 1000 deployed at sea via a terrestrial base station 4000 and a communication satellite 3000, and can also receive the operating status and measurement data of the unmanned watercraft 1000.

[0015] The unmanned watercraft 1000 includes a parent unit 1001 capable of communicating with a communication satellite 3000 and a child unit 1002 capable of communicating directly or indirectly with the parent unit 1001, and a communication network is established between the multiple child units 1002 and the parent unit 1001. The multiple child units 1002 and the parent unit 1001 each have the function of measuring ships, divers, drifting objects, castaways, marine life such as whales, and other objects that are at least partially exposed above water using measurement sensors (optical cameras, IR cameras, laser sensors such as LiDAR, radar sensors such as millimeter-wave sensors and microwave sensors, and sonic sensors such as sonar) mounted on the device.

[0016] The detection determination results and measurement data of the object detected by the unmanned vessel 1000, as well as various information on the operational status of the unmanned vessel 1000, are transmitted to the overall control system 2000 via the communications satellite 3000 and the terrestrial base station 4000. The overall control system 2000 determines operational commands for the unmanned vessel 1000 based on information acquired from the unmanned vessel 1000 and pre-registered information. The generated information, such as the operational commands, is transmitted to the cooperative system 5000, and the system can also obtain intervention commands from the cooperative system.

[0017] (A-1-1-2. Example of Implementation of Information Control System 1 in Real Space) Figure 2 is a diagram showing an example of an implementation image when the information control system 1 is implemented in real space. In the example shown in Figure 2, a terrestrial base station 4000 and a central control system 2000 are provided on the ground side shown in the upper right of the figure. Also provided on the ground side is a cooperative system 5000 including facilities related to external cooperative organizations such as facilities related to marine research organizations and facilities related to private law enforcement organizations (including private security organizations, private rescue organizations, etc.), and further provided is an external system 6000 such as an AIS (Automatic Identification System) control center or AIS base station that obtains information about ships navigating the ocean via wireless communication and manages this ship information from such ships.

[0018] On the other hand, on the ocean side shown on the left side of the figure, there is deployed an unmanned vessel 1000, an object 7000 for monitoring and investigation, such as a suspicious vessel or marine life, and part of a cooperative system 5000, such as a surveillance boat or research boat operated by an external cooperative organization. The unmanned vessel 1000 also has multiple platoons (platoon a, platoon b, platoon c) consisting of a master unit and multiple slave units, and each platoon can communicate directly or via a communication satellite 3000. The unmanned vessel 1000 can also communicate with a surveillance boat directly or via the communication satellite 3000. For example, the unmanned vessel 1000 can notify the surveillance boat (or research vessel) of detection information regarding the object 7000. The unmanned vessel 1000 may also be communicatively connected to an AIS base station to acquire AIS information. In addition, the unmanned vessel 1000 can communicate with manned or unmanned air vehicles 8100 and submersible vessels 8200, either directly or via a communication satellite 3000, and can, for example, measure the target object 7000 in cooperation with the unmanned vessel 1000, the air vehicle 81000, and the submersible vessel 8200.

[0019] In the example shown in Figure 2, the overall control system 2000 is implemented in a facility on land, but this is not limited to this. All or part of the functions implemented in the overall control system 2000 shown in this embodiment can also be installed on other coastal field bases on land or manned mother ships at sea, not shown, and the operation and management of the unmanned boat 1000 can be performed at the coastal field base or manned mother ship.

[0020] (A-1-2. Stakeholders Related to the Information Control System 1) Figure 3 is a diagram showing the stakeholders related to the information control system 1. As shown in Figure 3, the information control system 1 has an operator who operates the unmanned watercraft 1000 by inputting and outputting information via a user interface unit 2800 of the overall control system 2000. If all or part of the functions implemented in the overall control system 2000 shown in this embodiment are implemented in a coastal field base on land or a manned mother ship at sea (not shown), the operator can manage the operation of the unmanned watercraft 1000 at the coastal field base or the manned mother ship.

[0021] In addition, the cooperative system 5000 has a monitoring manager at the private law enforcement organization-related facilities and monitors on the monitoring boats, who work together to monitor nuisance behavior in the marine area. The marine research organization-related facilities also have a monitoring manager and researchers on the research boats, who work together to investigate marine life in the marine area. The cooperative system 5000 may also include private security companies and private rescue organizations. The external system 6000's AIS control center also has a person in charge of generating, operating, and managing AIS information.

[0022] Furthermore, the objects 7000 that are the targets of monitoring and investigation by the information control system 1 and the collaborative system 5000 include suspicious ships, marine buoys, divers, and marine life (such as whales). The information control system 1 can monitor or investigate the objects 7000 more efficiently by communicating and coordinating with the collaborative system 5000 and the external system 6000.

[0023] (A-1-3. Configuration of the unmanned watercraft 1000) Figure 4 is a configuration diagram showing a platoon 1010 made up of unmanned watercraft 1000. As shown in Figure 4, the unmanned watercraft 1000 is made up of one or more platoons 1010 (1010a, 1010b). Each platoon 1010 has at least one master unit 1001 and multiple slave units 1002. The master unit 1001 is connected to a communication satellite 3000 for communication, and has the function of aggregating information collected from the multiple slave units 1002 and transmitting it to the communication satellite 3000, as well as transmitting information related to operational commands obtained from the communication satellite 3000 and information generated by the master unit 1001 directly or indirectly to each slave unit 1002.

[0024] 4 includes a primary connection slave device 10021 that is communicatively connected to a master device 1001, a secondary connection slave device 10022 that is communicatively connected to the primary connection slave device 10021, and a tertiary connection slave device 1023 that is communicatively connected to the secondary connection slave device 10022. Each slave device (primary connection slave device 10021, secondary connection slave device 10022, tertiary connection slave device 1023) has a function of relaying information received from another master device 1001 or slave device 1002 to the other master device 1001 or slave device 1002, thereby forming a communication network between the master device 1001 and the multiple slave devices 1002.

[0025] Furthermore, at least some of the parent devices 1001 or child devices 1002 in the platoon are connected to the air vehicle 8100 or submersible vehicle 8200 via a wireless communication network, and the unmanned vessel 1000 and the air vehicle 8100 or submersible vehicle 8200 can operate in cooperation with each other to measure the target object 7000. Furthermore, the air vehicle 8100 and the submersible vehicle 8200 can transmit measurement data of the target object 7000 to the unmanned vessel 1000 via wireless communication.

[0026] 5 is a conceptual diagram showing how an unmanned vessel 1000 deployed on the sea monitors or investigates an object 7000. As shown in FIG. 5, multiple unmanned vessels (parent vessel 1001, child vessels 10021, 10022, 10023) are deployed on the sea, and the measurement sensors 1110 mounted on each unmanned vessel 1000 can measure the object 7000 present within their measurable range. Measurement data and detection judgment results of the object 7000 detected by the measurement sensors 1110 are collected in the parent vessel 1001 via a wireless communication network between the unmanned vessels 1000, transmitted from the parent vessel 1001 to a communication satellite 3000, and then transmitted to the overall control system 2000 via a terrestrial base station 4000 and the Internet. In addition, each unmanned boat 1000 is equipped with a navigation unit 1300 that can navigate the unmanned boat in any direction, and can perform detailed measurement operations on the target object 7000 based on operation commands generated by the overall control system 2000 or the parent unit 1001.

[0027] 5, an air vehicle 8100 flying in the sky is connected to at least some of the unmanned craft 1000 in the platoon via a wireless communication network. The air vehicle 8100 is equipped with measurement sensors (one or more optical cameras, IR cameras, laser sensors such as LiDAR, radar sensors such as millimeter-wave sensors and microwave sensors, and sonic sensors such as sonar), and can measure the target object 7000 from the sky in cooperation with the unmanned craft 1000. The submersible craft 8200 navigating underwater is also connected to at least some of the unmanned craft 1000 in the platoon via a wireless communication network. The submersible craft 8200 is also equipped with measurement sensors (one or more optical cameras, IR cameras, laser sensors such as LiDAR, radar sensors such as millimeter-wave sensors and microwave sensors, and sonic sensors such as sonar), and can measure the target object 7000 from underwater in cooperation with the unmanned craft 1000.

[0028] In the configuration of the present embodiment described in Figures 1 to 5, a non-terrestrial network using a communication satellite 3000 launched into a geosynchronous orbit (GEO), a medium earth orbit (MEO), a low earth orbit (LOW EOR), or another orbit is used as the communication network for transmitting and receiving information between the overall control system 2000 and the unmanned watercraft 1000. However, the present invention is not limited to this, and a non-terrestrial network using an unmanned air vehicle called a High Altitude Platform Station (HAPS) can also be used. In this case, for example, an unmanned air vehicle that circles at an altitude of approximately 8 to 50 km can be used. Furthermore, as the communication network for transmitting and receiving information between the overall control system 2000 and the unmanned watercraft 1000, a communication network directly connecting the terrestrial base station 4000 to the unmanned watercraft 1000 via wireless communication can also be used, without going through the communication satellite 3000 or HAPS. Note that the terrestrial base station 4000 is not limited to a fixed base station, and may be a mobile base station.

[0029] (A-1-4. Configuration of unmanned watercraft 1000) Next, the functions and details implemented in unmanned watercraft 1000 will be described using Figure 6. In the present invention, an unmanned watercraft is intended to mean a mobile body capable of navigating on or underwater, regardless of whether it is autonomous or remotely controlled, and refers to a mobile body including a mobile buoy equipped with a thrust generating unit.

[0030] Figure 6 is a functional block diagram showing the functional configuration of the unmanned watercraft 1000. Note that while Figure 6 illustrates the functional block diagram of the unmanned watercraft 1000, the parent unit 1001 and child unit 1002 of the unmanned watercraft 1000 can implement functions similar to those shown in Figure 6. The unmanned watercraft 1000 includes a measurement unit 1100, a host state determination unit 1200, a navigation unit 1300, a communication unit 1400, a determination unit 1500, a recording unit 1600, and an other action execution unit 1700.

[0031] The measurement unit 1100 is a functional unit that detects an object 7000 present within a measurable range around the unmanned watercraft 1000 using a measurement sensor 1110, and acquires measurement information about the object 7000. The measurement unit 1100 includes the measurement sensor 1110 and a measurement control unit 1120.

[0032] The measurement sensor 1110 may include one (monocular) or multiple electro-optical sensors that acquire image data of the sea, optical sensors such as optical cameras, infrared sensors (IR sensors), and stereo cameras, laser sensors such as LiDAR that acquire point cloud data, optical distance measuring sensors such as ToF sensors (Time of Flight sensors), and radar sensors that detect millimeter waves and microwaves. The measurement sensor 1110 measures the periphery of the unmanned watercraft 1000 to acquire measurement data of an object 7000 that exists within a measurable range on the sea. In addition, each of the above sensors can be used as a distance measuring sensor that measures the distance to an object based on the measurement data.

[0033] In addition to the above-described sensors, the measurement sensor 1110 may also include an acoustic sensor (also referred to as an acoustic measurement unit) that includes sonar, which uses sound waves such as ultrasonic waves. The acoustic sensor can be used not only underwater but also in the air above the water. When used in the air, the acoustic sensor can be used as a distance sensor that measures the distance to an object by measuring the sound waves that are generated and reflected by the object. When used underwater, the acoustic sensor may be either an active sonar that generates sound waves and measures the sound waves that resonate with underwater objects, or a passive sonar that measures the sound generated by underwater objects. The active sonar may be, for example, a side-scan sonar, a multi-beam sonar, or a single-beam sonar. The acoustic sensor may also be a USBL transceiver, an acoustic communication modem, or the like.

[0034] The measurement control unit 1120 also operates a sensor attitude changing device that can change the attitude of the measurement sensor 1110 to control at least one of the attitude angles of the measurement sensor 1110 around three axes relative to the unmanned watercraft 1000. For example, if the measurement sensor is an optical sensor, the measurement control unit 1120 can adjust the frame rate, shutter speed, etc. If the measurement sensor is a laser sensor, the measurement control unit 1120 can adjust the output of the irradiated laser. If the measurement sensor is a radar sensor, the measurement control unit 1120 can adjust the output of millimeter waves or microwaves. The measurement control unit 1120 can adjust the measurement sensitivity of the measurement sensor to a desired control amount. If the measurement sensor is an optical sensor, the measurement control unit 1120 can change the zoom amount or resolution of the optical sensor to a desired control amount.

[0035] Next, the aircraft state determination unit 1200 includes a navigation state determination unit 1210, an internal state determination unit 1220, and an external state determination unit 1230, and is a functional unit that determines the navigation state, internal state, and external state of the unmanned watercraft 1000. The navigation state determination unit 1210 determines the aircraft's position (two-dimensional or three-dimensional), movement speed, heading, movement direction, movement acceleration / deceleration, turning speed, and other state quantities related to the navigation state. The internal state determination unit 1220 determines the remaining energy of the battery and fuel installed in the aircraft, the travelable distance that can be calculated based on the remaining energy, temporary abnormalities (temperature abnormalities, communication abnormalities, etc.) of equipment installed in the aircraft, and equipment failure states. In addition, the external condition determination unit 1230 determines the communication conditions such as communication strength (dB value, etc.) and communication speed with other unmanned boats 1000 in the communicating platoon 1010, or the ocean currents and tides (flow speed, flow direction), wind speed (wind speed, wind direction), wave height, and weather (rain, snow, cloudy, etc.) around the aircraft.

[0036] The method by which the navigation state determination unit 1210 determines the position, movement speed, movement direction, and acceleration / deceleration of the aircraft itself is not particularly limited. For example, the current position, movement speed, and movement direction of the aircraft itself can be determined using a Global Navigation Satellite System (GNSS), a Global Positioning System (GPS), a Real Time Kinematic - Global Navigation Satellite System (RTK-GNSS), or the like. Here, the aircraft's position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information. Furthermore, the acceleration / deceleration can be calculated based on the amount of change over time in the determined movement speed.

[0037] The method for measuring the aircraft's heading may determine the aircraft's heading at the current time using, for example, a geomagnetic sensor, a GNSS compass, or SLAM technology using the seabed shape. The heading includes an attitude angle (orientation) in a planar view around at least the Z axis, and may preferably be attitude information around three axes: the X axis, the Y axis, and the Z axis. The turning speed may be calculated based on the amount of change over time in the determined heading information.

[0038] Next, the navigation unit 1300 is a functional unit that includes a thrust generating unit, an attitude control mechanism, and a navigation control unit, and that navigates the parent unit 1001 in any direction according to operational commands received via the communication unit 1400. The thrust generating unit is configured, for example, with a propeller, and can generate thrust by driving the propeller using the power of an engine or an electric motor. The thrust generating unit 1310 can also be configured with a sail that receives wind to generate thrust, or with a wave glider that receives wave power to generate thrust.

[0039] The attitude control mechanism is composed of a rudder mounted on the aircraft body and a propeller attitude change mechanism that can change the propeller attitude angle (mainly the yaw angle around the Z axis), and by changing these angles it is possible to control the nose direction (yaw angle) of the unmanned watercraft 1000. In addition, a center of gravity position change mechanism that changes the position of a heavy object inside the aircraft body using an actuator can also control the attitude angles of the aircraft body, namely the roll angle around the X axis and the pitch angle around the Y axis.

[0040] The navigation control unit is a functional unit that controls the output from the thrust generating unit and the attitude control mechanism to control the navigation operation of the aircraft. The navigation control unit has one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), an MPU, or a DSP), and includes a processing unit that can access a memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more processing steps.

[0041] The processing unit includes a control module configured to control the navigation status of the aircraft. For example, the control module adjusts the aircraft's position on the sea surface, movement speed, movement acceleration / deceleration, heading, turning speed, and attitude angle around three axes. That is, the navigation control unit 1330 controls the navigation operation of the aircraft by causing the aircraft to perform various operations such as forward movement, reverse movement, acceleration, deceleration, and turning.

[0042] Next, the communication unit 1400 is equipped with an unmanned craft-to-unmanned craft communication unit 1410, a satellite communication unit 1420, and an external communication unit 1430, and is a functional unit that communicates with other unmanned crafts 1000 in the platoon 1010, the communication satellite 3000, external flying bodies 8100, submersible craft 8200, patrol craft, and AIS base stations. The unmanned craft-to-unmanned craft communication unit 1410 is equipped with a communication antenna for unmanned craft-to-unmanned craft communication, and communicates with other unmanned crafts 1000 in the platoon 1010. The satellite communication unit 1420 is equipped with a satellite communication antenna, and communicates with the communication satellite 3000. The external communication unit 1430 is equipped with an AIS antenna and a VHF antenna, and communicates with external patrol craft and AIS base stations.

[0043] Next, the determination unit 1500 is a functional unit that makes a determination regarding the target object 7000. The determination unit 1500 includes an object detection determination unit 1510. The object detection determination unit 1510 interprets the measurement data acquired by the measurement sensor 1110 and determines the presence or absence of an object, the size of the object, and the like.

[0044] The object detection determination unit 1510 determines whether or not to transmit the measurement data to the integrated control system 2000 and perform object analysis based on the interpretation information of the measurement data. For example, even if the object detection determination unit 1510 detects an object on the water, if the estimated size of the object is small and it is highly likely that it is driftwood or the like that is not a monitored object, it can determine that it is not necessary to transmit the measurement data to the integrated control system 2000. On the other hand, if the estimated size of the object detected by the object detection determination unit 1510 is larger than a predetermined standard and it is highly likely that it is a ship or the like that is a monitored object, it can determine that it is necessary to transmit the measurement data to the integrated control system 2000.

[0045] Next, the recording unit 1600 includes a measurement data recording unit 1610, a host device state recording unit 1620, and a determination information recording unit 1630. The measurement data recording unit 1610 records measurement data measured by the measurement unit 1100. The host device state recording unit 1620 records various state information related to the host device determined by the host device state determination unit 1200. Furthermore, the determination information recording unit 1630 records various determination information determined by the determination unit 1500.

[0046] The other action execution unit 1700 is a functional unit that communicates with a detected object 7000 such as a ship by communication or light emission, and issues calls or warnings by voice. The other action execution unit 1700 includes a communication unit 1710 and a voice output unit 1720.

[0047] The communication unit 1710 can communicate with the ship, which is the target 7000, by wireless communication via the communication unit 1400. The communication unit 1710 can also communicate by transmitting Morse code signals by illuminating a light emitting unit provided on the unmanned boat 1000. The audio output unit 1720 is configured, for example, with a speaker, and can issue voice calls or warnings to the target 7000.

[0048] (A-1-5. Configuration of Overall Control System 2000) Next, the functions and contents of the overall control system 2000 will be described with reference to Fig. 7. Fig. 7 is a functional block diagram showing the functional configuration of the overall control system 2000. As shown in Fig. 7, the overall control system 2000 includes an information import unit 2100, a detection determination unit 2200, a position calculation method determination unit 2300, a detailed measurement operation control unit 2400, a detailed position determination unit 2500, an additional operation control unit 2600, an object analysis unit 2700, a user interface unit 2800, a recording unit 2910, and an information communication unit 2920.

[0049] (A-1-5-1. Information import unit 2100) The information import unit 2100 is a functional unit that imports information to be processed or used in each functional unit within the overall control system 2000 from the unmanned watercraft 1000, the cooperative system 5000, or the external system 6000. The information import unit 2100 includes a judgment condition acquisition unit 2110, a measurement data acquisition unit 2120, an external information acquisition unit 2130, and an external intervention information acquisition unit 2140.

[0050] The judgment condition acquisition unit 2110 is a functional unit that acquires in advance the criteria for each judgment and decision made by the judgment unit 1500 of the unmanned watercraft 1000, the detection judgment unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position judgment unit 2500, and the additional operation control unit 2600 of the overall control system 2000. The judgment condition acquisition unit 2110 can acquire, for example, a reference value such as the size of a detected object as a judgment criterion for the object detection judgment unit 1510 of the judgment unit 1500 to determine whether measurement data needs to be transmitted to the overall control system 2000 or whether target analysis is needed. The judgment condition acquisition unit 2110 can also acquire judgment criteria for each judgment item made by the detection judgment unit 2200 shown in FIG. 8.

[0051] The measurement data acquisition unit 2120 is a functional unit that acquires, via a communication satellite 3000, a HAPS, a terrestrial base station 4000, etc., the determination results determined by the determination units 1500 of the multiple unmanned vessels 1000, and measurement data measured by the multiple unmanned vessels 1000, the air vehicle 8100, and the submarine 8200. The measurement data acquired by the measurement data acquisition unit 2120 includes first measurement data acquired by a monitoring measurement operation by the unmanned vessel 1000, detailed measurement data acquired based on a detailed measurement operation commanded to the unmanned vessel 1000 by a detailed measurement operation control unit 2400 (described later), and additional measurement data acquired based on an additional measurement operation commanded to the unmanned vessel 1000 by an additional operation control unit 2600.

[0052] The external information acquisition unit 2130 is a functional unit that acquires navigation information of ships in the ocean area where the unmanned watercraft 1000 is deployed or the surrounding area from the AIS control center of the external system 6000. The navigation information of ships may also be acquired from another VHF data exchange system included in the external system 6000. The external information acquisition unit 2130 may also acquire weather information for the ocean area where the unmanned watercraft 1000 is deployed or the surrounding area from the external system 6000, such as the Japan Meteorological Agency or a private weather information providing system.

[0053] The external intervention information acquisition unit 2140 is a functional unit that receives intervention command information from the collaboration system 5000. For example, the external intervention information acquisition unit 2140 can receive, from the collaboration system 5000, intervention command information regarding candidate information for measurement operation commands and information on the necessity of additional measurement, which has been sent to the collaboration system 5000 via the information communication unit 2920 (described later).

[0054] (A-1-5-2. Detection and Determination Unit 2200) The detection and determination unit 2200 is a functional unit that analyzes the object based on the information on the determination criteria acquired by the determination condition acquisition unit 2110 and the measurement data (particularly the primary measurement data) of the object acquired by the measurement data acquisition unit 2120, and determines information and the state of the object.

[0055] The object detection determination by the detection determination unit 2200 will be described below with reference to FIG. 8. FIG. 8 is a diagram showing the determination items by the detection determination unit 2200. As shown in FIG. 8, the detection determination unit 2200 can include a monitoring target determination of whether or not an object is a monitoring target. In addition, the object characteristic determination items of the object can include the type, shape, size, and orientation of the object. In addition, the static state determination items can include the relative distance, orientation, and position coordinates of the object. In addition, the dynamic state determination items can include the moving state / stationary state, moving direction, moving speed, past moving route history, and future predicted route.

[0056] An example of a method for determining a static state will be described. First, the relative distance can be obtained based on analysis processing of image data obtained by one or more optical sensors. Furthermore, the relative distance can be obtained based on measurement data obtained by a laser sensor such as a LiDAR or a ToF (Time of Flight) sensor, a radar sensor that detects millimeter waves or microwaves, or a sonic sensor including a sonar that uses sound waves such as ultrasound.

[0057] The orientation of an object as seen from the unmanned watercraft 1000 can be determined from orientation information of the object detected from measurement data acquired by an optical sensor, laser sensor, radar sensor, or the like. The orientation information of the object may be calculated as an absolute orientation in a global coordinate system, or as a relative orientation in a local coordinate system based on, for example, the position and heading orientation of the unmanned watercraft 1000. Furthermore, the position coordinates of the object can be calculated based on the self-position coordinate information detected by the unmanned watercraft 1000's own-ship state determination unit 1200 and the above-mentioned relative orientation and relative distance information. The position coordinates may be two-dimensional coordinates on a horizontal XY plane, but are preferably three-dimensional coordinates in an XYZ space that also includes information on the height direction. The position coordinates may be local position coordinates in a local coordinate system or global position coordinates in a global coordinate system (absolute coordinate system).

[0058] Next, an example of a method for determining a dynamic state will be described. Based on a change analysis of measurement data obtained by performing measurements multiple times in a time series, it is possible to determine whether the state is moving or stationary, and to determine the direction and speed of movement. Furthermore, it is possible to determine the history of past movement routes based on past measurement data. Furthermore, it is possible to determine a predicted future route based on the determination results of the past movement history, the current direction of movement, orientation, etc.

[0059] Here, as described above, the detection and determination unit 2200 can determine the relative distance, relative orientation, and position coordinates as static state determination items for the object, but the accuracy of these position-related determination information is not sufficiently high because it is information determined based on primary measurement data acquired by a single unmanned watercraft 1000. Therefore, in order to obtain highly accurate position information required for detailed analysis of the object or for requesting an external system to rush to the site, more detailed position information is determined by each functional unit described below.

[0060] Here, the type determination of detected objects by the detection determination unit 2200 can determine objects that are at least partially above water, such as suspicious ships, divers, castaways, drifting ships, flotsam, marine buoys, and marine life (whales, dolphins, schools of fish, etc.). In this way, by detecting suspicious ships and divers, it is possible to monitor nuisances at sea. By detecting castaways and drifting ships, it is possible to rescue castaways. By detecting flotsam (debris, wood, etc.), it is possible to obtain obstacle information for safe ship navigation. By detecting marine buoys, it is possible to support systems that recover and communicate with marine buoys. By monitoring marine life (whales, dolphins, schools of fish, etc.), it is possible to support marine surveys.

[0061] (A-1-5-3. Position calculation method determination unit 2300) Next, we will explain the position calculation method determination unit 2300. The position calculation method determination unit 2300 is a functional unit that determines a calculation method for calculating position information including at least one of the relative position coordinates, relative distance, relative orientation, and absolute position coordinates of a detected object from multiple candidate position calculation methods.

[0062] For example, the candidate position calculation methods selected by the position calculation method determination unit 2300 may include, for example, position calculation using point surveying, position calculation using triangulation, and position calculation using geometric calculation. Each position calculation method will be described below with reference to FIGS.

[0063] FIG. 9 is a conceptual diagram illustrating a position calculation method using independent surveying. The example shown in FIG. 9 illustrates a case in which position information for an object 7000 is calculated using three unmanned boats (1000a, 1000b, and 1000c). When independent surveying is used as shown in this figure, unmanned boat 1000a calculates the relative distance from unmanned boat 1000a to the object 7000 by analyzing and processing measurement data obtained by its own measurement sensor, and obtains detected orientation information for the object 7000 as seen from unmanned boat 1000a. Furthermore, unmanned boat 1000a obtains its own position information (its own position coordinate information) using its own boat status determination unit 1200, and calculates the position coordinates of the object 7000 based on this acquired information. Furthermore, unmanned boat 1000b performs a similar process to calculate the position coordinates of the object 7000. By comparing the calculated position coordinates of the object 7000, more accurate position coordinates can be determined.

[0064] 9, the same processing is performed on the unmanned boat 1000c to calculate the position coordinates of the object 7000, and by comparing the position coordinates of the object 7000 calculated based on the measurement data acquired by the three unmanned boats (1000a, 1000b, 1000c), it is possible to more accurately determine the position coordinates of the object 7000. Details of the position calculation method when using the single survey shown in FIG. 9 will be described later.

[0065] FIG. 10 is a conceptual diagram illustrating a position calculation method using triangulation. The example shown in FIG. 10 illustrates a case in which position information of an object 7000 is calculated using three unmanned boats (1000a, 1000b, and 1000c). When using triangulation as shown in this figure, unmanned boats 1000a and 1000b calculate their relative distances from each other's position coordinates or the signal reception strength (dB) of their wireless communications. Furthermore, unmanned boat 1000a acquires the relative angle between unmanned boat 1000b and object 7000 as seen from its own boat, and unmanned boat 1000b acquires the relative angle between unmanned boat 1000a and object 7000 as seen from its own boat. Here, the position coordinates of the object 7000 are calculated using triangulation based on the position coordinate information of at least one of unmanned boats 1000a and 1000b and the acquired information described above. In this way, by performing triangulation based on measurement data measured from the positions of multiple unmanned vessels 1000, it is possible to calculate the position information of the target object 7000 more accurately than with a position calculation method using a single unmanned vessel 1000.

[0066] Furthermore, in the example shown in Figure 10, by using another unmanned boat 1000c in addition to the two unmanned boats 1000a and 1000b described above, it is possible to obtain, for example, the relative distance between unmanned boat 1000a and unmanned boat 1000c, the relative angle between unmanned boat 1000c and object 7000 as seen from unmanned boat 1000a, and the relative angle between unmanned boat 1000a and object 7000 as seen from unmanned boat 1000c, and then calculate the position coordinates of object 7000 by triangulation based on the position coordinate information of at least one of unmanned boat 1000a and unmanned boat 1000c and the above-mentioned acquired information. Furthermore, by comparing the position coordinates of the object 7000 calculated using unmanned boat 1000a and unmanned boat 1000b with the position coordinates of the object 7000 calculated using unmanned boat 1000a and unmanned boat 1000c, the position information of the object 7000 can be calculated more accurately than when two unmanned boats are used.

[0067] In addition to the above method, it is also possible to calculate the position coordinates of the object 7000 using unmanned boats 1000b and 1000c, and by adding this position information to the comparison process, it is possible to calculate the position information of the object 7000 more accurately. The position calculation method when using triangulation shown in Figure 10 will be described in detail later.

[0068] FIG. 11 is a conceptual diagram illustrating a position calculation method using geometric calculations. The example shown in FIG. 11 illustrates a case in which position information of an object 7000 is calculated using three unmanned boats (1000a, 1000b, and 1000c). When using geometric calculations as shown in this figure, first, a geometric formula for a line connecting unmanned boat 1000a and object 7000 is calculated from the position coordinate information of unmanned boat 1000a and the detected orientation information of object 7000. Similarly, for unmanned boat 1000b, a geometric formula for a line connecting unmanned boat 1000b and object 7000 is calculated from the position coordinate information of unmanned boat 1000b and the detected orientation information of object 7000. Furthermore, by calculating the position coordinates of the intersection of these two lines, the position information of object 7000 can be calculated. In this way, by performing calculations using geometric formulas based on measurement data measured from the positions of multiple unmanned vessels 1000, it is possible to calculate the position information of the target object 7000 more accurately than with a position calculation method using a single unmanned vessel 1000.

[0069] 11 , by using another unmanned boat 1000c in addition to the two unmanned boats 1000a and 1000b described above, for example, a geometric formula for a line connecting the unmanned boat 1000c and the object 7000 can be calculated from the position coordinate information of the unmanned boat 1000c and the detected orientation information of the object 7000, and the intersection of this line formula with the line connecting the unmanned boat 1000a and the object 7000 or the line connecting the unmanned boat 1000b and the object 7000 can be calculated. In this way, by performing calculations using geometric formulas based on measurement data measured from the positions of the three unmanned boats 1000, it is possible to calculate the position information of the object 7000 more accurately than with a position calculation method using two unmanned boats 1000. Details of the position calculation method using the three geometric formulas shown in FIG. 11 will be described later.

[0070] As described above, the candidate position calculation methods selected by the position calculation method determination unit 2300 may include, for example, position calculation using point surveying as shown in Fig. 9, position calculation using triangulation as shown in Fig. 10, and position calculation using geometric calculation as shown in Fig. 11, but are not limited to these and may also include other position calculation methods. Furthermore, the candidate position calculation methods selected by the position calculation method determination unit 2300 may also include a method that combines the above-mentioned position calculation methods.

[0071] (A-1-5-4. Detailed measurement operation control unit 2400) The detailed measurement operation control unit 2400 is a functional unit that controls the operation of the unmanned boat 1000, the flying body 8100, and the submarine 8200 to measure the measurement data and acquire other information required to execute the position calculation method selected by the position calculation method determination unit 2300. The detailed measurement operation control unit 2400 includes a measurement vehicle placement determination unit 2410, a measurement vehicle selection unit 2420, an vehicle operation plan determination unit 2430, and a detailed measurement operation command unit 2440.

[0072] The measurement vehicle arrangement determination unit 2410 has a function of determining measurement positions suitable for executing the position calculation method selected by the position calculation method determination unit 2300. For example, it determines multiple measurement positions, such as the measurement positions of the unmanned boats (1000a, 1000b, 1000c) shown in Figures 9, 10, and 11, based on criteria that measurement positions must satisfy, including that the distance from the object is within a reference distance, that the relative distance between multiple measurement positions is equal to or greater than a reference distance, or that the angle formed by multiple measurement positions as seen from the object is equal to or greater than a reference angle. Note that information on the above criteria can be acquired by the determination condition acquisition unit 2110.

[0073] The measurement vehicle selection unit 2420 is a functional unit that selects a vehicle to perform measurement operations at multiple measurement positions based on information on multiple measurement positions determined by the measurement vehicle placement determination unit 2410 and current position information of each unmanned vehicle 1000, air vehicle 8100, and submersible vehicle 8200. For example, it is possible to select an unmanned vehicle 1000 that is located close to the multiple measurement positions determined by the measurement vehicle placement determination unit 2410, or an unmanned vehicle that has already been deployed at the multiple determined measurement positions, as the vehicle to perform measurement operations.

[0074] The vehicle motion plan determination unit 2430 is a functional unit that determines a motion plan for the unmanned vessel 1000 selected by the measurement vessel selection unit 2420 to travel to multiple measurement positions determined by the measurement vessel placement determination unit 2410 and perform measurements. For example, the motion plan may be to travel along the shortest route to travel in the shortest time, or to travel along a route that requires less energy by utilizing tides and wind. It may also calculate a predicted time required to complete travel to the measurement position. Furthermore, when performing measurements at a measurement position, the motion plan may be to synchronize the measurement timing with other unmanned vessels 1000, or to perform measurements at the top dead center of vertical movement due to waves.

[0075] The detailed measurement operation command unit 2440 is a functional unit that outputs an operation command for a detailed measurement operation to the unmanned boat 1000 selected by the measurement vehicle selection unit 2420 to move to multiple measurement positions determined by the measurement vehicle placement determination unit 2410 in accordance with the movement plan determined by the vehicle operation plan determination unit 2430 and acquire measurement data of the target object.

[0076] (A-1-5-5. Detailed position determination unit 2500) Next, we will explain the function of the detailed position determination unit 2500. The detailed position determination unit 2500 is a functional unit that calculates the position information of an object using the position calculation method determined by the position calculation method determination unit 2300 based on measurement data measured by the unmanned boat 1000 or the like, and determines the suitability of the calculated position information. The detailed position determination unit 2500 includes an object position calculation unit 2510 and a calculated position suitability determination unit 2520.

[0077] The object position calculation unit 2510 is a functional unit that calculates the position information of the object using the position calculation method determined by the position calculation method determination unit 2300, based on detailed measurement data measured by at least two or more unmanned watercraft 1000 in accordance with a detailed measurement operation command from the detailed measurement operation control unit 2400, or additional measurement data measured by at least two or more unmanned watercraft 1000 in accordance with an additional measurement operation command from the additional operation control unit 2600. The object position calculation unit 2510 can calculate position information including at least one of the relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object as the position information of the object.

[0078] When calculating the position information of an object based on measurement data measured by at least two or more unmanned watercraft 1000, it is necessary to determine whether the same object is included in the multiple measurement data. Therefore, the object position calculation unit 2510 determines the identity of the object whose position information is to be calculated based on the characteristics of the object, such as its shape and size, determined by the detection determination unit 2200. The position calculation method used by the object position calculation unit 2510 will be described in detail later.

[0079] The calculated position suitability determination unit 2520 is a functional unit that determines whether the position information of the object calculated by the object position calculation unit 2510 is appropriate. For example, if the calculated position suitability determination unit 2520 determines that the calculated position information of the object is inappropriate, it determines that the position information is indefinite. The calculated position suitability determination unit 2520 can also determine that the accuracy of the calculated position information of the object is insufficient. A detailed processing method by the calculated position suitability determination unit 2520 will be described later.

[0080] (A-1-5-6. Additional operation control unit 2600) Next, we will explain the additional operation control unit 2600. The additional operation control unit 2600 is a functional unit that controls additional operations by the unmanned boat 1000, the air vehicle 8100, and the submarine 8200, depending on the determination results of the detailed position determination unit 2500 and the detection determination unit 2200. The additional operation control unit 2600 includes an additional operation determination unit 2610 and an additional operation command unit 2620.

[0081] The additional operation determination unit 2610 is a functional unit that determines additional operations to be performed by the unmanned boat 1000, the air vehicle 8100, or the submersible 8200. For example, if the calculated position suitability determination unit 2520 determines that the calculated position information of the target object is uncertain, the additional operation is to perform an additional detailed measurement operation under the same conditions or from a different angle. Furthermore, if the accuracy of the calculated position information of the target object is determined to be insufficient, the additional operation is to acquire a closer image (measurement data) from a closer position or to acquire an image with higher resolution.

[0082] As another example, if the detection determination unit 2200 determines that the object is a target for monitoring, or if the type of object is determined to be a ship, the additional action determination unit 2610 can determine that the additional action to be taken is for the unmanned boat 1000, the air vehicle 8100, or the submarine 8200 to communicate with the object by radio communication or light emission (including Morse code, etc.), or to make a call by voice (audio output from a speaker) or by a display device, or to issue a warning (to stop illegal activity, to leave the current sea area, etc.).

[0083] In this way, as an additional operation, when communicating with an object by wireless communication or light emission, or calling out or warning by voice or display device, the communication, call, or warning may be made by an operator of the integrated control system 2000 via the user interface unit 2800 described later, or wireless communication with the collaborative system 5000 may be performed via the information communication unit 2920 described later, and the communication, call, or warning may be made directly by a user of the collaborative system 5000, such as a private enforcement organization.

[0084] The additional action command unit 2620 is a functional unit that commands the additional action determined by the additional action determination unit 2610 to the unmanned boat 1000, the flying vehicle 8100, and the submarine 8200.

[0085] (A-1-5-7. Object analysis unit 2700) Next, we will explain the object analysis unit 2700. The object analysis unit 2700 is a functional unit that analyzes the object based on measurement data measured by at least two or more unmanned watercraft 1000 in accordance with a detailed measurement operation command from the detailed measurement operation control unit 2400, position information of the object determined by the detailed position determination unit 2500, or additional measurement data obtained by an additional operation.

[0086] The object analysis unit 2700 can make a determination on the determination items determined by the detection determination unit 2200 shown in Fig. 8, for example. The object analysis unit 2700 makes a determination on these determination items based on measurement data measured by at least two or more unmanned boats 1000 in accordance with a detailed measurement operation command from the detailed measurement operation control unit 2400, position information of the object determined by the detailed position determination unit 2500, and additional measurement data obtained by an additional operation. Therefore, the object analysis unit 2700 can make these determinations with higher accuracy than the determination by the detection determination unit 2200.

[0087] As an example, when determining whether a detected object corresponds to a suspicious ship previously registered as a monitoring target, the object analysis unit 2700 can make a definitive determination of whether the object corresponds to a suspicious ship based on close-up image data from a nearby position or high-resolution image data acquired as additional measurement data. Alternatively, the object analysis unit 2700 can make a definitive determination of whether the object corresponds to a suspicious ship based on image data measured from an angle different from the previous measurement data as additional measurement data. In this way, when making a definitive determination of whether the object corresponds to a suspicious ship, for example, by comparing the name, shape characteristics, color, ship registration number, etc. of the previously registered suspicious ship with information that can be read from the acquired detailed image data, a definitive determination of whether the object corresponds to a suspicious ship can be made.

[0088] As another example, when the object analysis unit 2700 determines that the object is a ship, it compares the information about the object determined from the measurement data based on the position information of the object determined by the detailed position determination unit 2500 with the ship information about the object obtained from the object by the AIS control center (Automatic Identification System) of the external system 6000 via wireless communication, and based on the comparison result, it can make a definitive determination as to whether the object corresponds to a predetermined monitoring target.

[0089] As another example, when the additional operation control unit 2600 causes the unmanned vessel 1000, the air vehicle 8100, or the submarine 8200 to execute an additional operation of communicating with an object by radio or light emission, or calling or warning with audio or a display device (to stop illegal activity, leave the current sea area, etc.), the object analysis unit 2700 can determine whether the object corresponds to a predetermined monitoring target based on the object's behavior in response to the communication, call, or warning. For example, if a ship or the like attempts to flee in response to the communication, call, or warning, or if there is no response to the call, etc., the ship can be definitively determined to be a suspicious ship.

[0090] (A-1-5-8. User Interface Unit 2800) Next, the user interface unit 2800 will be described. The user interface unit 2800 is a functional unit that notifies or displays to the user the determination results and control states of the detection and determination unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position determination unit 2500, the additional operation control unit 2600, and the object analysis unit 2700, and accepts user input information from the user regarding commands for measurement operations of the unmanned watercraft 1000, etc. The user interface unit 2800 includes a display unit 2810 and a user input acceptance unit 2820. The user interface unit 2800 may be a portable mobile terminal such as a smartphone, tablet terminal, or laptop PC.

[0091] The display unit 2810 is a functional unit that notifies or displays to the user the measurement data acquired by the measurement data acquisition unit 2120, the results of each judgment or decision made by the detection determination unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position determination unit 2500, the additional operation control unit 2600, the object analysis unit 2700, etc., candidate operation commands for the unmanned watercraft 1000, etc., the operation history (including the movement path, etc.) of the unmanned watercraft 1000, etc. When notifying the user, such as when the latest measurement data has been acquired or an object has been detected, the display unit 2810 can notify the user not only by display output but also by sound, light emission, or vibration.

[0092] For example, the display unit 2810 can display and output the measurement data acquired by the measurement data acquisition unit 2120 and the detection judgment results (judgment results for each judgment item shown in FIG. 8 ) judged by the detection judgment unit 2200. It can also display and output the position calculation method determined by the position calculation method determination unit 2300. It can also display and output operation commands or candidates for detailed measurement operations, including measurement aircraft placement, measurement aircraft selection results, and aircraft movement plans, determined by the detailed measurement operation control unit 2400. It can also display and output detailed position information of the object judged by the detailed position judgment unit 2500 and the judgment results for the appropriateness of the calculated position information. It can also display and output operation commands or candidates for additional operations determined by the additional operation control unit 2600. It can also display and output the detection judgment results (judgment results for each judgment item shown in FIG. 8 ) judged by the object analysis unit 2700.

[0093] In addition, when information about an object is sent to the collaborative system 5000 or other external systems using the information communication unit 2920 described below, information such as the destination's contact details, contact method, location, etc. may be displayed on the display unit 2810.

[0094] The user input accepting unit 2820 is a functional unit that accepts user input for each piece of information displayed by the display unit 2810. User input information can also be accepted via operation buttons provided on the display screen of the display unit 2810. For example, the user input accepting unit 2820 can accept a user input for approving or changing the position calculation method determined by the position calculation method determining unit 2300, or can accept an input for specifying an arbitrary position calculation method.

[0095] The user input accepting unit 2820 can also accept user input for approval or change of the operation command or candidate for the detailed measurement operation, including the measurement aircraft arrangement, measurement aircraft selection result, and aircraft operation plan, determined by the detailed measurement operation control unit 2400, or can accept designation input of any measurement aircraft arrangement, measurement aircraft selection result, or aircraft movement plan. The user input accepting unit 2820 can also accept user input for approval or change of the operation command or candidate for the additional operation, determined by the additional operation control unit 2600, or can accept designation input of any additional operation, including a command to obtain more detailed position information.

[0096] In addition, when the additional operation determination unit 2610 determines that the additional operation is to communicate with an object from the unmanned boat 1000, the flying vehicle 8100, or the submarine 8200 by wireless communication or light emission, or to call or warn by voice or display device, the user input acceptance unit 2820 can accept input from the user in the form of voice, text, or commands for each of the communication, call, and warning operations.

[0097] Furthermore, when displaying measurement data and each determination result on the display unit 2810, the user input accepting unit 2820 can accept from the user a display mode that includes displaying the latest measurement data and determination results, or displaying measurement data and determination results at a specified time in the past that are recorded in a recording unit (described later). Furthermore, when the unmanned watercraft 1000 acquires the latest measurement data, the display unit 2810 or the like may notify the user that the latest measurement data has been updated. Furthermore, when a request to display past measurement data is accepted, an estimated time required for displaying the data may be displayed.

[0098] (A-1-5-9. Recording unit 2910) The recording unit 2910 is a functional unit that records the measurement data acquired by the measurement data acquisition unit 2120, the results of each judgment and decision made by the detection determination unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position determination unit 2500, the additional operation control unit 2600, the object analysis unit 2700, etc., candidate operation commands for the unmanned watercraft 1000, etc., and the operation history of the unmanned watercraft 1000, etc. (including measurement operations, communications, calls, warnings, etc.). In addition, this information can be recorded in association with identification information of the detected object.

[0099] In addition to the above-mentioned information, the recording unit 2910 may also include historical information on the data transmitted and received between the unmanned boat 1000 and the overall control system 2000, user input information regarding data transmission and reception, oceanographic data (wave height, current, weather, etc.), and information regarding loss of target position capture.

[0100] (A-1-5-10. Information communication unit 2920) The information communication unit 2920 is a functional unit that outputs information similar to the information displayed on the display unit 1810, that is, the measurement data acquired by the measurement data acquisition unit 2120, the results of each judgment and decision by the detection judgment unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position judgment unit 2500, the additional operation control unit 2600, the object analysis unit 2700, etc., candidate operation commands for the unmanned vessel 1000, etc., the operation history of the unmanned vessel 1000, etc., etc., to the cooperative system 5000, the external system 6000, or other external systems.

[0101] The information communication unit 2920 can also acquire information similar to the information accepted by the user input accepting unit 2820 from the collaborative system 5000, the external system 6000, or other external systems. That is, the information communication unit 2920 can receive user input information for approval or change of an operation command or candidate for a detailed measurement operation, including a measurement aircraft placement, a measurement aircraft selection result, and an aircraft movement plan, determined by the detailed measurement operation control unit 2400, or can receive a designation input for any measurement aircraft placement, a measurement aircraft selection result, or an aircraft movement plan. The information communication unit 2920 can also receive user input information for approval or change of an operation command or candidate for an additional operation, determined by the additional operation control unit 2600, or can receive a designation input for any additional operation.

[0102] In addition, when the additional operation determination unit 2610 determines that the additional operation is to communicate with an object from the unmanned boat 1000, the air vehicle 8100, or the submarine 8200 by wireless communication or light emission, or to call or warn by voice or display device, the information communication unit 2920 can receive input in the form of voice, text, or commands for each of the communication, call, and warning operations from the collaborative system 5000, the external system 6000, or other external systems.

[0103] The functions implemented in the unmanned watercraft 1000 and the overall control system 2000 described above using Figures 6 and 7 are merely one embodiment, and the present invention is not limited to this implementation example. In other words, some of the functions implemented in the unmanned watercraft 1000 shown in Figure 6 (mainly the functions of the determination unit 1500) can be implemented in the overall control system 2000. On the other hand, some of the functions implemented in the overall control system 2000 shown in Figure 7 (mainly at least one of the information import unit 2100, detection determination unit 2200, position calculation method determination unit 2300, detailed measurement operation control unit 2400, detailed position determination unit 2500, additional operation control unit 2600, and object analysis unit 2700) can also be implemented in the unmanned watercraft 1000. In addition, in this embodiment, an example has been described in which the function of initially detecting an object based on measurement data is implemented in the judgment unit 1500 on the unmanned boat 1000 side and the detection judgment unit 2200 on the overall control system 2000 side, but it is also possible to implement all of this initial detection judgment function on the unmanned boat side, and conversely, it is also possible to implement all of the initial detection judgment function on the overall control system 2000 side.

[0104] (A-1-6. Hardware Configuration) Fig. 12 is a hardware configuration diagram of an overall control system 2000. Here, the overall control system 2000 in the present invention is an information processing device such as a server device or a PC. As shown in the figure, the overall control system 2000 has an input device 100, an output device 200, a processing device 300, a main memory device 400, an auxiliary memory device 500, a communication device 600, and a bus 700 that electrically connects these devices.

[0105] The input device 100 can constitute the user input receiving unit 2820 of the user interface unit 2800, and is a device that allows a user to input information and instructions to the integrated control system 2000. Specifically, the input device 100 is, for example, a touch panel, a keyboard, a mouse, or an audio input device such as a microphone.

[0106] The output device 200 is a device that outputs various types of information generated by the integrated control system 2000, and can constitute a display unit 2810 of the user interface unit 2800. Specifically, the output device 200 can constitute the display unit 2810 using a display device for eyewear, AR, or VR, or can also be a printer or a speaker.

[0107] The processing device 300 is, for example, a device that performs arithmetic processing. Specifically, the processing device 300 is, for example, a CPU, a microprocessor, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or other semiconductor devices capable of performing calculations.

[0108] The main storage device 400 is a memory device such as a RAM that temporarily stores various read information and a ROM that stores programs, application programs, and other various information executed by the processing device 300. The auxiliary storage device 500 is a non-volatile storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory that can store digital information.

[0109] The communication device 600 is a device that performs wireless or wired information communication with the outside, and can constitute the information communication unit 2920 described above.

[0110] (A-1-7. Control Flow of Information Control System 1) Next, we will explain the overall control flow of the information control system 1. FIG.

[0111] First, the information import unit 2100 acquires various information from the external system 6000 or the like (step 101). In this step, for example, navigation information of ships in the ocean area where the unmanned watercraft 1000 is deployed or in the surrounding area can be acquired, and information on the judgment criteria used in the judgment process by the judgment unit 1500 of the unmanned watercraft 1000 and the detection and judgment unit 2200 of the overall control system 2000 can be acquired from the user input acceptance unit 2820 or the cooperative system 5000.

[0112] Next, the unmanned watercraft 1000 or the like performs monitoring and measurement of the area to be monitored (step 102).

[0113] Next, the determination unit 1500 and the detection determination unit 2200 of the unmanned watercraft 1000 perform a primary detection determination of the object (step 103). In this step, for example, object detection from the measurement data and determination of each determination item as shown in FIG.

[0114] Next, the processing step to transition to is determined depending on whether or not a monitoring target has been detected by the determination unit 1500 or the detection determination unit 2200 (step 104). If a monitoring target candidate is detected in this step, the processing transitions to step 105, whereas if a monitoring target candidate is not detected, the processing returns to step 103.

[0115] Next, the position calculation method is determined by the position calculation method determination unit 2300 (step 105). In this step, a calculation method for calculating position information including at least one of the relative position coordinates, relative distance, relative direction, and absolute position coordinates of the detected object is determined from a plurality of candidate position calculation methods.

[0116] Next, the detailed measurement operation control unit 2400 determines the location of the unmanned craft 1000 or the like that will perform detailed measurement (i.e., the measurement position) and selects the craft (step 106).

[0117] Next, detailed measurement is carried out by the unmanned watercraft 1000 or the like based on a detailed measurement operation command from the detailed measurement operation control unit 2400 (step 107).

[0118] Next, the detailed position determination unit 2500 calculates detailed position information of the object (step 108).

[0119] Next, the additional operation control unit 2600 determines the operation of additional measurement (step 109).

[0120] Next, based on an additional operation command from the additional operation control unit 2600, an additional measurement operation or the like is executed (step 110).

[0121] Next, the object analysis unit 2700 executes an analysis process of the object (step 111).

[0122] Next, information such as the determination results in each functional unit is recorded in the recording unit 2910, displayed on the display unit 2810, or transmitted from the information communication unit 2920 to an external device such as the cooperative system 5000 (step 112).

[0123] (A-1-8. Control Sequence in Information Control System 1) Next, a description will be given of a control sequence between the systems in the information control system 1. FIG.

[0124] First, when the determination unit 1500 of one of the mobile bodies such as the unmanned boat 1000 (for example, the slave unit 1002a) determines that the measurement data should be analyzed by the overall control system 2000, the primary measurement data measured by the slave unit 1002a is transmitted to the master unit 1001. Furthermore, the master unit 1001 transmits the received primary measurement data to the overall control system 2000.

[0125] Next, the detection determination unit 2200 of the overall control system 2000 determines whether an object has been detected, and the position calculation method determination unit 2300 determines the method for calculating the position of the object. Furthermore, in accordance with these determination results, the detailed measurement operation control unit 2400 generates an operation command for detailed measurement.

[0126] Next, the integrated control system 2000 transmits the candidate operation command for detailed measurement to the collaborative system 5000, and receives approval or modification of the candidate operation command or input of a designation of an arbitrary operation command from the collaborative system 5000.

[0127] Next, the overall control system 2000 determines a detailed measurement command based on the designated input information received from the cooperative system 5000 , and transmits this detailed measurement command to the parent device 1001 .

[0128] Next, a detailed measurement command is transmitted from the parent device 1001 to the child devices 1002a and 1002b, and the child devices 1002a and 1002b execute detailed measurement. The child devices 1002a and 1002b transmit the acquired detailed measurement data to the parent device 1001. Furthermore, the parent device 1001 transmits the received detailed measurement data to the integrated control system 2000.

[0129] Next, detailed position information of the object is calculated by the detailed position determination unit 2500 of the overall control system 2000, and an operation command for an additional operation is generated based on the calculated position information by the additional operation control unit 2600.

[0130] Next, the integrated control system 2000 transmits candidate operation commands for additional operations to the collaborative system 5000, and receives approval or modification of the candidate operation commands or input of a designation of an arbitrary operation command from the collaborative system 5000.

[0131] Next, the overall control system 2000 determines an additional operation command based on the designated input information received from the cooperative system 5000 , and transmits this additional operation command to the parent device 1001 .

[0132] Next, an additional operation command is transmitted from the parent device 1001 to the child devices 1002a and 1002b, and the child devices 1002a and 1002b execute the additional operation. The child devices 1002a and 1002b transmit the additional measurement data acquired by the additional operation to the parent device 1001. Furthermore, the parent device 1001 transmits the received additional measurement data to the overall control system 2000.

[0133] (A-1-9. Position Calculation Method Determination Processing) Next, the position calculation method determination processing will be described. Fig. 15 is a flowchart showing an example of the position calculation method determination processing flow by the position calculation method determination unit 2300. In particular, Fig. 15 shows the detailed processing flow of step 105 in the flowchart shown in Fig. 13.

[0134] First, the position calculation method determination unit 2300 acquires various information (image data and other distance measurement data, relative distance to other unmanned vessels, positioning accuracy information of the vessel's own position coordinates, etc.) from the unmanned vessel 1000 that acquired the primary measurement data (step 201).

[0135] Next, the process step to transition to is determined depending on whether the image accuracy of the image data and other distance measurement data acquired from the unmanned watercraft 1000 is lower than a predetermined standard (step 202). If the image accuracy is lower than the predetermined standard in this step, the process transitions to step 203. On the other hand, if the image accuracy is higher than the predetermined standard, the process transitions to step 206.

[0136] For example, if the measurement data obtained from a measurement sensor installed on an unmanned boat 1000 or the like is image data acquired by an optical camera, in step 202, if it is detected that the exposure of the image data is higher than the upper limit tolerance, the amount of light in the image data is lower than the lower limit tolerance, or the field of view of the image data is shorter than the lower limit distance, it is determined that the image accuracy is poor.

[0137] Next, if it is determined in step 202 that the image accuracy is better than the predetermined standard, the position calculation method is determined to be single-point measurement (step 206). Here, the single-point measurement position calculation method is a method of obtaining the relative distance between the target object and the unmanned watercraft 1000 based on measurement data obtained from a measurement sensor provided on a single unmanned watercraft 1000, for example, as shown in Figure 9.

[0138] Next, if it is determined in step 202 that the image accuracy is worse than the predetermined standard, the processing step to transition to is determined depending on whether the calculation accuracy of the relative azimuth angle or absolute azimuth angle of the object as seen from the unmanned watercraft 1000 is determined to be worse than the predetermined standard or indefinite (step 203). If it is determined in this step that the calculation accuracy of the relative azimuth angle or absolute azimuth angle is worse than the predetermined standard, the processing transitions to step 209; on the other hand, if it is determined that the calculation accuracy of the relative azimuth angle or absolute azimuth angle is better than the predetermined standard, the processing transitions to step 204.

[0139] Next, if it is determined in step 203 that the calculation accuracy of the relative azimuth angle or absolute azimuth angle is better than the predetermined standard, the processing step to transition to is determined depending on whether the calculation accuracy of the relative distance to another unmanned watercraft 1000 is determined to be worse than the predetermined standard or is indefinite (step 204). In this step, if it is determined that the calculation accuracy of the relative distance is worse than the predetermined standard, the processing transitions to step 205, and on the other hand, if it is determined that the calculation accuracy of the relative distance is better than the predetermined standard, the processing transitions to step 207.

[0140] Next, if it is determined in step 204 that the calculation accuracy of the relative distance to other unmanned watercraft 1000 is better than a predetermined standard, triangulation is selected as the position calculation method (step 207). The triangulation position calculation method is a method of calculating the position information of an object based on information such as the azimuth angle of the object obtained from multiple unmanned watercraft 1000 whose relative distances to each other are known, as shown in Fig. 10, for example.

[0141] Next, if it is determined in step 204 that the calculation accuracy of the relative distance to another unmanned vessel 1000 is worse than a predetermined standard, the processing step to transition to is determined depending on whether the accuracy of the self-positioning of the unmanned vessel 1000 using GNSS or the like is worse than the predetermined standard (step 205). If it is determined in this step that the self-positioning accuracy is worse than the predetermined standard, the processing transitions to step 209, and if it is determined that the self-positioning accuracy is better than the predetermined standard, the processing transitions to step 208.

[0142] Next, if it is determined in step 205 that the self-positioning accuracy is better than the predetermined standard, a position calculation method using geometric calculation is determined as the position calculation method (step 208). Here, a position calculation method using geometric calculation is a method of calculating the position information of an object by using a geometric linear equation connecting the object and its own position using a plurality of unmanned watercraft 1000 capable of acquiring the self-position coordinates, as shown in Figure 11, for example.

[0143] Next, if it is determined in step 203 that the calculation accuracy of the relative azimuth angle or absolute azimuth angle is worse than a predetermined standard, or if it is determined in step 205 that the self-positioning accuracy is worse than a predetermined standard, a user command input regarding the position calculation method is accepted from the user input accepting unit 2820 or the cooperative system 5000 (step 209).

[0144] As shown in the flowchart of this figure, the position calculation method determination unit 2300 can determine a method for calculating the position information of the unmanned vessel 1000, etc. based on at least one of the following information: measurement data obtained from measurement sensors installed on the unmanned vessel 1000, etc.; information regarding the calculation accuracy of the unmanned vessel 1000's own position; information regarding the calculation accuracy of the relative distance between at least two of multiple unmanned vessels 1000, etc.; and information regarding the calculation accuracy of the relative orientation or absolute orientation of the unmanned vessel 1000, etc.

[0145] Furthermore, in step 202, if it is detected that the exposure of the image data is higher than the upper limit tolerance, the amount of light in the image data is lower than the lower limit tolerance, or the field of view of the image data is shorter than the lower limit distance, and if it is determined that the image accuracy is poor, it is possible to transition to a process of selecting the position calculation method from either triangulation or geometric calculation.

[0146] In the example shown in this figure, an example has been described in which the position calculation method is selected automatically or by user specification from among position calculation using stand-alone surveying, position calculation using triangulation, and position calculation by geometric calculation, but this embodiment is not limited to this, and a method that combines position calculation using stand-alone surveying, position calculation using triangulation, and position calculation by geometric calculation can also be selected as the position calculation method.

[0147] (A-1-10. Determining the detailed measurement position and aircraft) Next, the process of determining the detailed measurement position and aircraft will be described with reference to Figures 16 and 17. Also, the combination of aircraft for performing detailed measurement will be described with reference to Figures 18 to 20.

[0148] (A-1-10-1. Processing flow for determining detailed measurement position and aircraft) Fig. 16 is a flowchart showing an example of the processing flow for determining detailed measurement by the detailed measurement operation control unit 2400. In particular, Fig. 16 shows the detailed processing flow of step 106 in the flowchart shown in Fig. 13.

[0149] In the flowchart shown in FIG. 16, first, the measurement body arrangement determination unit 2410 narrows down a plurality of measurement positions to areas whose relative distance from the target object is within a reference distance (step 301).

[0150] Next, the measurement body arrangement determination unit 2410 narrows down the measurement positions to areas where the relative distance between the measurement positions is equal to or greater than a reference distance (step 302).

[0151] Next, the measurement body arrangement determination unit 2410 narrows down the measurement positions to an area where the angle formed by the measurement positions as seen from the target object falls within a reference angle range (step 303).

[0152] Next, the measurement body arrangement determination unit 2410 determines a plurality of measurement positions from the narrowed down area (step 304).

[0153] Next, the measurement vehicle selection unit 2420 selects an unmanned vessel 1000 or the like to perform measurement operations at the determined measurement positions (step 305). In this step, for example, the measurement vehicle selection unit 2420 can select the unmanned vessel 1000 or the like that is closest to the determined measurement positions as the vehicle to perform measurement operations, based on the current positions of the multiple unmanned vessels 1000, air vehicles 8100, and submersible vehicles 8200.

[0154] (A-1-10-1. Processing flow for determining detailed measurement position and aircraft) Figure 17 is a conceptual diagram showing the detailed measurement determination processing by the detailed measurement operation control unit 2400. Figure 17 shows the position 9000 of the object 7000, the position of the unmanned boat 1000a that acquired the primary measurement data including the object 7000, and multiple candidate positions (9001 to 9004) for other measurement positions.

[0155] First, in step 301 of FIG. 16, when narrowing down multiple measurement positions to areas within a reference distance (500 m in the example shown in this figure) from the object 7000, position 9001 is excluded from the candidates for measurement positions.

[0156] 16, the multiple measurement positions are narrowed down to areas where the relative distance between the multiple measurement positions (the relative distance from position 9000 shown in the figure to each of the measurement position candidates (9002 to 9004)) is equal to or greater than a reference distance (for example, 300 m or greater). At this time, position 9002 is excluded from the measurement position candidates.

[0157] 16, the multiple measurement positions are narrowed down to an area where the angle formed by the multiple measurement positions as seen from the object 7000 (angles α1 and α2 shown in this figure) is within a reference angle range. At this time, angle α2 is determined to be smaller than the reference angle, and position 9003 is excluded from the candidates for the measurement position.

[0158] 16, a plurality of measurement positions are determined from the narrowed-down area, so that positions 9000 and 9004 of the unmanned boat 1000a are determined as measurement positions.

[0159] 16, there are several possible cases in which the measurement vehicle selection unit 2420 selects the unmanned vessel 1000, etc. to perform measurement operations at the determined measurement positions. For example, if unmanned vessel 1000a is present at the position of unmanned vessel 1000a and another unmanned vessel 1000b is present at position 9004, unmanned vessel 1000a and unmanned vessel 1000b can be selected as the vessels to perform detailed measurement operations.

[0160] As another example, if there is no unmanned boat 1000a at position 9000 and no other unmanned boat 1000b at position 9004, a first aircraft to move to position 9000 and perform measurement operations, and a second aircraft to move to position 9004 and perform measurement operations can be selected from multiple unmanned boats, flying vehicles 8100, and submersible boats 8200.

[0161] When selecting the aircraft to move to positions 9000 and 9004 and perform measurement operations, the first aircraft that moves to position 9000 and performs measurement operations and the second aircraft that moves to position 9004 and performs measurement operations do not necessarily have to be different aircraft; the same aircraft can perform measurement at position 9000 and then move to position 9004 and perform measurement again. In this case, since the measurement data at each of positions 9000 and 9004 is acquired to calculate the position of the object 7000, as shown in Figures 9 to 11, it is necessary to quickly move the aircraft between positions 9000 and 9004 before the position of the object 7000 moves. Therefore, for example, when moving an unmanned watercraft 1000 having hydrofoils between positions 9000 and 9004, it is desirable to move it at high speed in hydrofoil lift mode.

[0162] (A-1-10-3. Combination of Aircraft Performing Detailed Measurement) Next, the combination of aircraft performing detailed measurement will be described using FIGS. 18 to 20 . FIG. 18 is a diagram showing a first example of an aircraft combination selected by the measurement aircraft selection unit 2420. In the example shown in FIG. 18 , in particular, unmanned boats (1000a, 1000b) and an air vehicle 8100 are selected as aircraft performing detailed measurement. As shown in this figure, the unmanned boats (1000a, 1000b) and the air vehicle 8100 each measure the target object 7000 using their measurement sensors and acquire measurement data. In addition, the air vehicle 8100 is connected to at least one unmanned boat via wireless communication, and can receive flight control commands and transmit acquired measurement data to the overall control system 2000. The air vehicle 8100 can be a manned or unmanned aircraft, and any type of aircraft can be used, such as a multicopter or fixed-wing aircraft.

[0163] FIG. 19 shows a second example of a combination of vehicles selected by the measurement vehicle selection unit 2420. In the example shown in FIG. 19, the unmanned vehicles (1000a, 1000b) and the submersible vehicle 8200 are selected as the vehicles to perform detailed measurements. As shown in this figure, the unmanned vehicles (1000a, 1000b) and the submersible vehicle 8200 each use their measurement sensors to measure the target object 7000 and acquire measurement data. The submersible vehicle 8200 is also connected to at least one unmanned vehicle 1000b via wireless communication, allowing it to receive flight control commands and transmit acquired measurement data to the overall control system 2000. The submersible vehicle 8200 can be a manned or unmanned submersible vehicle. The measurement sensor of the submersible vehicle 8200 can be, for example, a sonar sensor, which can acquire the relative distance to the target object 7000 as measurement data.

[0164] FIG. 20 shows a third example of a combination of aircraft selected by the measurement aircraft selection unit 2420. In the example shown in FIG. 20, the aircraft 8100 and the submersible 8200 are selected as the aircraft to perform detailed measurements. As shown in this figure, the aircraft 8100 and the submersible 8200 each measure the target object 7000 using their measurement sensors to acquire measurement data. Furthermore, the aircraft 8100 and the submersible 8200 are each connected to at least one unmanned vessel (1001, 1000b) via wireless communication, allowing them to receive flight control commands and navigation control commands and transmit acquired measurement data to the overall control system 2000. The submersible 8200 can be a manned or unmanned submersible. Furthermore, by applying a sensor such as sonar to the measurement sensor of the submersible 8200, the relative distance to the target object 7000 can be acquired as measurement data.

[0165] In the example shown in Figure 20, the unmanned boat 1000 acts as a communication relay between the aircraft 8100 and the submarine 8200, but by the unmanned boat 1000 also acquiring measurement data of the target object 7000, the accuracy of calculating the position information of the target object 7000 can be improved using measurement data acquired from three or more locations.

[0166] (A-1-11. Detailed Position Calculation Processing) Next, detailed position calculation processing by the detailed position determination unit 2500 will be described with reference to FIGS.

[0167] (A-1-11-1. Detailed position calculation processing by independent surveying) Fig. 21 is a flowchart showing an example of a position calculation processing flow using independent surveying by the detailed position determination unit 2500. In particular, it shows an example of calculating the position information of an object based on measurement data acquired by a stereo camera, monocular camera, laser distance measurement sensor, radar distance measurement sensor, sonic distance measurement sensor, optical distance measurement sensor, or other measurement sensor capable of measuring distance to the object.

[0168] First, the relative distance between the unmanned watercraft 1000a and the target object is calculated using measurement data from the unmanned watercraft 1000a (step 401). In this step, for example, if a stereo camera is used as the measurement sensor, the relative distance to the target object can be calculated from the acquired stereo image data, or by performing SfM (Structure from Motion) processing on the image data. As another example, if a monocular camera is used as the measurement sensor, the relative distance to the target object can be calculated using a monocular camera distance measurement method that utilizes AI, etc.

[0169] Next, azimuth information (relative azimuth angle or absolute azimuth angle) of the object as seen from the unmanned watercraft 1000a is obtained based on the measurement data (step 402).

[0170] Next, the unmanned vessel 1000a calculates the position coordinates of the target object from its own position coordinate information, which it has obtained by estimating its own position using GNSS or the like, the relative distance between the target object and the unmanned vessel 1000a obtained in step 401, and the orientation information of the target object as seen from the unmanned vessel 1000a obtained in step 402 (step 403).

[0171] Next, the processing of steps 401 to 403 is executed also in the other unmanned watercraft 1000b, thereby calculating the position coordinates of the target object in the same manner (step 404).

[0172] Next, the calculated position coordinates are compared (step 405).

[0173] Next, the process step to transition to is determined depending on whether the difference between the position coordinates is greater than a predetermined value (step 406). If it is determined in this step that the difference between the position coordinates is greater than the predetermined value, the process transitions to step 408. On the other hand, if it is determined that the difference between the position coordinates is smaller than the predetermined value, the process transitions to step 407.

[0174] Next, if it is determined in step 406 that the difference between the position coordinates is smaller than a predetermined value, the calculated position suitability determination unit 2520 determines the determination result of the position coordinates (step 407). In this step, any one of the position coordinates to be compared may be selected and determined as the calculation result, or the midpoint or intermediate position coordinate of the multiple position coordinates may be determined as the calculation result.

[0175] Next, if it is determined in step 406 that the difference between the position coordinates is greater than a predetermined value, the calculated position suitability determining unit 2520 determines the result of the position coordinate determination as indefinite (step 408).

[0176] In other words, the flowchart shown in this figure makes it possible to determine the position information of an object based on multiple measurement data (first measurement data, second measurement data) measured from multiple positions (first position, second position) and the position coordinates of the multiple positions (first position, second position).

[0177] Although this figure shows an example in which the position information of an object is determined from measurement data acquired from two positions, the position information of an object can also be determined based on three pieces of measurement data acquired from three positions and the coordinates of the three positions, as shown in Figure 9. In this case, the position information of the object can be determined with higher accuracy than when measurement data is acquired from two positions.

[0178] (A-1-11-2. Detailed Position Calculation Processing by Triangulation) FIG. 22 is a flowchart showing an example of the position calculation processing flow using triangulation by the detailed position determination unit 2500.

[0179] First, information on the relative distance between the unmanned watercraft 1000a and the unmanned watercraft 1000b is acquired (step 501).

[0180] Next, the angle formed between the object 7000 as seen from the unmanned boat 1000a and the unmanned boat 1000b, and the angle formed between the object 7000 as seen from the unmanned boat 1000b and the unmanned boat 1000a are obtained (step 502).

[0181] Next, the relative distance between unmanned boat 1000a and unmanned boat 1000b is calculated by triangulation using the relative distance between unmanned boat 1000a and unmanned boat 1000b, the angle between unmanned boat 1000b and object 7000 as seen from unmanned boat 1000a, and the angle between unmanned boat 1000a and unmanned boat 1000a as seen from unmanned boat 1000b (step 503).

[0182] Next, the unmanned boat 1000a calculates the position coordinates of the object 7000 from its own position coordinate information, which it has obtained by estimating its own position using GNSS or the like, and the relative distance between the object 7000 and the unmanned boat 1000a (step 504).

[0183] Next, the processing of steps 501 to 504 is also executed between the unmanned watercraft 1000a and the other unmanned watercraft 1000c, and the position coordinates of the target object 7000 are calculated (step 505).

[0184] Next, the calculated position coordinates are compared (step 506).

[0185] Next, the process step to transition to is determined depending on whether the difference between the position coordinates is greater than a predetermined value (step 507). If it is determined in this step that the difference between the position coordinates is greater than the predetermined value, the process transitions to step 509. On the other hand, if it is determined that the difference between the position coordinates is smaller than the predetermined value, the process transitions to step 508.

[0186] Next, if it is determined in step 507 that the difference between the position coordinates is smaller than a predetermined value, the calculated position suitability determination unit 2520 determines the determination result of the position coordinates (step 508). In this step, any one of the position coordinates to be compared may be selected and determined as the calculation result, or the midpoint or intermediate position coordinate of the multiple position coordinates may be determined as the calculation result.

[0187] Next, if it is determined in step 507 that the difference between the position coordinates is greater than a predetermined value, the calculated position suitability determining unit 2520 determines the result of the position coordinate determination as indefinite (step 509).

[0188] In the example shown in this figure, the position information of object 7000 calculated based on measurement data of object 7000 from unmanned boats at three locations (1000a, 1000b, 1000c) is compared using the triangle formed by unmanned boat 1000a, unmanned boat 1000b, and object 7000, and the triangle formed by unmanned boat 1000a, unmanned boat 1000c, and object 7000.However, by further calculating the position information of object 7000 using the triangle formed by unmanned boat 1000b, unmanned boat 1000c, and object 7000, and comparing the three calculated position information, it is possible to calculate the position information of object 7000 with even greater accuracy.

[0189] (A-1-11-3. Detailed Position Calculation Processing Using Geometric Calculation) FIG. 23 is a flowchart showing an example of the position calculation processing flow using geometric calculation by the detailed position determination unit 2500.

[0190] First, the unmanned vessel 1000a calculates a geometric straight line connecting the unmanned vessel 1000a and the target object 7000 from its own position coordinate information, which it has obtained by estimating its own position using GNSS or the like, and the azimuth information (relative azimuth angle or absolute azimuth angle) of the target object as seen from the unmanned vessel 1000a based on measurement data (step 601).

[0191] Next, the unmanned vessel 1000b calculates a geometric straight line connecting the unmanned vessel 1000b and the target object 7000 based on its own position coordinate information, which it has determined through self-position estimation using GNSS or the like, and the azimuth information (relative azimuth angle or absolute azimuth angle) of the target object as seen from the unmanned vessel 1000b based on the measurement data (step 602).

[0192] Next, the position coordinates of the intersection of the two lines calculated in steps 601 and 602 are calculated as the position coordinates of the object 7000 (step 603).

[0193] Next, the unmanned vessel 1000c calculates a geometric straight line connecting the unmanned vessel 1000c and the target object 7000 based on its own position coordinate information, which it has determined by estimating its own position using GNSS or the like, and the azimuth information (relative azimuth angle or absolute azimuth angle) of the target object as seen from the unmanned vessel 1000c based on the measurement data (step 604).

[0194] Next, the position coordinates of the intersection of the two lines calculated in steps 601 and 604 are calculated as the position coordinates of the object 7000 (step 605).

[0195] Next, the position coordinates calculated in steps 603 and 605 are compared (step 606).

[0196] Next, the process step to transition to is determined depending on whether the difference between the position coordinates is greater than a predetermined value (step 607). If it is determined in this step that the difference between the position coordinates is greater than the predetermined value, the process transitions to step 609. On the other hand, if it is determined that the difference between the position coordinates is smaller than the predetermined value, the process transitions to step 608.

[0197] Next, if it is determined in step 607 that the difference between the position coordinates is smaller than a predetermined value, the calculated position suitability determination unit 2520 determines the determination result of the position coordinates (step 608). In this step, any one of the position coordinates to be compared may be selected and determined as the calculation result, or the midpoint or intermediate position coordinate of the multiple position coordinates may be determined as the calculation result.

[0198] Next, if it is determined in step 607 that the difference between the position coordinates is greater than a predetermined value, the calculated position suitability determining unit 2520 determines the result of the position coordinate determination as indefinite (step 609).

[0199] (A-1-11-4. Detailed Position Calculation Processing Combining Multiple Position Calculation Methods) FIG. 24 is a flowchart showing an example of the position calculation processing flow in which multiple position calculation methods are combined by the detailed position determination unit 2500.

[0200] First, the position coordinates of the object 7000 are calculated using a single measurement as shown in FIG. 21 (step 701).

[0201] Next, the position coordinates of the object 7000 are calculated using triangulation as shown in FIG. 22 (step 702).

[0202] Next, the position coordinates of the object 7000 are calculated using a geometric linear equation as shown in FIG. 23 (step 703).

[0203] Next, the position coordinates calculated in steps 701 to 703 are compared (step 704).

[0204] Next, the process step to transition to is determined depending on whether the difference between the position coordinates is greater than a predetermined value (step 705). If it is determined that the difference between the position coordinates is greater than the predetermined value, the process transitions to step 707. On the other hand, if it is determined that the difference between the position coordinates is smaller than the predetermined value, the process transitions to step 706.

[0205] Next, if it is determined in step 705 that the difference between the position coordinates is smaller than a predetermined value, the calculated position suitability determination unit 2520 determines the determination result of the position coordinates (step 706). In this step, any one of the position coordinates to be compared may be selected and determined as the calculation result, or the midpoint or intermediate position coordinate of the multiple position coordinates may be determined as the calculation result.

[0206] Next, if it is determined in step 705 that the difference between the position coordinates is greater than a predetermined value, the calculated position suitability determining unit 2520 determines the result of the position coordinate determination as indefinite (step 707).

[0207] In the example shown in this figure, the position coordinates of the object are calculated using three methods: independent surveying, triangulation, and geometric linear equations, but a combination of any two of these three methods may also be used.

[0208] (A-1-12. Additional Action Determination Processing) Next, the additional action determination processing will be described. Fig. 25 is a flowchart showing an example of the additional action determination processing flow by the additional action control unit 2600. In particular, Fig. 25 shows the detailed processing flow of step 109 in the flowchart shown in Fig. 13.

[0209] First, the detailed position information of the object 7000 calculated by the detailed position determining unit 2500 is acquired (step 801).

[0210] Next, the process step to transition to is determined depending on whether the calculated position appropriateness determination unit 2520 has determined that the calculated result of the position information is indefinite (step 802). If it is determined that the calculated result of the position information is indefinite in this step, the process transitions to step 805, whereas if it is determined that the calculated result of the position information is not indefinite, the process transitions to step 803.

[0211] Next, if it is determined in step 802 that the calculation result of the position information is not indefinite, the process determines the processing step to transition to depending on whether or not the calculation accuracy of the calculated position information is insufficient (step 803). If it is determined in this step that the calculation accuracy of the position information is insufficient, the process transitions to step 805, whereas if it is determined that the calculation accuracy of the position information is not insufficient, the process transitions to step 804.

[0212] Next, if it is determined in step 803 that the calculation accuracy of the location information is sufficient, the process determines the processing step to transition to depending on whether or not there is a request for detailed location information from the user (step 804). In this step, the request for detailed location information is acquired from the user interface unit 2800 or the external collaboration system 5000. In this step, if there is a request for detailed location information, the process transitions to step 806, and on the other hand, if there is no request for detailed location information, the process transitions to step 807.

[0213] Next, if it is determined in step 802 that the calculation result of the position information is indefinite, or if it is determined in step 803 that the calculation accuracy of the position information is insufficient, it is decided to re-execute the detailed measurement as an additional operation (step 805). In the re-execution of the detailed measurement in this step, measurement data may be acquired from the same measurement position and under the same conditions as in the previous detailed measurement operation, or measurement data may be acquired by changing the measurement position or measurement conditions.

[0214] Next, if there is a request to obtain detailed location information in step 804, the additional operation is determined to be to perform a measurement from a position closer to the object than the previous measurement position and obtain proximity measurement data (including proximity image data, etc.), or to obtain measurement data (including image data) with a higher resolution than the previous measurement data (step 806).

[0215] Next, if there is no request for detailed position information in step 804, the process step to transition to is determined depending on whether there is a request from the user for communication with the object 7000 or the like (step 807). In this step, the request for communication or the like is acquired from the user interface unit 2800 or the external cooperative system 5000 or the like. The request for communication or the like is a request signal from the unmanned watercraft 1000 or the like to the object via wireless communication or light emission, or to issue a call or warning by voice or display device. If there is a request for communication or the like in this step, the process transitions to step 809; on the other hand, if there is no request for communication or the like, the process transitions to step 808.

[0216] Next, if there is no request for communication or the like in step 807, it is determined that no additional action is required (step 808).

[0217] Next, if there is a request for communication, etc. in step 809, depending on the content of the request for communication, etc., an additional action is determined to be communication by radio communication or light emission from the unmanned boat 1000, etc. to the target object, or an action to call or warn by voice or display device (step 809).

[0218] (A-1-13. Object Analysis Processing) Next, an explanation will be given of the object analysis processing by the object analysis unit 2700. Fig. 26 is a flowchart showing an example of the object analysis processing flow by the object analysis unit 2700. Fig. 26 shows a detailed processing flow of step 111 in the flowchart shown in Fig. 13, and in particular shows the processing flow when the object 7000 is a ship.

[0219] First, the position information of the object 7000 (ship) calculated by the detailed position determining unit 2500 is acquired (step 901).

[0220] Next, the measurement data acquired from the unmanned watercraft 1000 etc. is analyzed (step 902). In this step, for example, an analysis is performed for each determination item as shown in FIG.

[0221] Next, based on the position information of the object 7000 (ship) obtained in step 901, the correspondence between the object 7000 (ship) detected by the measurement data and the AIS information of the ship obtained from the AIS control center of the external system 6000 is determined, and the determination results from the analysis process in step 902 are compared with the contents of the AIS information (step 903).

[0222] Next, the analysis result of the object 7000 (ship) obtained in step 902 is compared with pre-recorded database information on the characteristics of the suspicious ship, etc. (step 904). Here, the database information on the characteristics of the suspicious ship, etc. may include the sound signature, hull number, ship silhouette image data, etc. of the suspicious ship.

[0223] Next, the behavior of the object 7000 (ship) in response to additional actions such as communication, calling, and warnings performed by the unmanned watercraft 1000 or the like in accordance with the additional action command generated by the additional action control unit 2600 is analyzed (step 905). Examples of behavior of the object 7000 (ship) in response to additional actions such as communication, calling, and warnings include fleeing, not responding, or mooring and responding.

[0224] Next, a definitive determination is made for each determination item of the object 7000 (ship) according to the results of each process in steps 901 to 905 (step 906). In this step, for example, a definitive determination can be made for multiple determination items shown in Fig. 8, and a definitive determination is made as to whether or not the object corresponds to a suspicious ship that is a target of monitoring, the type of object, etc.

[0225] In this step 906, for example, if there are any mismatches among the judgment items shown in Figure 8 based on the results of comparing the judgment content obtained by the analysis processing of the measurement data in step 903 with the content of the AIS information, the object 7000 can be judged to be a suspicious ship.

[0226] Furthermore, in step 906, for example, based on the results of comparing the analysis results of the object 7000 (ship) in the measurement data obtained in step 904 with pre-recorded database information regarding the characteristics of suspicious ships, etc., if the analysis results of the object 7000 (ship) (the judgment results of the judgment items shown in Figure 8) match the characteristics of suspicious ships, etc. in the database information, the object 7000 can be judged to be a suspicious ship.

[0227] In addition, in step 906, if, for example, fleeing behavior or no reaction is detected from the object 7000 (ship) in response to additional actions such as communication, calling, or warning in step 905, the object 7000 can be determined to be a suspicious ship.

[0228] The above-described embodiments are merely examples for facilitating understanding of the present invention, and are not intended to limit the present invention. The present invention can be modified and improved without departing from the spirit thereof, and it goes without saying that the present invention includes equivalents thereof.

[0229] [A-2. Effects of this embodiment] The above-described embodiment can provide a system or control method for more accurately determining the position of an object or more accurately analyzing the object when monitoring or investigating an object on water using an unmanned aerial vehicle. As an example, by measuring the object from different positions using a mobile body, the position of the object can be calculated more accurately, and the object can be analyzed more accurately.

[0230] 1...Control system (system) 100...Input device 200...Output device 300...Processing device 400...Main memory device 500...Auxiliary memory device 600...Communication device 700...Bus 1000...Unmanned boat 1001...Master unit 1002...Slave unit 10021...Primary connected slave unit 10022...Secondary connected slave unit 10023...Tertiary connected slave unit 1010...Platoon 1100...Measurement unit 1110...Measurement sensor 1120...Measurement control unit 1200...Own aircraft state determination unit 1210...Navigation state determination unit 1220...Internal state determination unit 1230...External state determination unit 1300...Navigation unit 1400...Communication unit 1410...Unmanned boat-to-unmanned boat communication unit 1420...Satellite communication unit 1430...External communication unit 1500...Determination unit DESCRIPTION OF SYMBOLS 1510...Object detection determination unit 1600...Recording unit 1610...Measurement data recording unit 1620...Own aircraft status recording unit 1630...Determination information recording unit 1700...Other action execution unit 1710...Communication unit 1720...Audio output unit 2000...Overall control system 2100...Information import unit 2110...Determination condition acquisition unit 2120...Measurement data acquisition unit 2130...External information acquisition unit 2140...External intervention information acquisition unit 2200...Detection determination unit 2300...Position calculation method determination unit 2400...Detailed measurement operation control unit 2410...Measurement aircraft placement determination unit 2420...Measurement aircraft selection unit 2430...Aircraft movement plan determination unit 2440...Detailed measurement operation command unit 2500...Detailed position determination unit 2510...Object position calculation unit 2520...Calculated position suitability determination unit 2600: Additional operation control unit 2610: Additional operation determination unit 2620: Additional operation command unit 2700: Object analysis unit 2800: User interface unit 2810: Display unit 2820: User input reception unit 2910: Recording unit 2920: Information communication unit 3000: Communication satellite 4000: Terrestrial base station 5000: Cooperative system 6000: External system7000...Object 8100...Aircraft 8200...Submarine

Claims

1. An information control system comprising: a mobile body group consisting of multiple mobile bodies equipped with measurement sensors capable of acquiring measurement data measuring an object with at least a portion thereof above water; a mobile body operation control unit that controls the operation of the mobile bodies; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one mobile body of the mobile body group and second measurement data obtained by measuring the object from a second position by the mobile body or another mobile body; an object position determination unit that determines position information including at least one of relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; and an information output unit that displays or transmits to an external device the position information of the object determined by the object position determination unit, or displays or transmits to an external device the analysis processing results of an object analysis unit that analyzes the object based on the position information.

2. An information control system according to claim 1, wherein the object position determination unit calculates the position information of the object based on the angle between the object and the second position as seen from the first position obtained from the first measurement data, the angle between the object and the first position as seen from the second position obtained from the second measurement data, and information on the relative distance between the first position and the second position.

3. An information control system according to claim 2, wherein the measurement data acquisition unit acquires third measurement data obtained by measuring the object from a third position, and the object position determination unit calculates the position information of the object based on the relative angle between the object and the first position as seen from the third position obtained from the third measurement data, the relative angle between the object and the third position as seen from the first position obtained from the first measurement data, and information on the relative distance between the first position and the third position, and compares the calculated multiple pieces of position information of the object to determine the position information of the object.

4. An information control system according to claim 2, wherein the measurement data acquisition unit acquires third measurement data obtained by measuring the object from a third position; the object position determination unit calculates the position information of the object based on information on the relative angle between the object and the first position as seen from the third position obtained from the third measurement data, the relative angle between the object and the third position as seen from the first position obtained from the first measurement data, and the relative distance between the first position and the third position; calculates the position information of the object based on information on the relative angle between the object and the second position as seen from the third position obtained from the third measurement data, the relative angle between the object and the third position as seen from the second position obtained from the second measurement data, and the relative distance between the second position and the third position; and determines the position information of the object by comparing the calculated multiple pieces of position information of the object.

5. An information control system according to claim 1, wherein the measurement sensor includes a stereo camera, a monocular camera, a laser ranging sensor, a radar ranging sensor, an acoustic ranging sensor, an optical ranging sensor, or any other sensor capable of measuring distance, and the object position determination unit determines the position information of the object based on the first measurement data acquired by the measurement sensor from the first position, the position coordinates of the first position, the second measurement data acquired by the measurement sensor from the second position, and the position coordinates of the second position.

6. An information control system according to claim 5, wherein the object position determination unit determines the position information of the object based on the first measurement data, the position coordinates of the first position, the second measurement data, the position coordinates of the second position, third measurement data obtained by measuring the object by the measurement sensor from a third position, and the position coordinates of the third position.

7. An information control system according to claim 1, wherein the object position determination unit determines the position information of the object by calculating the position coordinates of the geometric intersection of a line passing through the object and the first position and a line passing through the object and the second position, based on the position coordinates of the first position, the relative or absolute orientation of the object as seen from the first position obtained from the first measurement data, the position coordinates of the second position, and the relative or absolute orientation of the object as seen from the second position obtained from the second measurement data.

8. An information control system according to claim 7, wherein the measurement data acquisition unit acquires third measurement data obtained by measuring the object from a third position, and the object position determination unit determines the position information of the object by calculating position coordinates of the geometric intersection of a line passing through the object and the first position, a line passing through the object and the second position, and a line passing through the object and the third position, based on the position coordinates of the first position, the relative or absolute orientation of the object as seen from the first position, the position coordinates of the second position, the relative or absolute orientation of the object as seen from the second position, the position coordinates of the third position, and the relative or absolute orientation of the object as seen from the third position obtained from the third measurement data.

9. An information control system according to claim 1, wherein the group of mobile objects is composed of a plurality of unmanned boats capable of navigating on water, and the first measurement data and the second measurement data are acquired by the measurement sensors provided on the unmanned boats.

10. An information control system as described in claim 9, wherein the plurality of unmanned vessels include at least a first unmanned vessel and a second unmanned vessel, the first measurement data is acquired at the first position by the measurement sensor provided on the first unmanned vessel, and the second measurement data is acquired at the second position by the measurement sensor provided on the second unmanned vessel.

11. An information control system as described in claim 9, wherein the plurality of unmanned vessels include at least a first unmanned vessel, the first measurement data is acquired at the first position by the measurement sensor provided on the first unmanned vessel, and the second measurement data is acquired at the second position by the measurement sensor provided on the first unmanned vessel that has moved from the first position to the second position.

12. An information control system as described in claim 1, wherein the group of mobile objects includes an unmanned boat capable of navigating on water and an air vehicle capable of flying in the sky, the first measurement data is acquired by the measurement sensor provided on the unmanned boat, and the second measurement data is acquired by the measurement sensor provided on the air vehicle.

13. An information control system as described in claim 1, wherein the group of mobile objects includes an unmanned craft capable of navigating on the surface of water and a submersible craft capable of navigating underwater, the first measurement data being acquired by the measurement sensor provided on the unmanned craft, and the second measurement data being acquired by the measurement sensor provided on the submersible craft.

14. An information control system as described in claim 1, wherein the group of mobile objects includes an air vehicle capable of flying in the sky and a submersible vehicle capable of navigating underwater, the first measurement data being acquired by the measurement sensor provided on the air vehicle, and the second measurement data being acquired by the measurement sensor provided on the submersible vehicle.

15. An information control system according to claim 1, comprising a position calculation method determination unit that determines a calculation method for the position information of the object from any of the following combinations of methods: triangulation that calculates the position of the object using the relative angle between the object and the second position as seen from the first position and the relative angle between the object and the first position as seen from the second position; standalone surveying that obtains the relative distance between the object and the mobile body based on measurement data obtained from a measurement sensor provided on one of the mobile bodies; geometric calculation that calculates the position coordinates of the geometric intersection of a line passing through the object and the first position and a line passing through the object and the second position.

16. An information control system according to claim 15, wherein the position calculation method determination unit determines a method for calculating the position information of the target object based on at least one of the measurement data obtained from the measurement sensor provided on the moving body, information on the calculation accuracy of the moving body's own position, information on the calculation accuracy of the relative distance between at least two of the moving bodies, and information on the calculation accuracy of the relative orientation or absolute orientation of the moving body.

17. An information control system as described in claim 15, wherein, when the measurement data acquired from the measurement sensor provided on the moving body is image data acquired by an optical camera, the position calculation method determination unit determines a method for calculating the position information of the object from either the triangulation or the geometric calculation when it detects that the exposure of the image data is higher than an upper limit tolerance, the amount of light in the image data is lower than a lower limit tolerance, or the field of view of the image data is shorter than a lower limit distance.

18. An information control system according to claim 1, wherein the mobile object operation control unit determines the first position and the second position so as to satisfy at least one of the following conditions: the relative distance from the object to the first position and the relative distance from the object to the second position are within a predetermined distance; the relative distance between the first position and the second position is within a predetermined distance; and the angle between the first position and the second position as seen from the object is within a predetermined angle range.

19. An information control system according to claim 18, wherein the mobile body operation control unit determines, based on the first position and the second position, a first mobile body to be moved to the first position and instructed to acquire the measurement data, and a second mobile body to be moved to the second position and instructed to acquire the measurement data.

20. An information control system according to claim 1, wherein the mobile body operation control unit commands a first mobile body and a second mobile body present at the first position and the second position that satisfy at least one of the following conditions: the relative distance from the object to the first position and the relative distance from the object to the second position are within a predetermined distance, the relative distance between the first position and the second position is within a predetermined distance, and the angle between the first position and the second position as seen from the object is within a predetermined angle range, to acquire the measurement data.

21. An information control system as described in claim 1, wherein, when the object analysis unit determines that the object is a ship, it compares information about the object determined from the measurement data based on the position information of the object determined by the object position determination unit with information about the object obtained from the object by wireless communication by an external automatic ship identification system, and determines, based on the result of the comparison, whether the object corresponds to a predetermined monitoring target.

22. An information control system as described in claim 1, wherein when the mobile object operation control unit causes the mobile object to communicate with the object by radio communication or light emission, or to call or warn by voice or display device, the object analysis unit determines whether the object corresponds to a predetermined monitoring target based on the object's behavior in response to the communication, call, or warning.

23. An information control system as described in claim 1, comprising a recording unit that records at least one of the position information of the object determined by the object position determination unit, the analysis processing results of the object analysis unit, the measurement data obtained by measuring the object, and the operation history of the mobile body in association with the identification information of the object.

24. A control method for a system that measures an object using a group of mobile objects consisting of multiple mobile objects equipped with measurement sensors that can acquire measurement data of the object when at least a portion of the object is above water, wherein a computer executes the following steps: a mobile object operation control step that controls the operation of the mobile objects; a measurement data acquisition step that acquires first measurement data obtained by measuring the object from a first position by at least one mobile object of the group of mobile objects, and second measurement data obtained by measuring the object from a second position by that mobile object or another mobile object; an object position determination step that determines position information including at least one of relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object based on the first measurement data and the second measurement data; and an information output step that displays or transmits to the outside the position information of the object determined by the object position determination step, or displays or transmits to the outside the analysis processing results of an object analysis unit that analyzes the object based on the position information.

25. A program usable in a system for measuring an object using a group of mobile objects consisting of multiple mobile objects equipped with measurement sensors capable of acquiring measurement data of the object when at least a portion of the object is above water, the program causing a computer to execute: a mobile object operation control command to control the operation of the mobile objects; a measurement data acquisition command to acquire first measurement data obtained by measuring the object from a first position by at least one mobile object in the group of mobile objects, and second measurement data obtained by measuring the object from a second position by the mobile object or another mobile object; an object position determination command to determine position information including at least one of relative position coordinates, absolute position coordinates, relative distance, relative direction, and absolute direction of the object based on the first measurement data and the second measurement data; and an information output command to display or transmit to an external device the position information of the object determined by the object position determination command, or to display or transmit to an external device the analysis processing results of an object analysis unit that analyzes the object based on the position information.

Images