Information control system, control method and program

Abstract

When using a drone to monitor or survey an object on the water, the position of the object can be determined more accurately, or the object can be analyzed more accurately. [Solution] The present invention is an information control system comprising a group of moving bodies consisting of multiple moving bodies equipped with measurement sensors capable of acquiring measurement data measuring an object when at least a portion of it is above water, a moving body operation control unit that controls the operation of the moving bodies, a measurement data acquisition unit that acquires first measurement data measured of the object from a first position by at least one of the moving bodies in the group of moving bodies and second measurement data measured of the object from a second position by the moving body or another moving body, an object position determination unit that determines position information of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information, and an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit.

Description

[Technical field]

[0001] The present invention relates to an information control system, a control method, and a program. [Background technology]

[0002] Patent document 1 discloses a technology that improves the ease of identifying suspicious ships by using an external device that is connected via a network to an imaging device equipped with an imaging unit for imaging a ship as a subject, and that is equipped with an acquisition means for acquiring image data including the ship imaged by the imaging unit, a display means for displaying the image data, an analysis means for extracting estimated ship information of any ship contained in the image data based on the image data, a receiving means for receiving ship information based on wireless communication from the ship, and a comparison means for comparing the estimated ship information extracted by the analysis means with the ship information received by the receiving means, and the display means displays a warning in addition to the image data if the comparison means detects a difference between the estimated ship information and the ship information.

[0003] Marine surveillance has been carried out by manned surveillance boats and research vessels for the purpose of preventing nuisances and illegal fishing caused by ships navigating on the sea or divers navigating underwater, or for the purpose of ecological surveys of marine life such as whales and dolphins. However, since the range of the marine area to be monitored or surveyed is very vast, there is a limit to the area that can be monitored or surveyed by manned surveillance boats and research vessels, and there is a problem that areas remain that cannot be monitored or surveyed. In addition, when monitoring or surveying is carried out by coordinating a large number of manned surveillance boats, it is not easy to quickly control and control them manually, and it is difficult to appropriately perform actions such as measuring the monitored object in cooperation. In addition, it is not easy to train personnel with the skills to control and manage them. In this background, in recent years, the use of unmanned ships that can navigate the sea autonomously has been considered, and it is expected that they will be used for the above-mentioned monitoring of suspicious ships and ecological surveys. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-19359 A Summary of the Invention [Problem to be solved by the invention]

[0005] As in Patent Document 1, when determining whether a ship is suspicious by comparing the ship's characteristic information extracted from captured image data with the ship information received from the ship via wireless communication, if the exact position of the ship cannot be determined, there is a possibility that the target ship to be compared with the ship information received from the ship will be incorrect, so it is necessary to detect the ship's position more accurately.

[0006] In addition, when an unmanned vessel requests other unmanned vessels or external systems to perform continuous measurement tasks, such as continuously measuring ships, divers, or marine life such as whales and dolphins detected by the unmanned vessel, or a tracking task, the provision of more accurate location information of the detected objects is required.

[0007] Therefore, the present invention has been made in consideration of at least one of the above problems, and one object of the present invention is to provide a system or control method, etc., that can more accurately determine the position of an object or more accurately analyze an object when monitoring or investigating an object on water using an unmanned aerial vehicle. [Means for solving the problem]

[0008] According to the present invention, there is provided a mobile body group consisting of multiple mobile bodies equipped with measurement sensors capable of acquiring measurement data measuring an object with at least a portion thereof above water, a mobile body operation control unit for controlling the operation of the mobile bodies, a measurement data acquisition unit for acquiring first measurement data measuring the object from a first position by at least one of the mobile bodies in the mobile body group and second measurement data measuring the object from a second position by the mobile body or another mobile body, and an object position determination unit for determining position information including at least any of relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object based on the first measurement data and the second measurement data, or for determining the validity of the determined position information. An information control system is obtained which is provided with an information output unit which displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside the analysis processing results of an object analysis unit which analyzes the object based on the position information. Effect of the Invention

[0009] According to the present invention, when an unmanned aerial vehicle is used to monitor or survey an object on water, the position of the object can be determined more accurately, or the object can be analyzed more accurately. [Brief description of the drawings]

[0010] [Figure 1] 1 is an overall configuration diagram of an information control system 1 according to one embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of an implementation image when the information control system 1 is implemented in a real space. [Diagram 3] FIG. 2 is a diagram showing stakeholders related to the information control system 1. [Figure 4] FIG. 1 is a configuration diagram showing a platoon made up of unmanned boats 1000. [Diagram 5] 1 is a conceptual diagram showing an unmanned boat 1000 deployed on the sea monitoring or investigating an object 7000. [Figure 6]2 is a functional block diagram showing the functional configuration of the unmanned boat 1000. FIG. [Figure 7] FIG. 2 is a functional block diagram showing the functional configuration of an overall control system 2000. [Figure 8] FIG. 13 is a diagram showing items determined by a detection determining unit 2200. [Figure 9] FIG. 13 is a conceptual diagram showing a position calculation method when using a single survey. [Figure 10] FIG. 13 is a conceptual diagram showing a position calculation method when triangulation is used. [Figure 11] FIG. 13 is a conceptual diagram showing a position calculation method when a geometric calculation is used. [Figure 12] FIG. 2 is a hardware configuration diagram of the overall control system 2000. [Figure 13] FIG. 2 is a flowchart showing a processing flow of the information control system 1. [Figure 14] 2 is a sequence diagram showing signal exchange between systems in the information control system 1. FIG. [Figure 15] 13 is a flowchart showing an example of a process flow for determining a position calculation method by the position calculation method determination unit 2300. FIG. [Figure 16] 13 is a flowchart showing an example of a process flow for determining a detailed measurement operation by a detailed measurement operation control unit 2400. FIG. [Figure 17] FIG. 13 is a conceptual diagram showing a detailed measurement decision process performed by the detailed measurement operation control unit 2400. [Figure 18] FIG. 13 is a diagram showing a first example of an aircraft combination selected by the measurement aircraft selection unit 2420. [Figure 19] FIG. 13 is a diagram showing a second example of an aircraft combination selected by the measurement aircraft selection unit 2420. [Figure 20] FIG. 13 is a diagram showing a third example of an aircraft combination selected by the measurement aircraft selection unit 2420. [Figure 21] FIG. 13 is a flowchart showing an example of a position calculation process flow using single point surveying by a detailed position determination unit 2500. [Figure 22]FIG. 13 is a flowchart showing an example of a position calculation process flow using triangulation by a detailed position determination unit 2500. [Diagram 23] FIG. 13 is a flowchart showing an example of a position calculation process flow using geometric calculation by the detailed position determining unit 2500. [Figure 24] FIG. 13 is a flowchart showing an example of a position calculation process flow in which a detailed position determination unit 2500 combines a plurality of position calculation methods. [Diagram 25] FIG. 13 is a flowchart showing an example of a process flow for determining an additional operation by the additional operation control unit 2600. [Figure 26] FIG. 13 is a flowchart showing an example of an object analysis process flow by the object analysis unit 2700. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present invention will be described below with reference to the preferred embodiments thereof. [Item 1] a group of moving bodies including a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data of an object at least a part of which is above water; A moving body operation control unit that controls the operation of the moving body; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination unit that determines position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; An information control system comprising: an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside the analysis processing results of an object analysis unit that analyzes the object based on the position information. [Item 2] In the information control system according to item 1, The object position determination unit calculates the position information of the object based on information on the angle between the object and the second position as seen from the first position obtained from the first measurement data, the angle between the object and the first position as seen from the second position obtained from the second measurement data, and the relative distance between the first position and the second position. [Item 3] In the information control system according to item 1 or 2, the measurement data acquisition unit acquires third measurement data obtained by measuring the object from the third position; The object position determination unit is calculating the position information of the object based on information on a relative angle between the object and the first position as viewed from the third position obtained from the third measurement data, a relative angle between the object and the third position as viewed from the first position obtained from the first measurement data, and a relative distance between the first position and the third position; An information control system that compares the calculated multiple pieces of position information of the object to determine the position information of the object. [Item 4] In the information control system according to any one of items 1 to 3, the measurement data acquisition unit acquires third measurement data obtained by measuring the object from the third position; The object position determination unit is calculating the position information of the object based on information on a relative angle between the object and the first position as viewed from the third position obtained from the third measurement data, a relative angle between the object and the third position as viewed from the first position obtained from the first measurement data, and a relative distance between the first position and the third position; calculating the position information of the object based on information on a relative angle between the object and the second position as viewed from the third position obtained from the third measurement data, a relative angle between the object and the third position as viewed from the second position obtained from the second measurement data, and a relative distance between the second position and the third position; An information control system that compares the calculated multiple pieces of position information of the object to determine the position information of the object. [Item 5] In the information control system according to any one of items 1 to 4, The measurement sensor includes a stereo camera, a monocular camera, a laser distance measuring sensor, a radar distance measuring sensor, an acoustic distance measuring sensor, an optical distance measuring sensor, or any other sensor capable of measuring distance; An information control system, wherein the object position determination unit determines the position information of the object based on the first measurement data acquired by the measurement sensor from the first position, the position coordinates of the first position, the second measurement data acquired by the measurement sensor from the second position, and the position coordinates of the second position. [Item 6] In the information control system according to any one of items 1 to 5, An information control system, wherein the object position determination unit determines the position information of the object based on the first measurement data, the position coordinates of the first position, the second measurement data, the position coordinates of the second position, third measurement data measured by the measurement sensor from the third position, and the position coordinates of the third position. [Item 7] In the information control system according to any one of items 1 to 6, the object position determination unit determines, based on the position coordinates of the first position, the relative orientation or absolute orientation of the object seen from the first position obtained from the first measurement data, the position coordinates of the second position, and the relative orientation or absolute orientation of the object seen from the second position obtained from the second measurement data, An information control system that determines the position information of the object by calculating position coordinates of a geometric intersection of a line passing through the object and the first position and a line passing through the object and the second position. [Item 8] In the information control system according to any one of items 1 to 7, the measurement data acquisition unit acquires third measurement data obtained by measuring the object from the third position; the object position determination unit determines, based on the position coordinates of the first position, the relative orientation or absolute orientation of the object seen from the first position, the position coordinates of the second position, the relative orientation or absolute orientation of the object seen from the second position, the position coordinates of the third position, and the relative orientation or absolute orientation of the object seen from the third position obtained from the third measurement data, An information control system that determines the position information of the object by calculating position coordinates of a geometric intersection of a line passing through the object and the first position, a line passing through the object and the second position, and a line passing through the object and the third position. [Item 9] In the information control system according to any one of items 1 to 8, The group of moving bodies is composed of a plurality of unmanned boats capable of navigating on water, An information control system, wherein the first measurement data and the second measurement data are acquired by the measurement sensor provided on the unmanned craft. [Item 10] In the information control system according to any one of items 1 to 9, the plurality of unmanned craft include at least a first unmanned craft and a second unmanned craft; the first measurement data is acquired at the first position by the measurement sensor provided on the first unmanned craft; An information control system, wherein the second measurement data is acquired at the second position by the measurement sensor provided on the second unmanned craft. [Item 11] In the information control system according to any one of items 1 to 10, the plurality of unmanned craft include at least a first unmanned craft; the first measurement data is acquired at the first position by the measurement sensor provided on the first unmanned craft; An information control system, wherein the second measurement data is acquired at the second position by the measurement sensor provided on the first unmanned craft that has moved from the first position to the second position. [Item 12] In the information control system according to any one of items 1 to 11, The group of moving objects includes an unmanned boat capable of navigating on water and an aircraft capable of flying in the sky, the first measurement data is acquired by the measurement sensor provided on the unmanned watercraft; An information control system, wherein the second measurement data is acquired by the measurement sensor provided on the aircraft. [Item 13] In the information control system according to any one of items 1 to 12, The group of moving bodies includes an unmanned craft capable of navigating on the surface of water and a submarine craft capable of navigating underwater, the first measurement data is acquired by the measurement sensor provided on the unmanned watercraft; An information control system, wherein the second measurement data is acquired by the measurement sensor provided on the submersible. [Item 14] In the information control system according to any one of items 1 to 13, The group of moving objects includes an aircraft capable of flying in the sky and a submarine capable of navigating underwater, the first measurement data is acquired by the measurement sensor provided on the aircraft; the second measurement data is acquired by the measurement sensor provided in the submersible; Information and control systems. [Item 15] In the information control system according to any one of items 1 to 14, triangulation to calculate a position of the object using a relative angle between the object and the second position as viewed from the first position and a relative angle between the object and the first position as viewed from the second position; A single survey to acquire a relative distance between the object and one of the moving bodies based on measurement data acquired from the measurement sensor provided on one of the moving bodies; A geometric calculation for calculating a position coordinate of a geometric intersection point between a straight line passing through the object and the first position and a straight line passing through the object and the second position; An information control system comprising a position calculation method determination unit that determines a method for calculating the position information of the object from any one of these combinations of methods. [Item 16] In the information control system according to any one of items 1 to 15, An information control system, wherein the position calculation method determination unit determines a method for calculating the position information of the object based on at least any of the following information: the measurement data obtained from the measurement sensor provided on the moving body, information regarding the calculation accuracy of the moving body's own position, information regarding the calculation accuracy of the relative distance between at least two of the moving body group, and information regarding the calculation accuracy of the relative orientation or absolute orientation of the moving body. [Item 17] In the information control system according to any one of items 1 to 16, When the measurement data acquired from the measurement sensor provided on the moving body is image data acquired by an optical camera, The position calculation method determination unit determines a method for calculating the position information of the object from either the triangulation or the geometric calculation when it detects that the exposure of the image data is greater than an upper tolerance, the amount of light of the image data is less than a lower tolerance, or the field of view of the image data is shorter than a lower limit distance. [Item 18] In the information control system according to any one of items 1 to 17, The mobile object operation control unit determines the first position and the second position so as to satisfy at least one of the following conditions: a relative distance from the object to the first position and a relative distance from the object to the second position are within a predetermined distance, a relative distance between the first position and the second position is within a predetermined distance, and an angle between the first position and the second position as seen from the object is within a predetermined angle range. [Item 19] In the information control system according to any one of items 1 to 18, The mobile body operation control unit determines, based on the first position and the second position, a first mobile body to be moved to the first position and instructed to acquire the measurement data, and a second mobile body to be moved to the second position and instructed to acquire the measurement data. [Item 20] In the information control system according to any one of items 1 to 19, The mobile object operation control unit instructs a first mobile object and a second mobile object that are present at the first position and the second position, respectively, to acquire the measurement data when the relative distance from the object to the first position and the relative distance from the object to the second position are within a predetermined distance, the relative distance between the first position and the second position is within a predetermined distance, and the angle between the first position and the second position as seen from the object is within a predetermined angle range. [Item 21] In the information control system according to any one of items 1 to 20, The object analysis unit includes: If the object is determined to be a ship, An information control system that, based on the position information of the object determined by the object position determination unit, compares information about the object determined from the measurement data with information about the object obtained from the object via wireless communication by an external automatic ship identification system, and determines whether the object corresponds to a predetermined monitoring target based on the comparison result. [Item 22] In the information control system according to any one of items 1 to 21, When the unmanned aircraft operation control unit communicates with the target object by wireless communication or light emission, or issues a call or warning by voice or display device from the moving object, An information control system in which the object analysis unit determines whether or not the object corresponds to a predetermined monitoring target based on the object's behavior in response to the communication, call, or warning. [Item 23] In the information control system according to any one of items 1 to 22, An information control system comprising a recording unit that records at least any of the position information of the object determined by the object position determination unit, the analysis processing results of the object analysis unit, the measurement data obtained by measuring the object, and the operation history of the moving body in association with identification information of the object. [Item 24] A method for controlling a system for measuring an object using a group of moving bodies, the group being made up of a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data obtained by measuring the object when at least a portion of the object is above water, comprising the steps of: The computer a moving object motion control step of controlling a motion of the moving object; a measurement data acquisition step of acquiring first measurement data obtained by measuring the object from a first position by at least one of the moving bodies of the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination step of determining position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data; an information output step of displaying or transmitting to the outside the position information of the object determined by the object position determination step, or displaying or transmitting to the outside the analysis processing result of an object analysis unit that analyzes the object based on the position information; A control method for performing the above. [Item 25] A program usable in a system for measuring an object using a group of moving objects, the group being made up of a plurality of moving objects each having a measurement sensor capable of acquiring measurement data obtained by measuring the object when at least a portion of the object is above water, the program comprising: On the computer, A moving object operation control command for controlling an operation of the moving object; a measurement data acquisition command for acquiring first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination command for determining position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data; An information output command for displaying and outputting the position information of the object determined by the object position determination step, or transmitting it externally, or displaying and outputting the analysis processing result of the object analysis unit that analyzes and processes the object based on the position information, or transmitting it externally, and A program for executing.

[0012] <A. First Embodiment> Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted. Further, the embodiments shown below are merely examples, and other known elements and alternative means can be adopted according to the use, purpose, scale, etc.

[0013] [A-1. Configuration] (A-1-1. System Configuration) First, with reference to FIGS. 1 and 2, the system configuration of an information control system 1 according to an embodiment of the present invention will be described.

[0014] (A-1-1-1. Outline of System Configuration) FIG. 1 is an overall configuration diagram of an information control system 1 (hereinafter also referred to as "system 1") according to an embodiment of the present invention. As shown in FIG. 1, the information control system 1 includes an unmanned boat 1000 and a general control system 2000. Further, the general control system 2000 is configured to be able to communicate with an external cooperation system 5000 and an external system 6000 via an Internet line or the like, and can perform input and output of information. The general control system 2000 can transmit a control command to the unmanned boat 1000 deployed at sea via a communication satellite 3000 and a ground base station 4000, and can receive the operation status and measurement data of the unmanned boat 1000.

[0015] The unmanned boat 1000 includes a parent unit 1001 capable of communicating with a communication satellite 3000, and a child unit 1002 capable of communicating directly or indirectly with the parent unit 1001, and establishes a communication network between the multiple child units 1002 and the parent unit 1001. The multiple child units 1002 and the parent unit 1001 also have a function of measuring ships, divers, drifting objects, castaways, marine life such as whales, and other objects at least partially exposed above the water using measurement sensors (optical cameras, IR cameras, laser sensors such as LiDAR, radar sensors such as millimeter wave sensors and microwave sensors, sonar, and the like) mounted on the boats.

[0016] The detection determination results and measurement data of the object detected by the unmanned boat 1000, as well as various information on the operation status of the unmanned boat 1000, are transmitted to the overall control system 2000 via the communication satellite 3000 and the terrestrial base station 4000. The overall control system 2000 determines operation commands for the unmanned boat 1000 based on information acquired from the unmanned boat 1000 and pre-registered information. Information such as the generated operation commands is transmitted to the cooperative system 5000, and the overall control system 2000 can also obtain intervention commands from the cooperative system.

[0017] (A-1-1-2. Example of implementation of information control system 1 in real space) Fig. 2 is a diagram showing an example of an implementation image when the information control system 1 is implemented in real space. In the example shown in Fig. 2, a terrestrial base station 4000 and a central control system 2000 are provided on the ground side shown in the upper right of the drawing. Also provided on the ground side is a cooperative system 5000 including related facilities of external cooperative organizations such as marine research organization related facilities and private law enforcement organization related facilities (including private security organizations, private rescue organizations, etc.), and further, an external system 6000 such as an AIS (Automatic Identification System) control center and an AIS base station that obtain information about ships from ships navigating the ocean via wireless communication and manage the ship information is provided.

[0018] On the other hand, on the ocean side shown on the left side of the drawing, an unmanned boat 1000, an object 7000 for monitoring and investigation such as a suspicious ship or marine life, and a part of a cooperative system 5000 such as a surveillance boat or a survey boat operated by an external cooperative organization are deployed. The unmanned boat 1000 also has a plurality of platoons (platoon a, platoon b, platoon c) consisting of a parent unit and a plurality of child units, and each platoon can communicate directly or via a communication satellite 3000. The unmanned boat 1000 can also communicate with a surveillance boat directly or via a communication satellite 3000, and for example, the unmanned boat 1000 can notify the surveillance boat (or survey vessel) of detection information regarding the object 7000. The unmanned boat 1000 may also be connected to an AIS base station so as to be able to communicate with it and acquire AIS information. In addition, the unmanned boat 1000 can communicate with manned or unmanned air vehicles 8100 and submersible boat 8200, either directly or via a communication satellite 3000, and can, for example, measure the target object 7000 in cooperation with the unmanned boat 1000, the air vehicle 81000, and the submersible boat 8200.

[0019] In the example shown in Figure 2, the overall control system 2000 is implemented in a facility on land, but this is not limited to this. All or some of the functions implemented in the overall control system 2000 shown in this embodiment can also be installed on other coastal field bases on land or manned mother ships at sea (not shown), and the unmanned boat 1000 can be operated and managed at the coastal field base or manned mother ship.

[0020] (A-1-2. Stakeholders regarding Information and Control System 1) Fig. 3 is a diagram showing stakeholders related to the information control system 1. As shown in Fig. 3, the information control system 1 has an operator who operates the unmanned boat 1000 by inputting and outputting information via a user interface unit 2800 of the overall control system 2000. When all or a part of the functions implemented in the overall control system 2000 shown in this embodiment are implemented in a coastal field base on land or a manned mother ship on the sea (not shown), the operator can manage the operation of the unmanned boat 1000 at the coastal field base or the manned mother ship.

[0021] In addition, the cooperative system 5000 has a monitoring manager at the private law enforcement organization-related facility and a monitor on the monitoring boat, and they work together to monitor nuisances in the marine area. In addition, the marine research organization-related facility has an investigation manager and a research vessel has an investigator, and they work together to investigate marine life in the marine area. In addition, the cooperative system 5000 may also include private security companies and private rescue organizations. In addition, the AIS control center of the external system 6000 has a person in charge of generating, operating, and managing AIS information.

[0022] Furthermore, the objects 7000 that are the targets of monitoring and investigation by the information control system 1 and the cooperative system 5000 include suspicious ships, marine buoys, divers, and marine life (such as whales). The information control system 1 can more efficiently monitor or investigate the objects 7000 by communicating with and coordinating with the cooperative system 5000 and the external system 6000.

[0023] (A-1-3. Configuration of the unmanned boat 1000) Fig. 4 is a configuration diagram showing a platoon 1010 composed of unmanned boats 1000. As shown in Fig. 4, the unmanned boat 1000 is composed of one or more platoons 1010 (1010a, 1010b). Each platoon 1010 has at least one master unit 1001 and multiple slave units 1002. The master unit 1001 has a communication connection with a communication satellite 3000, and has the function of aggregating information collected from the multiple slave units 1002 and transmitting the information to the communication satellite 3000 and transmitting information related to operation commands obtained from the communication satellite 3000 and information generated by the master unit 1001 directly or indirectly to each slave unit 1002.

[0024] 4 includes a primary connection slave device 10021 that is communicatively connected to the parent device 1001, a secondary connection slave device 10022 that is communicatively connected to the primary connection slave device 10021, and a tertiary connection slave device 1023 that is communicatively connected to the secondary connection slave device 10022. Each slave device (primary connection slave device 10021, secondary connection slave device 10022, tertiary connection slave device 1023) has a function of relaying information received from another parent device 1001 or slave device 1002 to the other parent device 1001 or slave device 1002, thereby forming a communication network between the parent device 1001 and the multiple slave devices 1002.

[0025] In addition, at least some of the parent units 1001 or child units 1002 in the platoon are connected to the flying object 8100 or the submarine 8200 via a wireless communication network, and the unmanned boat 1000 and the flying object 8100 or the submarine 8200 can operate in cooperation with each other to measure the target object 7000. In addition, the flying object 8100 and the submarine 8200 can transmit measurement data of the target object 7000 to the unmanned boat 1000 via wireless communication.

[0026] Fig. 5 is a conceptual diagram showing how an unmanned boat 1000 deployed on the sea monitors or investigates an object 7000. As shown in Fig. 5, multiple unmanned boats (master boat 1001, slave boats 10021, 10022, 10023) are deployed on the sea, and the object 7000 present within a measurable range can be measured by a measurement sensor 1110 mounted on each unmanned boat 1000. Measurement data and detection judgment results of the object 7000 detected by the measurement sensor 1110 are collected in the master boat 1001 via a wireless communication network between the unmanned boats 1000, transmitted from the master boat 1001 to a communication satellite 3000, and transmitted to an overall control system 2000 via a terrestrial base station 4000 and an Internet line. In addition, each unmanned boat 1000 is equipped with a navigation unit 1300 that can navigate the unmanned boat in any direction, and can perform detailed measurement operations on the target object 7000 based on operational commands generated by the overall control system 2000 or the parent unit 1001.

[0027] As shown in FIG. 5, the flying object 8100 flying in the sky area is connected to at least some of the unmanned boats 1000 in the platoon via a wireless communication network. The flying object 8100 is equipped with measurement sensors (one or more optical cameras, IR cameras, laser sensors such as LiDAR, radar sensors such as millimeter wave sensors and microwave sensors, sound wave sensors such as sonar, etc.) and can measure the object 7000 from the sky in cooperation with the unmanned boat 1000. The submarine 8200 traveling underwater is also connected to at least some of the unmanned boats 1000 in the platoon via a wireless communication network. The submarine 8200 is also equipped with measurement sensors (one or more optical cameras, IR cameras, laser sensors such as LiDAR, radar sensors such as millimeter wave sensors and microwave sensors, sound wave sensors such as sonar, etc.) and can measure the object 7000 from underwater in cooperation with the unmanned boat 1000.

[0028] As the configuration of the present embodiment described in FIG. 1 to FIG. 5, an example has been described in which a non-terrestrial network using a communication satellite 3000 placed in a geosynchronous orbit, a medium earth orbit (MEO), a low earth orbit, or another orbit is used as a communication network for transmitting and receiving information between the overall control system 2000 and the unmanned boat 1000. However, the present invention is not limited to this, and a non-terrestrial network using an unmanned aerial vehicle called a High Altitude Platform Station (HAPS) can also be used. In this case, for example, an unmanned aerial vehicle that circles at an altitude of about 8 to 50 km can be used. In addition, as a communication network for transmitting and receiving information between the overall control system 2000 and the unmanned boat 1000, a communication network that directly connects the terrestrial base station 4000 to the unmanned boat 1000 by wireless communication without going through the communication satellite 3000 or the HAPS can also be used. Note that the terrestrial base station 4000 is not limited to a fixed base station that is immobile, and may be configured as a mobile base station that can be moved.

[0029] (A-1-4. Configuration of the unmanned boat 1000) Next, the functions and their contents implemented in the unmanned boat 1000 will be described with reference to Fig. 6. In the present invention, the unmanned boat refers to a moving body capable of navigating on or underwater, regardless of whether it is an autonomous navigation or remote control type, and means a moving body including a mobile buoy equipped with a thrust generating unit.

[0030] Fig. 6 is a functional block diagram showing the functional configuration of the unmanned boat 1000. Note that Fig. 6 describes the functional block diagram of the unmanned boat 1000, but the parent unit 1001 and child unit 1002 of the unmanned boat 1000 can implement functions similar to those shown in Fig. 6. The unmanned boat 1000 includes a measurement unit 1100, a host state determination unit 1200, a navigation unit 1300, a communication unit 1400, a determination unit 1500, a recording unit 1600, and an other action execution unit 1700.

[0031] The measurement unit 1100 is a functional unit that detects an object 7000 present within a measurable range around the unmanned boat 1000 using a measurement sensor 1110, and acquires measurement information regarding the object 7000. The measurement unit 1100 includes the measurement sensor 1110 and a measurement control unit 1120.

[0032] The measurement sensor 1110 may include one (monocular) or multiple electro-optical sensors that acquire image data on the sea, optical sensors such as optical cameras, infrared sensors (IR sensors), and stereo cameras, laser sensors such as LiDAR that acquire point cloud data, optical distance measuring sensors such as ToF sensors (Time of Flight sensors), and radar sensors that detect millimeter waves and microwaves. The measurement sensor 1110 acquires measurement data of an object 7000 that exists within a measurable range on the sea by measuring the periphery of the unmanned boat 1000. In addition, each of the above-mentioned sensors can be used as a distance measuring sensor that measures the distance to an object based on the measurement data.

[0033] In addition to the above-mentioned sensors, the measurement sensor 1110 may have a sonic sensor (also called a sonic measurement unit) including a sonar that uses sound waves such as ultrasonic waves. The sonic sensor can be used not only underwater but also in the air above the water. When the sonic sensor is used in the air, it can be used as a distance measurement sensor that measures the distance to an object by measuring the sound waves that are generated and reflected by the object and return. When the sonic sensor is used underwater, the sonic sensor may be either an active sonar that generates sound waves and measures the sound waves that reverberate from an object in the water, or a passive sonar that measures the sound generated from an object in the water. The active sonar may be, for example, a side scan sonar, a multi-beam sonar, or a single beam sonar. The sonic sensor may be a USBL transceiver, a modem for acoustic communication, or the like.

[0034] Furthermore, the measurement control unit 1120 operates a sensor attitude changing device capable of changing the attitude of the measurement sensor 1110 to control at least one of the attitude angles around the three axes of the measurement sensor 1110 relative to the unmanned boat 1000. Furthermore, for example, when the measurement sensor is an optical sensor, the measurement control unit 1120 can adjust the frame rate, the shutter speed, and the like. Furthermore, when the measurement sensor is a laser sensor, the measurement control unit 1120 can adjust the output of the irradiated laser. Furthermore, when the measurement sensor is a radar sensor, the measurement control unit 1120 can adjust the output of the millimeter wave or microwave. Furthermore, the measurement control unit 1120 can adjust the measurement sensitivity of the measurement sensor to an arbitrary control amount. Furthermore, when the measurement sensor is an optical sensor, the measurement control unit 1120 can change the zoom amount or resolution of the optical sensor to an arbitrary control amount.

[0035] Next, the vehicle state determination unit 1200 includes a navigation state determination unit 1210, an internal state determination unit 1220, and an external state determination unit 1230, and is a functional unit that determines the navigation state and internal and external states of the unmanned boat 1000. The navigation state determination unit 1210 determines the vehicle's position (two-dimensional or three-dimensional), moving speed, heading, moving direction, moving acceleration / deceleration, turning speed, and other state quantities related to the navigation state. The internal state determination unit 1220 determines the remaining energy of the battery and fuel installed in the vehicle, the travelable distance that can be calculated based on the remaining energy, temporary abnormal states of equipment installed in the vehicle (temperature abnormality, communication abnormality, etc.), and equipment failure states. In addition, the external condition determination unit 1230 determines communication conditions such as communication strength (dB value, etc.) and communication speed with other unmanned boats 1000 in the platoon 1010 with which it is communicating, as well as ocean currents and tides (current speed, flow direction), wind speed (wind speed, wind direction), wave height, and weather (rain, snow, cloudy, etc.) around the aircraft.

[0036] The method of determining the position, moving speed, moving direction, and acceleration / deceleration of the aircraft by the navigation state determination unit 1210 is not particularly limited, but for example, the position, moving speed, and moving direction of the aircraft at the current time can be determined using GNSS (Global Navigation Satellite System), GPS (Global Positioning System), RTK-GNSS (Real Time Kinematic - Global Navigation Satellite System), etc. Here, the aircraft's own position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information. Moreover, the acceleration / deceleration can be calculated based on the amount of change over time in the determined moving speed.

[0037] The method of measuring the heading of the aircraft at the current time is to determine the heading of the aircraft using, for example, a geomagnetic sensor, a GNSS compass, SLAM technology using the shape of the seabed, etc. The heading includes an attitude angle (orientation) in a planar view around at least the Z axis, and may be attitude information around three axes, preferably the X axis, the Y axis, and the Z axis. The turning speed can be calculated based on the amount of change over time in the determined heading information.

[0038] Next, the navigation unit 1300 is a functional unit that includes a thrust generating unit, an attitude control mechanism, and a navigation control unit, and navigates the parent unit 1001 in any direction according to an operation command received via the communication unit 1400. The thrust generating unit is, for example, configured with a propeller, and can generate thrust by driving the propeller using the power of an engine or an electric motor. The thrust generating unit 1310 can also be configured with a sail that receives wind to generate thrust, or can be configured with a wave glider that receives wave power to generate thrust.

[0039] The attitude control mechanism is composed of a rudder provided on the aircraft body, a propeller attitude changing mechanism capable of changing the propeller attitude angle (mainly the yaw angle around the Z axis), and the like, and by changing these angles it is possible to control the nose direction (yaw angle) of the unmanned boat 1000. In addition, the attitude angles of the roll angle around the X axis and the pitch angle around the Y axis of the aircraft body can also be controlled by a center of gravity position changing mechanism that changes the position of a heavy object inside the aircraft body with an actuator.

[0040] The navigation control unit is a functional unit that controls the output from the thrust generating unit and the attitude control mechanism to control the navigation operation of the aircraft. The navigation control unit includes a processing unit having one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP), and that can access a memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more processing steps.

[0041] The processing unit includes a control module configured to control the navigation state of the aircraft. For example, the control module adjusts the aircraft's position on the sea surface, moving speed, moving acceleration / deceleration, heading, turning speed, and attitude angle around three axes. That is, the navigation control unit 1330 controls the navigation operation of the aircraft by making the aircraft perform various operations such as forward movement, reverse movement, acceleration, deceleration, and turning.

[0042] Next, the communication unit 1400 includes an unmanned boat-to-unmanned boat communication unit 1410, a satellite communication unit 1420, and an external communication unit 1430, and is a functional unit that communicates with other unmanned boats 1000 in the platoon 1010, a communication satellite 3000, external flying objects 8100, submarines 8200, surveillance boats, and AIS base stations. The unmanned boat-to-unmanned boat communication unit 1410 includes a communication antenna for unmanned boat-to-unmanned boat communication, and communicates with other unmanned boats 1000 in the platoon 1010. The satellite communication unit 1420 includes a satellite communication antenna, and communicates with the communication satellite 3000. The external communication unit 1430 includes an AIS antenna and a VHF antenna, and communicates with external surveillance boats and AIS base stations.

[0043] Next, the determination unit 1500 is a functional unit that performs determination regarding the target object 7000. The determination unit 1500 includes an object detection determination unit 1510. The object detection determination unit 1510 interprets the measurement data acquired by the measurement sensor 1110 and determines the presence or absence of an object, the size of the object, and the like.

[0044] The object detection determination unit 1510 determines whether or not to transmit the measurement data to the integrated control system 2000 and perform an object analysis based on the interpretation information of the measurement data. For example, even if the object detection determination unit 1510 detects an object on the water, if the estimated size of the object is small and it is highly likely to be driftwood or the like that is not a monitored object, it can determine that it is unnecessary to transmit the measurement data to the integrated control system 2000. On the other hand, if the estimated size of the object detected by the object detection determination unit 1510 is larger than a predetermined standard and it is highly likely to be a ship or the like that is a monitored object, it can determine that it is necessary to transmit the measurement data to the integrated control system 2000.

[0045] Next, the recording unit 1600 includes a measurement data recording unit 1610, a device's state recording unit 1620, and a determination information recording unit 1630. The measurement data recording unit 1610 records measurement data measured by the measurement unit 1100. The device's state recording unit 1620 records various types of state information related to the device determined by the device's state determination unit 1200. Furthermore, the determination information recording unit 1630 records various types of determination information determined by the determination unit 1500.

[0046] The other action execution unit 1700 is a functional unit that communicates with a detected object 7000 such as a ship by communication or light emission, and issues a call or warning by voice. The other action execution unit 1700 includes a communication unit 1710 and a voice output unit 1720.

[0047] The communication unit 1710 can communicate with the ship, which is the target 7000, by wireless communication via the communication unit 1400. The communication unit 1710 can also communicate by transmitting a Morse code by illuminating a light emitting unit provided on the unmanned boat 1000. The audio output unit 1720 is composed of, for example, a speaker, and can issue voice calls and warnings to the target 7000.

[0048] (A-1-5. Configuration of the integrated control system 2000) Next, the functions and contents of the overall control system 2000 will be described with reference to Fig. 7. Fig. 7 is a functional block diagram showing the functional configuration of the overall control system 2000. As shown in Fig. 7, the overall control system 2000 includes an information import unit 2100, a detection determination unit 2200, a position calculation method determination unit 2300, a detailed measurement operation control unit 2400, a detailed position determination unit 2500, an additional operation control unit 2600, an object analysis unit 2700, a user interface unit 2800, a recording unit 2910, and an information communication unit 2920.

[0049] (A-1-5-1. Information import unit 2100) The information import unit 2100 is a functional unit that imports information to be processed or used in each functional unit in the overall control system 2000 from the unmanned boat 1000, the cooperative system 5000, or the external system 6000. The information import unit 2100 includes a judgment condition acquisition unit 2110, a measurement data acquisition unit 2120, an external information acquisition unit 2130, and an external intervention information acquisition unit 2140.

[0050] The judgment condition acquisition unit 2110 is a functional unit that acquires in advance criteria for each judgment and decision executed by the judgment unit 1500 of the unmanned boat 1000, the detection judgment unit 2200, the position calculation method decision unit 2300, the detailed measurement operation control unit 2400, the detailed position judgment unit 2500, and the additional operation control unit 2600 of the overall control system 2000. The judgment condition acquisition unit 2110 can acquire, for example, a reference value such as the size of a detected object as a judgment criterion for the object detection judgment unit 1510 of the judgment unit 1500 to judge whether or not measurement data needs to be transmitted to the overall control system 2000 and whether or not target analysis is needed. In addition, the judgment condition acquisition unit 2110 can acquire judgment criteria for each judgment item by the detection judgment unit 2200 shown in FIG. 8.

[0051] The measurement data acquisition unit 2120 is a functional unit that acquires the determination results determined by the determination units 1500 of the multiple unmanned boats 1000 via the communication satellite 3000, the HAPS, the terrestrial base station 4000, etc., and the measurement data measured by the multiple unmanned boats 1000, the air vehicle 8100, and the submarine 8200. The measurement data acquired by the measurement data acquisition unit 2120 includes first measurement data acquired by a monitoring measurement operation by the unmanned boat 1000, detailed measurement data acquired based on a detailed measurement operation commanded to the unmanned boat 1000 by a detailed measurement operation control unit 2400 described later, and additional measurement data acquired based on an additional measurement operation commanded to the unmanned boat 1000 by an additional operation control unit 2600.

[0052] The external information acquisition unit 2130 is a functional unit that acquires navigation information of ships in the ocean area where the unmanned boat 1000 is deployed or the surrounding area from the AIS control center of the external system 6000. The external information acquisition unit 2130 may also acquire navigation information of ships from another VHF data exchange system included in the external system 6000. The external information acquisition unit 2130 may also acquire weather information for the ocean area where the unmanned boat 1000 is deployed or the surrounding area from the external system 6000, such as the Japan Meteorological Agency or a private weather information providing system.

[0053] The external intervention information acquisition unit 2140 is a functional unit that receives intervention command information from the cooperative system 5000. For example, the external intervention information acquisition unit 2140 can receive intervention command information for candidate information of measurement operation commands and information on the necessity of additional measurement, which are sent to the cooperative system 5000 via an information communication unit 2920 described later, from the cooperative system 5000.

[0054] (A-1-5-2. Detection and determination unit 2200) The detection and judgment unit 2200 is a functional unit that analyzes the object based on the judgment criteria information acquired by the judgment condition acquisition unit 2110 and the measurement data of the object (particularly the primary measurement data) acquired by the measurement data acquisition unit 2120, and judges information and conditions regarding the object.

[0055] The object detection and determination by the detection and determination unit 2200 will be described below with reference to FIG. 8. FIG. 8 is a diagram showing the determination items by the detection and determination unit 2200. As shown in FIG. 8, the detection and determination unit 2200 can include a monitoring target determination of whether or not an object is a monitoring target. In addition, as the object characteristic determination items of the object, the type, shape, size, and orientation of the object can be determined. In addition, as the static state determination items, the relative distance, orientation, and position coordinates of the object can be determined. In addition, as the dynamic state determination, the moving state / stationary state, moving direction, moving speed, past moving route history, future predicted route, and the like can be determined.

[0056] An example of a method for determining a static state will be described. First, the relative distance can be obtained based on analysis processing of image data obtained by one or more optical sensors. In addition, the relative distance can be obtained based on measurement data obtained by laser sensors such as LiDAR and ToF sensors (Time of Flight sensors), radar sensors that detect millimeter waves and microwaves, and sonars that use sound waves such as ultrasonic waves.

[0057] Also, the direction of the object as seen from the unmanned boat 1000 can be determined from the direction information of the object detected from the measurement data acquired by an optical sensor, a laser sensor, a radar sensor, or the like. Here, the direction information of the object may be calculated as an absolute direction in a global coordinate system, or may be calculated as a relative direction in a local coordinate system based on the position and nose direction of the unmanned boat 1000, for example. Furthermore, the position coordinates of the object can be calculated based on the information of the self-position coordinates detected by the own-craft state determination unit 1200 of the unmanned boat 1000 and the above-mentioned relative direction and relative distance information. Here, the position coordinates may be two-dimensional coordinates on a horizontal XY plane, but are preferably three-dimensional coordinates in an XYZ space including information in the height direction. Moreover, the position coordinates may be local position coordinates in a local coordinate system or global position coordinates in a global coordinate system (absolute coordinate system).

[0058] An example of a method for determining a dynamic state will be described below. Based on a change analysis of measurement data obtained by performing measurements multiple times in a time series, it is possible to determine whether the state is moving or stationary, and to determine the direction and speed of movement. In addition, it is possible to determine the history of past movement routes based on past measurement data. Furthermore, it is possible to determine a predicted future route based on the results of past movement history, current movement direction, orientation, and the like.

[0059] As described above, the detection and determination unit 2200 can determine the relative distance, relative orientation, and position coordinates as static state determination items for the object, but the accuracy of the determination information is not sufficiently high because each piece of position-related determination information is determined based on primary measurement data acquired by a single unmanned boat 1000. Therefore, in order to obtain highly accurate position information required for detailed analysis of the object or for requesting an external system to rush to the site, more detailed position information is determined by each functional unit described below.

[0060] Here, in the type determination of the detected object by the detection determination unit 2200, for example, it is possible to determine objects at least partially above the water, such as suspicious ships, divers, castaways, drifting ships, drifting objects, marine buoys, and marine life (whales, dolphins, schools of fish, etc.). In this way, by detecting suspicious ships and divers, it is possible to monitor nuisances at sea. By detecting castaways and drifting ships, it is possible to rescue castaways. By detecting drifting objects (debris, wood, etc.), it is possible to obtain obstacle information for safe navigation of ships. By detecting marine buoys, it is possible to support a system that recovers marine buoys and communicates with marine buoys. By monitoring marine life (whales, dolphins, schools of fish, etc.), it is possible to support marine surveys.

[0061] (A-1-5-3. Position calculation method determination unit 2300) Next, a description will be given of the position calculation method determination unit 2300. The position calculation method determination unit 2300 is a functional unit that determines a calculation method for calculating position information including at least one of a relative position coordinate, a relative distance, a relative orientation, and an absolute position coordinate of a detected object from a plurality of candidate position calculation methods.

[0062] For example, the candidates of the position calculation method selected by the position calculation method decision unit 2300 may include, for example, position calculation using a single point survey, position calculation using triangulation survey, and position calculation using geometric calculation. Each position calculation method will be described below with reference to Figs. 9 to 11.

[0063] FIG. 9 is a conceptual diagram showing a position calculation method when independent surveying is used. In the example shown in FIG. 9, a case is shown in which position information of an object 7000 is calculated using three unmanned boats (1000a, 1000b, 1000c). When independent surveying is used as shown in this figure, the unmanned boat 1000a calculates the relative distance from the unmanned boat 1000a to the object 7000 by analyzing and processing measurement data obtained by its own measurement sensor, and obtains detection direction information of the object 7000 as seen from the unmanned boat 1000a. Furthermore, the unmanned boat 1000a obtains its own position information (position coordinate information of its own boat) by its own boat state determination unit 1200, and calculates the position coordinates of the object 7000 based on the obtained information. Furthermore, the unmanned boat 1000b also performs a similar process to calculate the position coordinates of the object 7000, and by comparing the calculation results of the position coordinates of the object 7000 calculated by each of them, it is possible to determine the position coordinates more accurately.

[0064] 9, the same process is performed in the unmanned boat 1000c to calculate the position coordinates of the object 7000, and the position coordinates of the object 7000 calculated based on the measurement data acquired by the three unmanned boats (1000a, 1000b, 1000c) are compared, thereby making it possible to more accurately determine the position coordinates of the object 7000. The position calculation method when using the independent survey shown in FIG. 9 will be described in detail later.

[0065] FIG. 10 is a conceptual diagram showing a position calculation method when triangulation is used. In the example shown in FIG. 10, a case is shown in which position information of an object 7000 is calculated using three unmanned boats (1000a, 1000b, 1000c). When triangulation is used as shown in this figure, the unmanned boats 1000a and 1000b calculate the relative distance from each other's position coordinates or the signal reception strength (dB) of each other's wireless communication. In addition, the unmanned boat 1000a acquires the relative angle between the object 7000 and the unmanned boat 1000b as seen from the unmanned boat 1000a, and the unmanned boat 1000b acquires the relative angle between the object 7000 and the unmanned boat 1000a as seen from the unmanned boat 1000a. Here, the position coordinates of the object 7000 are calculated by triangulation based on the position coordinate information of at least one of the unmanned boats 1000a and 1000b and the above-mentioned acquired information. In this way, by performing triangulation based on measurement data measured from the positions of multiple unmanned boats 1000, it is possible to calculate the position information of the target object 7000 more accurately than with a position calculation method using a single unmanned boat 1000.

[0066] In addition, in the example shown in Figure 10, by using another unmanned boat 1000c in addition to the two unmanned boats 1000a and 1000b described above, for example, the relative distance between unmanned boat 1000a and unmanned boat 1000c, the relative angle between unmanned boat 1000c and object 7000 as seen from unmanned boat 1000a, and the relative angle between unmanned boat 1000a and object 7000 as seen from unmanned boat 1000c, and the relative angle between unmanned boat 1000a and object 7000 as seen from unmanned boat 1000c, the position coordinates of object 7000 can be calculated by triangulation based on the position coordinate information of at least one of unmanned boat 1000a and unmanned boat 1000c and the above-mentioned acquired information. Furthermore, by comparing the position coordinates of the target 7000 calculated using unmanned boat 1000a and unmanned boat 1000b with the position coordinates of the target 7000 calculated using unmanned boat 1000a and unmanned boat 1000c, the position information of the target 7000 can be calculated more accurately than when two unmanned boats are used.

[0067] In addition to the above-mentioned method, it is also possible to calculate the position coordinates of the target object 7000 using unmanned boats 1000b and 1000c, and by adding this position information to the comparison process, it is possible to calculate the position information of the target object 7000 more accurately. The position calculation method when using triangulation shown in Fig. 10 will be described in detail later.

[0068] FIG. 11 is a conceptual diagram showing a position calculation method when geometric calculation is used. In the example shown in FIG. 11, a case is shown in which position information of an object 7000 is calculated using three unmanned boats (1000a, 1000b, 1000c). When using geometric calculation as shown in this figure, first, a geometric formula of a line connecting the unmanned boat 1000a and the object 7000 is calculated from the position coordinate information of the unmanned boat 1000a and the detected orientation information of the object 7000, and similarly, a geometric formula of a line connecting the unmanned boat 1000b and the object 7000 is calculated from the position coordinate information of the unmanned boat 1000b and the detected orientation information of the object 7000 for the unmanned boat 1000b. Furthermore, by calculating the position coordinates of the intersection of these two lines, the position information of the object 7000 can be calculated. In this way, by performing calculations using geometric formulas based on measurement data measured from the positions of multiple unmanned boats 1000, it is possible to calculate the position information of the target object 7000 more accurately than a position calculation method using a single unmanned boat 1000.

[0069] In the example shown in Fig. 11, by using another unmanned boat 1000c in addition to the two unmanned boats 1000a and 1000b described above, for example, a geometric formula for a line connecting the unmanned boat 1000c and the object 7000 is calculated from the position coordinate information of the unmanned boat 1000c and the detected orientation information of the object 7000, and the intersection of this line formula and the line connecting the unmanned boat 1000a and the object 7000 described above or the line connecting the unmanned boat 1000b and the object 7000 is calculated, thereby calculating the position information of the object 7000. In this way, by performing a calculation using a geometric formula based on measurement data measured from the positions of the three unmanned boats 1000, it is possible to calculate the position information of the object 7000 more accurately than the position calculation method using two unmanned boats 1000. Details of the position calculation method using the three geometric formulas shown in Fig. 11 will be described later.

[0070] As described above, the candidates for the position calculation method selected by the position calculation method determination unit 2300 may include, for example, the position calculation using the single point surveying shown in Fig. 9, the position calculation using the triangulation surveying shown in Fig. 10, and the position calculation using the geometric calculation shown in Fig. 11, but are not limited to these and may include other position calculation methods. Furthermore, the candidates for the position calculation method selected by the position calculation method determination unit 2300 may include a method that combines the above-mentioned position calculation methods.

[0071] (A-1-5-4. Detailed measurement operation control unit 2400) The detailed measurement operation control unit 2400 is a functional unit that controls the operation of the unmanned boat 1000, the flying body 8100, and the submarine 8200 to perform measurement operations of measurement data necessary to execute the position calculation method selected by the position calculation method determination unit 2300 and other information acquisition operations. The detailed measurement operation control unit 2400 includes a measurement vehicle placement determination unit 2410, a measurement vehicle selection unit 2420, an vehicle operation plan determination unit 2430, and a detailed measurement operation command unit 2440.

[0072] The measurement vehicle arrangement determination unit 2410 has a function of determining measurement positions suitable for executing the position calculation method selected by the position calculation method determination unit 2300. For example, it determines multiple measurement positions such as the measurement positions of the unmanned boats (1000a, 1000b, 1000c) shown in Figs. 9, 10, and 11 based on criteria that the measurement positions must satisfy, including the distance from the object being within a reference distance, the relative distance between multiple measurement positions being equal to or greater than a reference distance, or the angle formed by multiple measurement positions as seen from the object being equal to or greater than a reference angle. Note that information on the above criteria can be acquired by the judgment condition acquisition unit 2110.

[0073] The measurement vehicle selection unit 2420 is a functional unit that selects a vehicle to perform measurement operations at multiple measurement positions based on information on multiple measurement positions determined by the measurement vehicle placement determination unit 2410 and current position information of each unmanned boat 1000, flying body 8100, and submarine 8200. For example, an unmanned boat 1000 that is close to the multiple measurement positions determined by the measurement vehicle placement determination unit 2410, or an unmanned boat that has already been deployed at the multiple measurement positions determined, can be selected as the vehicle to perform measurement operations.

[0074] The vehicle motion plan determination unit 2430 is a functional unit that determines a motion plan for the unmanned vessel 1000 selected by the measurement vessel selection unit 2420 to move to a plurality of measurement positions determined by the measurement vessel placement determination unit 2410 and perform measurements. For example, the motion plan may be a motion plan to move along the shortest route in order to move in the shortest time, or a motion plan to move along a route that requires less energy by utilizing tides and wind. A predicted time required to complete movement to the measurement position may also be calculated. When performing measurements at a measurement position, the motion plan may be a motion plan to perform measurements by synchronizing the measurement timing with other unmanned vessels 1000, or to perform measurements at the top dead center of vertical movement caused by waves.

[0075] The detailed measurement operation command unit 2440 is a functional unit that outputs an operation command to the unmanned boat 1000 selected by the measurement aircraft selection unit 2420 to perform a detailed measurement operation to move to multiple measurement positions determined by the measurement aircraft placement determination unit 2410 in accordance with the movement plan determined by the aircraft operation plan determination unit 2430 and acquire measurement data of the target object.

[0076] (A-1-5-5. Detailed position determination section 2500) Next, a description will be given of the function of the detailed position determination unit 2500. The detailed position determination unit 2500 is a functional unit that calculates the position information of an object using the position calculation method determined by the position calculation method determination unit 2300 based on measurement data measured by the unmanned boat 1000 or the like, and determines whether the calculated position information is appropriate. The detailed position determination unit 2500 includes an object position calculation unit 2510 and a calculated position appropriateness determination unit 2520.

[0077] The object position calculation unit 2510 is a functional unit that calculates position information of an object using a position calculation method determined by the position calculation method determination unit 2300 based on detailed measurement data measured by at least two or more unmanned boats 1000 in accordance with a detailed measurement operation command from the detailed measurement operation control unit 2400, or additional measurement data measured by at least two or more unmanned boats 1000 in accordance with an additional measurement operation command from the additional operation control unit 2600. Note that the object position calculation unit 2510 can calculate position information including at least any of the relative position coordinates, absolute position coordinates, relative distance, relative orientation, and absolute orientation of the object as the position information of the object.

[0078] When calculating the position information of an object based on measurement data measured by at least two or more unmanned boats 1000, it is necessary to determine whether the same object is included in the multiple measurement data. Therefore, the object position calculation unit 2510 determines the identity of the object for which position information is to be calculated based on the characteristics of the object, such as the shape and size, determined by the detection determination unit 2200. The detailed position calculation method by the object position calculation unit 2510 will be described later.

[0079] The calculated position suitability determination unit 2520 is a functional unit that determines whether the position information of the object calculated by the object position calculation unit 2510 is appropriate. For example, when the calculated position suitability determination unit 2520 determines that the calculated position information of the object is not appropriate, it determines that the position information is indefinite. The calculated position suitability determination unit 2520 can also determine that the accuracy of the calculated position information of the object is insufficient. A detailed processing method by the calculated position suitability determination unit 2520 will be described later.

[0080] (A-1-5-6. Additional operation control unit 2600) Next, a description will be given of the additional operation control unit 2600. The additional operation control unit 2600 is a functional unit that controls additional operations by the unmanned boat 1000, the flying object 8100, and the submarine 8200 in accordance with the determination results of the detailed position determination unit 2500 and the detection determination unit 2200. The additional operation control unit 2600 includes an additional operation determination unit 2610 and an additional operation command unit 2620.

[0081] The additional operation decision unit 2610 is a functional unit that decides additional operations to be performed by the unmanned boat 1000, the flying body 8100, or the submarine 8200. For example, if the calculated position suitability determination unit 2520 determines that the calculated position information of the target object is uncertain, it decides to perform an additional detailed measurement operation under the same conditions or from a different angle as the additional operation. Also, if it determines that the accuracy of the calculated position information of the target object is insufficient, it decides to obtain a close-up image (measurement data) from an even closer position or obtain an image with a higher resolution as the additional operation.

[0082] As another example, if the detection determination unit 2200 determines that the object corresponds to a monitoring target or determines that the type of object is a ship, the additional operation decision unit 2610 can decide as an additional operation to communicate with the object from the unmanned boat 1000, the air vehicle 8100, or the submarine 8200 via wireless communication or light emission (including Morse code, etc.), or to call out to the object via voice (audio output from a speaker) or display device, or to issue a warning (to stop illegal activity, to leave the current sea area, etc.).

[0083] In this way, as an additional operation, when communicating with an object by wireless communication or light emission, or calling out or warning by voice or display device, the communication, call, or warning may be made by an operator of the overall control system 2000 via the user interface unit 2800 described later, or the communication, call, or warning may be made directly by a user of the cooperative system 5000, such as a private law enforcement organization, by wireless communication with the cooperative system 5000 via the information and communication unit 2920 described later.

[0084] The additional action command unit 2620 is a functional unit that commands the unmanned boat 1000, the flying body 8100, and the submarine 8200 to carry out the additional action determined by the additional action determination unit 2610.

[0085] (A-1-5-7. Object Analysis Unit 2700) Next, a description will be given of the object analysis unit 2700. The object analysis unit 2700 is a functional unit that analyzes an object based on measurement data measured by at least two or more unmanned boats 1000 or the like in accordance with a detailed measurement operation command from the detailed measurement operation control unit 2400, position information of the object determined by the detailed position determination unit 2500, or additional measurement data obtained by an additional operation.

[0086] The object analysis unit 2700 can, for example, make a judgment on the judgment items judged by the detection judgment unit 2200 as shown in Fig. 8. Note that the object analysis unit 2700 judges these judgment items based on measurement data measured by at least two or more unmanned boats 1000 in accordance with a detailed measurement operation command from the detailed measurement operation control unit 2400, position information of the object judged by the detailed position judgment unit 2500, and additional measurement data obtained by an additional operation, and therefore can make these judgments with higher accuracy than the judgments made by the detection judgment unit 2200.

[0087] As an example, when determining whether a detected object corresponds to a suspicious ship registered in advance as a monitoring target, the object analysis unit 2700 can make a definitive determination as to whether the object corresponds to a suspicious ship based on close-up image data from a nearby position or high-resolution image data acquired as additional measurement data. Alternatively, the object analysis unit 2700 can make a definitive determination as to whether the object corresponds to a suspicious ship based on image data measured from an angle different from the previous measurement as additional measurement data. In this way, when making a definitive determination as to whether the object corresponds to a suspicious ship, for example, by comparing the name, shape characteristics, color, ship registration number, etc. of the suspicious ship registered in advance with information that can be read from the acquired detailed image data, it is possible to make a definitive determination as to whether the object corresponds to a suspicious ship.

[0088] As another example, when the object analysis unit 2700 determines that the object is a ship, it compares the information about the object determined from the measurement data based on the position information of the object determined by the detailed position determination unit 2500 with the ship information about the object obtained from the object by the AIS control center (Automatic Identification System) of the external system 6000 via wireless communication, and based on the comparison result, it can make a definitive determination as to whether or not the object corresponds to a predetermined monitoring target.

[0089] As another example, when the additional operation control unit 2600 causes the unmanned boat 1000, the air vehicle 8100, or the submarine 8200 to execute an additional operation of communicating with an object by wireless communication or light emission, or calling or warning with a voice or display device (warning to stop illegal activities, to leave the current sea area, etc.), the object analysis unit 2700 can determine whether or not the object corresponds to a predetermined monitoring target according to the object's behavior in response to the communication, call, or warning. For example, if a ship or the like flees in response to the communication, call, or warning, or if there is no response to the call, etc., the ship can be definitively determined to be a suspicious ship.

[0090] (A-1-5-8. User interface unit 2800) Next, the user interface unit 2800 will be described. The user interface unit 2800 is a functional unit that notifies or displays the results of each determination and control state by the detection determination unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position determination unit 2500, the additional operation control unit 2600, and the object analysis unit 2700 to a user, and receives user input information related to commands for measurement operations of the unmanned boat 1000, etc. from the user. The user interface unit 2800 includes a display unit 2810 and a user input receiving unit 2820. The user interface unit 2800 may be a portable mobile terminal such as a smartphone, a tablet terminal, or a notebook PC.

[0091] The display unit 2810 is a functional unit that notifies or displays to the user the measurement data acquired by the measurement data acquisition unit 2120, the results of each judgment or decision by the detection and determination unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position determination unit 2500, the additional operation control unit 2600, the object analysis unit 2700, etc., operation command candidates for the unmanned boat 1000, etc., the operation history (including the movement path, etc.) of the unmanned boat 1000, etc. When notifying the user, such as when the latest measurement data has been acquired or an object has been detected, the display unit 2810 can notify the user not only by display output but also by sound, light emission, or vibration.

[0092] For example, the display unit 2810 can display and output the measurement data acquired by the measurement data acquisition unit 2120 and the detection judgment result judged by the detection judgment unit 2200 (the judgment result for each judgment item shown in FIG. 8). In addition, the display unit 2810 can display and output the position calculation method determined by the position calculation method decision unit 2300. In addition, the display unit 2810 can display and output the operation command or a candidate of the detailed measurement operation including the measurement device arrangement, the measurement device selection result, and the device movement plan determined by the detailed measurement operation control unit 2400. In addition, the display unit 2810 can display and output the detailed position information of the object judged by the detailed position judgment unit 2500 and the suitability judgment result of the calculated position information. In addition, the display unit 2810 can display and output the operation command or a candidate of the additional operation determined by the additional operation control unit 2600. In addition, the display unit 2810 can display and output the detection judgment result judged by the object analysis unit 2700 (the judgment result for each judgment item shown in FIG. 8).

[0093] In addition, when information about the object is transmitted to the collaborative system 5000 or other external systems by the information communication unit 2920 described below, information such as the destination's contact details, contact means, location, etc. may be displayed on the display unit 2810.

[0094] The user input accepting unit 2820 is a functional unit that accepts user input for each piece of information displayed by the display unit 2810. User input information can also be accepted via an operation button provided on the display screen of the display unit 2810. For example, the user input accepting unit 2820 can accept a user input for approval or change of the position calculation method determined by the position calculation method determining unit 2300, or can accept a designation input of an arbitrary position calculation method.

[0095] The user input accepting unit 2820 can also accept user input for approval or change of the detailed measurement operation operation command or its candidate including the measurement aircraft arrangement, the measurement aircraft selection result, and the aircraft operation plan determined by the detailed measurement operation control unit 2400, or can accept designation input of any measurement aircraft arrangement, the measurement aircraft selection result, and the aircraft movement plan. The user input accepting unit 2820 can also accept user input for approval or change of the operation command or its candidate for the additional operation determined by the additional operation control unit 2600, or can accept designation input of any additional operation including a command to obtain more detailed position information.

[0096] In addition, when the additional operation determination unit 2610 determines that an additional operation is communication by wireless communication or light emission from the unmanned boat 1000, the flying body 8100, or the submarine 8200 to an object, or an operation of calling out or issuing a warning using voice or a display device, the user input receiving unit 2820 can receive input from the user in the form of voice, text, or commands for each of the communication, call, and warning operations.

[0097] Furthermore, when displaying measurement data and each judgment result on the display unit 2810, the user input accepting unit 2820 can accept from the user a display mode including displaying the latest measurement data and judgment results, or displaying measurement data and judgment results at a specified time in the past recorded in a recording unit described later. Furthermore, when the unmanned boat 1000 acquires the latest measurement data, the display unit 2810 or the like may notify the user that the latest measurement data has been updated. Furthermore, an estimated time required for display when a request to display past measurement data is accepted may be displayed.

[0098] (A-1-5-9. Recording unit 2910) The recording unit 2910 is a functional unit that records the measurement data acquired by the measurement data acquisition unit 2120, the results of each judgment or decision by the detection determination unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position determination unit 2500, the additional operation control unit 2600, the object analysis unit 2700, etc., operation command candidates for the unmanned boat 1000, etc., the operation history of the unmanned boat 1000, etc. (including measurement operations, communications, calls, warnings, etc.) etc. Furthermore, this information can be recorded in association with the identification information of the detected object.

[0099] In addition to the above-mentioned information, the recording unit 2910 may also include historical information on the transmitted and received data sent and received between the unmanned boat 1000 and the overall control system 2000, user input information regarding data transmission and reception, ocean condition data (wave height, tides, weather, etc.), and information regarding loss of target position capture.

[0100] (A-1-5-10. Information and Communications Department 2920) The information communication unit 2920 is a functional unit that outputs various information similar to the information displayed on the display unit 1810, that is, the measurement data acquired by the measurement data acquisition unit 2120, the results of each judgment and decision by the detection judgment unit 2200, the position calculation method determination unit 2300, the detailed measurement operation control unit 2400, the detailed position judgment unit 2500, the additional operation control unit 2600, the target analysis unit 2700, etc., candidates for operation commands to the unmanned boat 1000, etc., and the operation history of the unmanned boat 1000, etc., to the cooperative system 5000, the external system 6000, or other external systems.

[0101] The information communication unit 2920 can also obtain information similar to the information received by the user input reception unit 2820 from the collaborative system 5000, the external system 6000, or other external systems. That is, the information communication unit 2920 can receive user input information of approval or change to the operation command or candidate of the detailed measurement operation including the measurement aircraft arrangement, the measurement aircraft selection result, and the aircraft movement plan determined by the detailed measurement operation control unit 2400, or can receive a designated input of any measurement aircraft arrangement, the measurement aircraft selection result, or the aircraft movement plan. The information communication unit 2920 can also receive user input information of approval or change to the operation command or candidate of the additional operation determined by the additional operation control unit 2600, or can receive a designated input of any additional operation.

[0102] In addition, when the additional operation determination unit 2610 determines that an additional operation is communication by wireless communication or light emission from the unmanned boat 1000, the air vehicle 8100, or the submarine 8200 to an object, or an operation of calling or warning using audio or a display device, the information communication unit 2920 can receive input in the form of audio, text, or commands for each of the communication, call, and warning operations from the collaborative system 5000, the external system 6000, or other external systems.

[0103] The functions implemented in the unmanned boat 1000 and the overall control system 2000 described above using Figures 6 and 7 are merely one embodiment, and the present invention is not limited to this implementation example. In other words, some of the functions implemented in the unmanned boat 1000 shown in Figure 6 (mainly the function of the determination unit 1500) can be implemented in the overall control system 2000. On the other hand, some of the functions implemented in the overall control system 2000 shown in Figure 7 (mainly at least one of the information import unit 2100, detection determination unit 2200, position calculation method determination unit 2300, detailed measurement operation control unit 2400, detailed position determination unit 2500, additional operation control unit 2600, and object analysis unit 2700) can also be implemented in the unmanned boat 1000. In addition, in this embodiment, an example has been described in which the function of initially detecting an object based on measurement data is implemented in the judgment unit 1500 on the unmanned boat 1000 side and the detection judgment unit 2200 on the overall control system 2000 side, but it is also possible to implement all of this initial detection judgment function on the unmanned boat side, and conversely, it is also possible to implement all of the initial detection judgment function on the overall control system 2000 side.

[0104] (A-1-6. Hardware configuration) 12 is a hardware configuration diagram of a supervisory control system 2000. Here, the supervisory control system 2000 in the present invention is an information processing device such as a server device or a PC. As shown in the figure, the supervisory control system 2000 has an input device 100, an output device 200, a processing device 300, a main memory device 400, an auxiliary memory device 500, a communication device 600, and a bus 700 that electrically connects these devices.

[0105] The input device 100 can constitute the user input receiving unit 2820 of the user interface unit 2800, and is a device for a user to input information and instructions to the integrated control system 2000. Specifically, the input device 100 is, for example, a touch panel, a keyboard, a mouse, or a voice input device such as a microphone.

[0106] The output device 200 is a device that outputs various information generated by the overall control system 2000, and can constitute the display unit 2810 of the user interface unit 2800. Specifically, the output device 200 can constitute the display unit 2810 using a display device for eyewear, AR, or VR, or can be a printer or a speaker.

[0107] The processing device 300 is, for example, a device that performs arithmetic processing. Specifically, the processing device 300 is, for example, a CPU, a microprocessor, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or other semiconductor devices capable of performing calculations.

[0108] The main storage device 400 is a memory device such as a RAM that temporarily stores various read information and a ROM that stores programs and application programs executed by the processing device 300 and various other information. The auxiliary storage device 500 is a non-volatile storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory that can store digital information.

[0109] The communication device 600 is a device for performing wireless or wired information communication with the outside, and can constitute the information communication unit 2920 described above.

[0110] (A-1-7. Control flow of information control system 1) Next, a description will be given of the overall control flow of the information control system 1. FIG 13 is a flow chart showing the process flow of the information control system 1.

[0111] First, the information import unit 2100 acquires various information from the external system 6000 etc. (step 101). In this step, for example, navigation information of ships in the ocean area where the unmanned boat 1000 is deployed or in the surrounding area can be acquired, and information on judgment criteria for judgment processing by the judgment unit 1500 of the unmanned boat 1000 and the detection judgment unit 2200 of the overall control system 2000 can be acquired from the user input acceptance unit 2820 or the cooperative system 5000.

[0112] Next, the unmanned boat 1000 or the like performs monitoring and measurement of the area to be monitored (step 102).

[0113] Next, the determination unit 1500 and the detection determination unit 2200 of the unmanned boat 1000 perform a primary detection determination of an object (step 103). In this step, for example, object detection from the measurement data and determination of each determination item as shown in FIG.

[0114] Next, the process step to transition to is determined depending on whether or not a monitoring target has been detected by the determination unit 1500 or the detection determination unit 2200 (step 104). If a monitoring target candidate is detected in this step, the process transitions to step 105, whereas if a monitoring target candidate is not detected, the process returns to step 103.

[0115] Next, the position calculation method is determined by the position calculation method determination unit 2300 (step 105). In this step, a calculation method for calculating position information including at least one of the relative position coordinates, the relative distance, the relative orientation, and the absolute position coordinates of the detected object is determined from a plurality of candidate position calculation methods.

[0116] Next, the detailed measurement operation control unit 2400 determines the location of the unmanned boat 1000 or the like that will perform detailed measurement (that is, the measurement position) and selects the boat (step 106).

[0117] Next, detailed measurement is performed by the unmanned boat 1000 etc. based on a detailed measurement operation command from the detailed measurement operation control unit 2400 (step 107).

[0118] Next, the detailed position determination unit 2500 calculates detailed position information of the object (step 108).

[0119] Next, the additional operation control unit 2600 determines the operation of additional measurement (step 109).

[0120] Next, based on an additional operation command from the additional operation control unit 2600, an additional measurement operation or the like is executed (step 110).

[0121] Next, the object analysis unit 2700 executes an analysis process of the object (step 111).

[0122] Next, information such as the determination results in each functional unit is recorded in recording unit 2910, displayed on display unit 2810, or transmitted from information communication unit 2920 to an external device such as cooperative system 5000 (step 112).

[0123] (A-1-8. Control sequence within information control system 1) Next, a description will be given of a control sequence between the systems in the information control system 1. FIG 14 is a sequence diagram showing the exchange of signals between the systems in the information control system 1.

[0124] First, when the determination unit 1500 of any one of the mobile bodies such as the unmanned boat 1000 (for example, the slave unit 1002a) determines that the measurement data should be subjected to object analysis in the overall control system 2000, the primary measurement data measured by the slave unit 1002a is transmitted to the parent unit 1001. Furthermore, the parent unit 1001 transmits the received primary measurement data to the overall control system 2000.

[0125] Next, the detection determination unit 2200 of the overall control system 2000 determines whether or not an object has been detected, and the position calculation method determination unit 2300 determines the method for calculating the position of the object. In addition, in response to these determination results, the detailed measurement operation control unit 2400 generates an operation command for detailed measurement.

[0126] Next, the supervisory control system 2000 transmits candidates for operation commands for detailed measurement to the cooperative system 5000, and receives approval or modification of the candidates for operation commands, or input of a designated operation command from the cooperative system 5000.

[0127] Next, the supervisory control system 2000 determines a detailed measurement command based on the designated input information received from the cooperative system 5000 , and transmits the detailed measurement command to the master unit 1001 .

[0128] Next, a detailed measurement command is transmitted from the parent unit 1001 to the child units 1002a and 1002b, and detailed measurement is performed in the child units 1002a and 1002b. The child units 1002a and 1002b transmit the acquired detailed measurement data to the parent unit 1001. Furthermore, the parent unit 1001 transmits the received detailed measurement data to the overall control system 2000.

[0129] Next, detailed position information of the object is calculated by the detailed position determination unit 2500 of the overall control system 2000, and an operation command for an additional operation is generated based on the calculated position information by the additional operation control unit 2600.

[0130] Next, the supervisory control system 2000 transmits candidate operation commands for the additional operations to the collaborative system 5000, and receives approval or modification of the candidate operation commands, or input of a designated operation command from the collaborative system 5000.

[0131] Next, the overall control system 2000 determines an additional operation command based on the designated input information received from the cooperative system 5000 , and transmits the additional operation command to the parent device 1001 .

[0132] Next, an additional operation command is transmitted from the parent unit 1001 to the child units 1002a and 1002b, and the additional operation is executed in the child units 1002a and 1002b. The child units 1002a and 1002b transmit additional measurement data acquired by the additional operation to the parent unit 1001. Furthermore, the parent unit 1001 transmits the received additional measurement data to the overall control system 2000.

[0133] (A-1-9. Position calculation method determination process) Next, the process of determining the position calculation method will be described. Fig. 15 is a flowchart showing an example of the process flow of determining the position calculation method by the position calculation method determination unit 2300. In particular, Fig. 15 shows a detailed process flow of step 105 in the flowchart shown in Fig. 13.

[0134] First, the position calculation method determination unit 2300 acquires various information (image data and other distance measurement data, relative distance to other unmanned boats, positioning accuracy information of the self-position coordinates, etc.) from the unmanned boat 1000 that acquired the primary measurement data (step 201).

[0135] Next, the process step to transition to is determined depending on whether the image accuracy of the image data and other distance measurement data acquired from the unmanned watercraft 1000 is worse than a predetermined standard (step 202). If the image accuracy is worse than the predetermined standard in this step, the process transitions to step 203, whereas if the image accuracy is better than the predetermined standard, the process transitions to step 206.

[0136] For example, in the case where the measurement data obtained from a measurement sensor installed on the unmanned boat 1000 or the like is image data acquired by an optical camera, in step 202, if it is detected that the exposure of the image data is higher than the upper limit tolerance, the amount of light in the image data is lower than the lower limit tolerance, the field of view of the image data is shorter than the lower limit distance, etc., it is determined that the image accuracy is poor.

[0137] Next, if it is determined in step 202 that the image accuracy is better than a predetermined standard, position calculation by independent surveying is determined as the position calculation method (step 206). Here, the position calculation method by independent surveying is a method of obtaining the relative distance between an object and an unmanned boat 1000 based on measurement data obtained from a measurement sensor provided on, for example, one unmanned boat 1000 as shown in Fig. 9.

[0138] Next, if it is determined in step 202 that the image accuracy is worse than the predetermined standard, the process determines which processing step to transition to depending on whether the calculation accuracy of the relative azimuth angle or absolute azimuth angle of the object seen from the unmanned boat 1000 is determined to be worse than the predetermined standard or indefinite (step 203). If it is determined in this step that the calculation accuracy of the relative azimuth angle or absolute azimuth angle is worse than the predetermined standard, the process transitions to step 209, whereas if it is determined that the calculation accuracy of the relative azimuth angle or absolute azimuth angle is better than the predetermined standard, the process transitions to step 204.

[0139] Next, if it is determined in step 203 that the calculation accuracy of the relative azimuth angle or absolute azimuth angle is better than a predetermined standard, the process determines which processing step to transition to depending on whether the calculation accuracy of the relative distance to another unmanned boat 1000 is determined to be worse than the predetermined standard or indefinite (step 204). In this step, if it is determined that the calculation accuracy of the relative distance is worse than the predetermined standard, the process transitions to step 205, whereas if it is determined that the calculation accuracy of the relative distance is better than the predetermined standard, the process transitions to step 207.

[0140] Next, if it is determined in step 204 that the calculation accuracy of the relative distance to other unmanned crafts 1000 is better than a predetermined standard, then triangulation is selected as the position calculation method (step 207). The triangulation position calculation method here is a method of calculating the position information of an object based on information such as the azimuth angle of the object obtained by multiple unmanned crafts 1000 whose relative distances to each other are known, as shown in Fig. 10, for example.

[0141] Next, if it is determined in step 204 that the calculation accuracy of the relative distance to another unmanned vessel 1000 is worse than a predetermined standard, a determination is made as to which processing step to transition to depending on whether the self-location positioning accuracy of the unmanned vessel 1000 using GNSS or the like is worse than the predetermined standard (step 205). If it is determined in this step that the self-location positioning accuracy is worse than the predetermined standard, processing transitions to step 209, whereas if it is determined that the self-location positioning accuracy is better than the predetermined standard, processing transitions to step 208.

[0142] Next, if it is determined in step 205 that the self-positioning accuracy is better than a predetermined standard, a position calculation method using geometric calculation is determined as the position calculation method (step 208). Here, the position calculation method using geometric calculation is a method of calculating the position information of an object by using a geometric straight line equation connecting the object and the self-position by a plurality of unmanned boats 1000 capable of acquiring the self-position coordinates as shown in Fig. 11, for example.

[0143] Next, if it is determined in step 203 that the calculation accuracy of the relative azimuth angle or the absolute azimuth angle is worse than a predetermined standard, or if it is determined in step 205 that the self-positioning accuracy is worse than a predetermined standard, a user command input regarding the position calculation method is accepted from the user input accepting unit 2820 or the cooperative system 5000 (step 209).

[0144] As shown in the flowchart of this figure, the position calculation method determination unit 2300 can determine a method for calculating position information of the unmanned boat 1000, etc. based on at least any of the following information: measurement data obtained from a measurement sensor installed on the unmanned boat 1000, etc.; information regarding the calculation accuracy of the self-position of the unmanned boat 1000, etc.; information regarding the calculation accuracy of the relative distance between at least two of multiple unmanned boats 1000, etc.; and information regarding the calculation accuracy of the relative orientation or absolute orientation of the unmanned boat 1000, etc.

[0145] Furthermore, in step 202, if it is detected that the exposure of the image data is higher than the upper tolerance, the amount of light in the image data is lower than the lower tolerance, the field of view of the image data is shorter than the lower limit distance, etc., and it is determined that the image accuracy is poor, then a transition can be made to a process of selecting the position calculation method from either triangulation or geometric calculation.

[0146] In the example shown in this figure, an example has been described in which the position calculation method is selected automatically or by user specification from among position calculation using single surveying, position calculation using triangulation, and position calculation by geometric calculation. However, this embodiment is not limited to this, and a method that combines position calculation using single surveying, position calculation using triangulation, and position calculation by geometric calculation can also be selected as the position calculation method.

[0147] (A-1-10. Determination of detailed measurement positions and aircraft) Next, the process of determining the detailed measurement position and the aircraft will be described with reference to Figures 16 and 17. Also, the combination of aircraft for performing detailed measurement will be described with reference to Figures 18 to 20.

[0148] (A-1-10-1. Detailed measurement position and aircraft determination process flow) 16 is a flowchart showing an example of a process flow for determining detailed measurement by the detailed measurement operation control unit 2400. In particular, FIG. 16 shows a detailed process flow of step 106 in the flowchart shown in FIG.

[0149] In the flowchart shown in FIG. 16, first, the measurement body arrangement determination unit 2410 narrows down a plurality of measurement positions to areas whose relative distance from the target object is within a reference distance (step 301).

[0150] Next, the measurement body arrangement determination unit 2410 narrows down the multiple measurement positions to areas where the relative distance between the multiple measurement positions is equal to or greater than a reference distance (step 302).

[0151] Next, the measurement body arrangement determination unit 2410 narrows down the multiple measurement positions to an area where the angle formed by the multiple measurement positions as seen from the target object falls within a reference angle range (step 303).

[0152] Next, the measurement body arrangement determination unit 2410 determines a plurality of measurement positions from the narrowed down area (step 304).

[0153] Next, the measurement vehicle selection unit 2420 selects the unmanned vessel 1000 etc. that will perform measurement operations at the determined measurement positions (step 305). In this step, for example, the measurement vehicle selection unit 2420 can select the unmanned vessel 1000 etc. that is closest to the determined measurement positions as the vehicle that will perform measurement operations, based on the current positions of the multiple unmanned vessels 1000, the air vehicle 8100, and the submarine 8200.

[0154] (A-1-10-1. Detailed measurement position and aircraft determination process flow) Fig. 17 is a conceptual diagram showing the detailed measurement decision process by the detailed measurement operation control unit 2400. Fig. 17 shows the position 9000 of the object 7000, the position of the unmanned boat 1000a from which primary measurement data including the object 7000 was acquired, and multiple candidate positions (9001 to 9004) for other measurement positions.

[0155] First, in step 301 in FIG. 16, when a plurality of measurement positions are narrowed down to areas within a reference distance (500 m in the example shown in this figure) from the object 7000, the position 9001 is excluded from candidates for the measurement position.

[0156] 16, the multiple measurement positions are narrowed down to areas where the relative distance between the multiple measurement positions (the relative distance from the position 9000 shown in the figure to each of the measurement position candidates (9002 to 9004)) is equal to or greater than a reference distance (e.g., 300 m or greater). At this time, the position 9002 is excluded from the measurement position candidates.

[0157] 16, the multiple measurement positions are narrowed down to an area where the angle formed by the multiple measurement positions as viewed from the target object 7000 (angles α1 and α2 shown in this figure) is within a reference angle range. At this time, angle α2 is determined to be smaller than the reference angle, and position 9003 is excluded from the candidates for the measurement position.

[0158] Next, in step 304 of Fig. 16, a plurality of measurement positions are determined from the narrowed-down area. Therefore, positions 9000 and 9004 of the unmanned boat 1000a are determined as measurement positions.

[0159] 16, there are several possible cases in which the measurement vehicle selection unit 2420 selects the unmanned boat 1000 etc. to perform measurement operations at the multiple measurement positions determined. For example, if unmanned boat 1000a is present at the position of unmanned boat 1000a and another unmanned boat 1000b is present at position 9004, unmanned boat 1000a and unmanned boat 1000b can be selected as vehicles to perform detailed measurement operations.

[0160] As another example, if there is no unmanned boat 1000a at position 9000 and no other unmanned boat 1000b at position 9004, a first aircraft to move to position 9000 and perform measurement operations, and a second aircraft to move to position 9004 and perform measurement operations can be selected from multiple unmanned boats, air vehicles 8100, or submarines 8200.

[0161] In addition, when selecting the aircraft to move to the position 9000 and the position 9004 and perform the measurement operation, the first aircraft to move to the position 9000 and perform the measurement operation and the second aircraft to move to the position 9004 and perform the measurement operation do not necessarily have to be different aircraft, and it is also possible for the same aircraft to perform the measurement at the position 9000 and then move to the position 9004 and perform the measurement again. In this case, since the measurement data at each of the positions 9000 and 9004 is acquired to calculate the position of the object 7000 as shown in Figs. 9 to 11, it is required to quickly move the aircraft between the positions 9000 and 9004 before the position of the object 7000 moves. Therefore, for example, when moving the unmanned boat 1000 having hydrofoils between the positions 9000 and 9004, it is desirable to move it at high speed in the hydrofoil floating mode.

[0162] (A-1-10-3. Combination of aircraft for performing detailed measurements) Next, a combination of aircraft for performing detailed measurement will be described with reference to Figs. 18 to 20. Fig. 18 is a diagram showing a first example of a combination of aircraft selected by the measurement aircraft selection unit 2420. In the example shown in Fig. 18, in particular, an example is shown in which an unmanned boat (1000a, 1000b) and an aircraft 8100 are selected as aircraft for performing detailed measurement. As shown in this figure, the unmanned boat (1000a, 1000b) and the aircraft 8100 each measure the target object 7000 with a measurement sensor and acquire measurement data. In addition, the aircraft 8100 is connected to at least one unmanned boat via wireless communication, and can receive flight control commands and transmit acquired measurement data to the overall control system 2000. In addition, the aircraft 8100 can be configured as a manned or unmanned aircraft, and any type of aircraft such as a multicopter or fixed wing can be applied.

[0163] FIG. 19 is a diagram showing a second example of a combination of vehicles selected by the measurement vehicle selection unit 2420. In the example shown in FIG. 19, in particular, an example is shown in which the unmanned vehicles (1000a, 1000b) and the submersible vehicle 8200 are selected as vehicles to perform detailed measurement. As shown in this figure, the unmanned vehicles (1000a, 1000b) and the submersible vehicle 8200 each measure the target object 7000 using a measurement sensor to obtain measurement data. The submersible vehicle 8200 is connected to at least one unmanned vehicle 1000b via wireless communication, and can receive flight control commands and transmit the obtained measurement data to the overall control system 2000. The submersible vehicle 8200 can be configured as a manned or unmanned submersible vehicle. In addition, by applying a sensor such as a sonar as the measurement sensor of the submersible vehicle 8200, the relative distance to the target object 7000 can be obtained as measurement data.

[0164] FIG. 20 is a diagram showing a third example of a combination of aircraft selected by the measurement aircraft selection unit 2420. In the example shown in FIG. 20, in particular, an example in which an aircraft 8100 and a submarine 8200 are selected as aircraft to perform detailed measurement is shown. As shown in this figure, the aircraft 8100 and the submarine 8200 each measure the target object 7000 using a measurement sensor to obtain measurement data. In addition, the aircraft 8100 and the submarine 8200 are each connected to at least one unmanned vessel (1001, 1000b) via wireless communication, and can receive flight control commands and navigation control commands, and can transmit the obtained measurement data to the overall control system 2000. The submarine 8200 can be configured as a manned or unmanned submarine. In addition, by applying a sensor such as a sonar as a measurement sensor of the submarine 8200, the relative distance to the target object 7000 can be obtained as measurement data.

[0165] In the example shown in Figure 20, the unmanned boat 1000 plays the role of relaying communications with the aircraft 8100 and the submarine 8200, but the unmanned boat 1000 can also acquire measurement data of the target object 7000, thereby improving the accuracy of calculating the position information of the target object 7000 using measurement data acquired from three or more locations.

[0166] (A-1-11. Detailed position calculation process) Next, the detailed position calculation process performed by the detailed position determining unit 2500 will be described with reference to FIGS.

[0167] (A-1-11-1. Calculation of detailed position by individual survey) 21 is a flow chart showing an example of a position calculation process flow using single-unit surveying by the detailed position determination unit 2500. In particular, the example shows calculation of position information of an object based on measurement data acquired by a stereo camera capable of measuring the distance to the object, a monocular camera, a laser distance measuring sensor, a radar distance measuring sensor, a sonic distance measuring sensor, an optical distance measuring sensor, or other measurement sensor capable of measuring distance.

[0168] First, the relative distance between the object and the unmanned boat 1000a is calculated based on the measurement data of the unmanned boat 1000a (step 401). In this step, for example, when a stereo camera is used as the measurement sensor, the relative distance to the object can be calculated from the acquired stereo image data, or the image data can be subjected to SfM (Structure from Motion) processing to calculate the relative distance. As another example, when a monocular camera is used as the measurement sensor, the relative distance to the object can be calculated by using a monocular camera distance measurement method that utilizes AI or the like.

[0169] Next, azimuth information (relative azimuth angle or absolute azimuth angle) of the object as viewed from the unmanned boat 1000a is obtained based on the measurement data (step 402).

[0170] Next, the unmanned boat 1000a calculates the position coordinates of the target from its own position coordinate information, which it has determined by estimating its own position using GNSS or the like, the relative distance between the target and the unmanned boat 1000a obtained in step 401, and the orientation information of the target as seen by the unmanned boat 1000a obtained in step 402 (step 403).

[0171] Next, the processing of steps 401 to 403 is executed also in the other unmanned boat 1000b, thereby calculating the position coordinates of the target object in a similar manner (step 404).

[0172] Next, the calculated position coordinates are compared (step 405).

[0173] Next, the process step to transition to is determined depending on whether the difference between each of the position coordinates is greater than a predetermined value (step 406). If it is determined in this step that the difference between each of the position coordinates is greater than the predetermined value, the process transitions to step 408, whereas if it is determined that the difference between each of the position coordinates is smaller than the predetermined value, the process transitions to step 407.

[0174] Next, if it is determined in step 406 that the difference between the position coordinates is smaller than a predetermined value, the calculated position suitability determination unit 2520 determines the determination result of the position coordinates (step 407). In this step, any one of the multiple position coordinates to be compared may be selected and determined as the calculation result, or the midpoint or intermediate position coordinate of the multiple position coordinates may be determined as the calculation result.

[0175] Next, if it is determined in step 406 that the difference between the position coordinates is greater than a predetermined value, the calculated position suitability determining unit 2520 determines the result of the position coordinate determination as indefinite (step 408).

[0176] In other words, the flowchart shown in this figure makes it possible to determine the position information of an object based on multiple measurement data (first measurement data, second measurement data) measured from multiple positions (first position, second position) and the position coordinates of the multiple positions (first position, second position).

[0177] Although this figure shows an example in which the position information of an object is determined from measurement data acquired from two positions, the position information of an object can also be determined based on three pieces of measurement data acquired from three positions and the three position coordinates, as shown in Figure 9. In this case, the position information of an object can be determined with higher accuracy than when measurement data is acquired from two positions.

[0178] (A-1-11-2. Calculation of precise position by triangulation) FIG. 22 is a flow chart showing an example of a position calculation process flow using triangulation by the detailed position determining unit 2500.

[0179] First, information on the relative distance between the unmanned boat 1000a and the unmanned boat 1000b is obtained (step 501).

[0180] Next, the angle formed between the object 7000 as seen from the unmanned boat 1000a and the unmanned boat 1000b, and the angle formed between the object 7000 as seen from the unmanned boat 1000b and the unmanned boat 1000a are obtained (step 502).

[0181] Next, the relative distance between unmanned boat 1000a and unmanned boat 1000b is calculated by triangulation using the relative distance between unmanned boat 1000a and unmanned boat 1000b, the angle between unmanned boat 1000b and object 7000 as seen from unmanned boat 1000a, and the angle between unmanned boat 1000a and object 7000 as seen from unmanned boat 1000b (step 503).

[0182] Next, the unmanned boat 1000a calculates the position coordinates of the object 7000 from its own position coordinate information obtained by self-position estimation using GNSS or the like and the relative distance between the object 7000 and the unmanned boat 1000a (step 504).

[0183] Next, the processing of steps 501 to 504 is also executed between the unmanned boat 1000a and the other unmanned boat 1000c, and the position coordinates of the target object 7000 are calculated (step 505).

[0184] Next, the calculated position coordinates are compared (step 506).

[0185] Next, the process step to transition to is determined depending on whether the difference between each of the position coordinates is greater than a predetermined value (step 507). If it is determined in this step that the difference between each of the position coordinates is greater than the predetermined value, the process transitions to step 509, whereas if it is determined that the difference between each of the position coordinates is smaller than the predetermined value, the process transitions to step 508.

[0186] Next, if it is determined in step 507 that the difference between the position coordinates is smaller than a predetermined value, the calculated position suitability determination unit 2520 determines the determination result of the position coordinates (step 508). In this step, any one of the multiple position coordinates to be compared may be selected and determined as the calculation result, or the midpoint or intermediate position coordinate of the multiple position coordinates may be determined as the calculation result.

[0187] Next, if it is determined in step 507 that the difference between the position coordinates is greater than a predetermined value, the calculated position suitability determining unit 2520 determines the result of the position coordinate determination as indefinite (step 509).

[0188] In the example shown in this figure, an example is shown in which the position information of object 7000 calculated based on measurement data obtained by measuring object 7000 from three unmanned boats (1000a, 1000b, 1000c) is compared using the triangle formed by unmanned boat 1000a, unmanned boat 1000b, and object 7000, and the triangle formed by unmanned boat 1000a, unmanned boat 1000c, and object 7000.However, by further calculating the position information of object 7000 using the triangle formed by unmanned boat 1000b, unmanned boat 1000c, and object 7000, and comparing the three calculated position information, it is possible to calculate the position information of object 7000 with even greater accuracy.

[0189] (A-1-11-3. Calculation of detailed position using geometric calculation) FIG. 23 is a flow chart showing an example of the position calculation process flow using geometric calculation by the detailed position determination unit 2500.

[0190] First, the unmanned vessel 1000a calculates a geometric straight line connecting the unmanned vessel 1000a and the target 7000 based on its own position coordinate information, which it has determined by estimating its own position using GNSS or the like, and the azimuth information (relative azimuth angle or absolute azimuth angle) of the target as seen from the unmanned vessel 1000a based on measurement data (step 601).

[0191] Next, the unmanned vessel 1000b calculates a geometric straight line connecting the unmanned vessel 1000b and the target 7000 based on its own position coordinate information, which it has determined by estimating its own position using GNSS or the like, and the azimuth information (relative azimuth angle or absolute azimuth angle) of the target as seen by the unmanned vessel 1000b based on the measurement data (step 602).

[0192] Next, the position coordinates of the intersection of the two lines calculated in steps 601 and 602 are calculated as the position coordinates of the object 7000 (step 603).

[0193] Next, the unmanned vessel 1000c calculates a geometric straight line connecting the unmanned vessel 1000c and the target 7000 based on its own position coordinate information, which it has determined by estimating its own position using GNSS or the like, and the azimuth information (relative azimuth angle or absolute azimuth angle) of the target as seen from the unmanned vessel 1000c based on the measurement data (step 604).

[0194] Next, the position coordinates of the intersection of the two lines calculated in steps 601 and 604 are calculated as the position coordinates of the object 7000 (step 605).

[0195] Next, the position coordinates calculated in steps 603 and 605 are compared (step 606).

[0196] Next, the process step to transition to is determined depending on whether the difference between each of the position coordinates is greater than a predetermined value (step 607). If it is determined in this step that the difference between each of the position coordinates is greater than the predetermined value, the process transitions to step 609, whereas if it is determined that the difference between each of the position coordinates is smaller than the predetermined value, the process transitions to step 608.

[0197] Next, if it is determined in step 607 that the difference between the position coordinates is smaller than a predetermined value, the calculated position suitability determination unit 2520 determines the determination result of the position coordinates (step 608). In this step, any one of the multiple position coordinates to be compared may be selected and determined as the calculation result, or the midpoint or intermediate position coordinate of the multiple position coordinates may be determined as the calculation result.

[0198] Next, if it is determined in step 607 that the difference between the position coordinates is greater than a predetermined value, the calculated position suitability determining unit 2520 determines the result of the position coordinate determination as indefinite (step 609).

[0199] (A-1-11-4. Detailed position calculation process combining multiple position calculation methods) FIG. 24 is a flow chart showing an example of a position calculation process flow in which the detailed position determining unit 2500 combines a plurality of position calculation methods.

[0200] First, the position coordinates of an object 7000 are calculated using a single measurement as shown in FIG. 21 (step 701).

[0201] Next, the position coordinates of the object 7000 are calculated using triangulation as shown in FIG. 22 (step 702).

[0202] Next, the position coordinates of the object 7000 are calculated using a geometric linear equation as shown in FIG. 23 (step 703).

[0203] Next, the position coordinates calculated in steps 701 to 703 are compared (step 704).

[0204] Next, the process step to transition to is determined depending on whether the difference between each of the position coordinates is greater than a predetermined value (step 705). If it is determined in this step that the difference between each of the position coordinates is greater than the predetermined value, the process transitions to step 707, whereas if it is determined that the difference between each of the position coordinates is smaller than the predetermined value, the process transitions to step 706.

[0205] Next, if it is determined in step 705 that the difference between the position coordinates is smaller than a predetermined value, the calculated position suitability determination unit 2520 determines the determination result of the position coordinates (step 706). In this step, any one of the multiple position coordinates to be compared may be selected and determined as the calculation result, or the midpoint or intermediate position coordinate of the multiple position coordinates may be determined as the calculation result.

[0206] Next, if it is determined in step 705 that the difference between the position coordinates is greater than a predetermined value, the calculated position suitability determining unit 2520 determines the result of the position coordinate determination as indefinite (step 707).

[0207] In the example shown in this figure, the position coordinates of the object are calculated using three methods: independent surveying, triangulation, and a geometric linear equation, but a combination of any two of these three methods may also be used.

[0208] (A-1-12. Additional Action Decision Process) Next, the process of determining an additional action will be described. Fig. 25 is a flow chart showing an example of the flow of the process of determining an additional action by the additional action control unit 2600. In particular, Fig. 25 shows a detailed process flow of step 109 in the flow chart shown in Fig. 13.

[0209] First, the detailed position information of the object 7000 calculated by the detailed position determination unit 2500 is obtained (step 801).

[0210] Next, the process step to transition to is determined depending on whether the calculated position suitability determination unit 2520 has determined that the calculated position information result is indefinite (step 802). If the calculated position information result is determined to be indefinite in this step, the process transitions to step 805, whereas if the calculated position information result is determined to be not indefinite, the process transitions to step 803.

[0211] Next, if it is determined in step 802 that the calculation result of the position information is not indefinite, the process determines the processing step to transition to depending on whether the calculation accuracy of the calculated position information is insufficient (step 803). If it is determined in this step that the calculation accuracy of the position information is insufficient, the process transitions to step 805, whereas if it is determined that the calculation accuracy of the position information is not insufficient, the process transitions to step 804.

[0212] Next, if it is determined in step 803 that the calculation accuracy of the location information is sufficient, the process determines the processing step to transition to depending on whether or not there is a request for detailed location information from the user (step 804). In this step, the request for detailed location information is acquired from the user interface unit 2800 or the external collaboration system 5000. In this step, if there is a request for detailed location information, the process transitions to step 806, whereas if there is no request for detailed location information, the process transitions to step 807.

[0213] Next, if it is determined in step 802 that the calculation result of the position information is indefinite, or if it is determined in step 803 that the calculation accuracy of the position information is insufficient, it is decided to re-execute the detailed measurement as an additional operation (step 805). In re-executing the detailed measurement in this step, measurement data may be obtained from the same measurement position and under the same conditions as the previous detailed measurement operation, or measurement data may be obtained by changing the measurement position or measurement conditions.

[0214] Next, if there is a request to obtain detailed position information in step 804, an additional operation is determined to be to perform a measurement from a position closer to the target object than the previous measurement position and obtain proximity measurement data (including proximity image data, etc.), or to obtain measurement data (including image data) with a higher resolution than the previous measurement data (step 806).

[0215] Next, if there is no request for detailed position information in step 804, the process step to transition to is determined depending on whether or not there is a request from the user for communication with the object 7000 (step 807). In this step, the request for communication is acquired from the user interface unit 2800 or the external cooperative system 5000. The request for communication is a request signal from the unmanned boat 1000 to the object via wireless communication or light emission, or a call or warning by voice or display device. If there is a request for communication in this step, the process transitions to step 809, and if there is no request for communication, the process transitions to step 808.

[0216] Next, if there is no request for communication or the like in step 807, it is determined that no additional action is required (step 808).

[0217] Next, if there is a request for communication etc. in step 809, depending on the content of the request for communication etc., an additional action is determined to be communication by wireless communication or light emission from the unmanned boat 1000 etc. to the target object, or an action of calling out or issuing a warning by voice or display device (step 809).

[0218] (A-1-13. Object Analysis Processing) Next, an explanation will be given of the analysis process of the object by the object analysis unit 2700. Fig. 26 is a flowchart showing an example of the flow of the object analysis process by the object analysis unit 2700. Fig. 26 shows a detailed process flow of step 111 in the flowchart shown in Fig. 13, and in particular shows the process flow when the object 7000 is a ship.

[0219] First, the position information of the object 7000 (ship) calculated by the detailed position determination unit 2500 is acquired (step 901).

[0220] Next, the measurement data acquired from the unmanned boat 1000 etc. is analyzed (step 902). In this step, for example, an analysis is performed for each determination item shown in Fig. 8 based on the measurement data.

[0221] Next, based on the position information of the object 7000 (ship) obtained in step 901, a correspondence is determined between the object 7000 (ship) detected by the measurement data and the AIS information of the ship obtained from the AIS control center of the external system 6000, and the results of the determination made by the analysis process in step 902 are compared with the contents of the AIS information (step 903).

[0222] Next, the analysis result of the object 7000 (ship) obtained in step 902 is compared with pre-recorded database information on the characteristics of suspicious ships, etc. (step 904). Here, the database information on the characteristics of suspicious ships, etc. may include the sound signature, hull number, ship silhouette image data, etc. of the suspicious ship.

[0223] Next, the behavior of the object 7000 (ship) in response to additional actions such as communication, calling, and warning performed by the unmanned boat 1000 or the like in accordance with the additional action command generated by the additional action control unit 2600 is analyzed (step 905). Examples of the behavior of the object 7000 (ship) in response to additional actions such as communication, calling, and warning include fleeing, not responding, or anchoring and responding.

[0224] Next, a definitive judgment is made for each judgment item of the object 7000 (ship) according to the results of each process in steps 901 to 905 (step 906). In this step, for example, a definitive judgment can be made for a plurality of judgment items shown in Fig. 8, and a definitive judgment is made as to whether or not the ship is a suspicious ship to be monitored, the type of the object, etc.

[0225] In this step 906, for example, if there are any mismatches among the judgment items shown in Figure 8 based on the comparison results between the judgment results obtained by the analysis processing of the measurement data in step 903 and the contents of the AIS information, the object 7000 can be judged to be a suspicious ship.

[0226] Furthermore, in step 906, for example, based on the results of comparing the analysis of the object 7000 (ship) in the measurement data in step 904 with pre-recorded database information regarding the characteristics of suspicious ships, etc., if the analysis results of the object 7000 (ship) (the judgment results of the judgment items shown in Figure 8) match the characteristics of a suspicious ship in the database information, the object 7000 can be judged to be a suspicious ship.

[0227] In addition, in step 906, for example, if fleeing behavior or no reaction is detected as the behavior of the object 7000 (ship) in response to additional actions such as communication, calling, and warning in step 905, the object 7000 can be determined to be a suspicious ship.

[0228] The above-described embodiment is merely an example for facilitating understanding of the present invention, and is not intended to limit the present invention. The present invention can be modified or improved without departing from the spirit of the present invention, and it goes without saying that the present invention includes equivalents thereof.

[0229] [A-2. Effects of this embodiment] According to the above-described embodiment, it is possible to provide a system or a control method for more accurately determining the position of an object or more accurately analyzing the object when monitoring or investigating an object on water using a drone. As an example, it is possible to more accurately calculate the position of the object and more accurately analyze the object by measuring the object from different positions using a moving body. [Explanation of symbols]

[0230] 1...Control system (system) 100...input device 200...output device 300... Processing device 400... Main storage device 500... Auxiliary storage device 600... Communication device 700…Bus 1000...Unmanned boat 1001... Parent unit 1002... Child unit 10021: Primary connection slave unit 10022: Secondary connection slave unit 10023...Tertiary connection handset 1010...Platoon 1100...Measuring unit 1110...Measuring sensor 1120: Measurement control unit 1200: Aircraft state determination unit 1210: Navigation state determination unit 1220: Internal state determination unit 1230: External state determination unit 1300...Navigation Department 1400…Communication section 1410…Unmanned vessel communication section 1420: Satellite communication unit 1430: External communication unit 1500: Determination unit 1510: Object detection determination unit 1600: Recording section 1610: Measurement data recording section 1620: Own aircraft status recording section 1630: Judgment information recording section 1700: Other action execution section 1710: Communication section 1720…Audio output section 2000…Comprehensive control system 2100: Information import unit 2110: Judgment condition acquisition unit 2120: Measurement data acquisition unit 2130: External information acquisition unit 2140…External intervention information acquisition department 2200…Detection and judgment unit 2300...Position calculation method determining unit 2400: Detailed measurement operation control unit 2410: Measurement device placement determination unit 2420: Measurement aircraft selection unit 2430: Aircraft movement plan determination unit 2440…Detailed measurement operation command unit 2500: Detailed position determination unit 2510: Object position calculation unit 2520: Calculated position suitability determination unit 2600: Additional operation control unit 2610: Additional operation determination unit 2620...Additional operation command section 2700…Object Analysis Department 2800...User interface section 2810: Display unit 2820: User input reception unit 2910…Recording Department 2920…Information and Communication Department 3000...Communication satellite 4000...Ground base station 5000... Collaborative system 6000... External system 7000…Object 8100…Flying object 8200...submersible

Claims

1. a group of moving bodies including a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data of an object at least a part of which is above water; A moving body operation control unit that controls the operation of the moving body; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination unit that determines position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside a determination result of whether the object corresponds to a predetermined monitoring target, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is located, the moving state, stationary state, moving direction, moving speed, past moving route history, or future predicted route of the object, based on the position information.

2. 2. The information control system according to claim 1, The object position determination unit calculates the position information of the object based on information on the angle between the object and the second position as seen from the first position obtained from the first measurement data, the angle between the object and the first position as seen from the second position obtained from the second measurement data, and the relative distance between the first position and the second position.

3. 3. The information control system according to claim 2, the measurement data acquisition unit acquires third measurement data obtained by measuring the object from a third position; The object position determination unit is calculating the position information of the object based on information on a relative angle between the object and the first position as viewed from the third position obtained from the third measurement data, a relative angle between the object and the third position as viewed from the first position obtained from the first measurement data, and a relative distance between the first position and the third position; An information control system that compares the calculated multiple pieces of position information of the object to determine the position information of the object.

4. 3. The information control system according to claim 2, the measurement data acquisition unit acquires third measurement data obtained by measuring the object from a third position; The object position determination unit is calculating the position information of the object based on information on a relative angle between the object and the first position as viewed from the third position obtained from the third measurement data, a relative angle between the object and the third position as viewed from the first position obtained from the first measurement data, and a relative distance between the first position and the third position; calculating the position information of the object based on information on a relative angle between the object and the second position as viewed from the third position obtained from the third measurement data, a relative angle between the object and the third position as viewed from the second position obtained from the second measurement data, and a relative distance between the second position and the third position; An information control system that compares the calculated multiple pieces of position information of the object to determine the position information of the object.

5. 2. The information control system according to claim 1, The measurement sensor includes a stereo camera, a monocular camera, a laser distance measuring sensor, a radar distance measuring sensor, an acoustic distance measuring sensor, an optical distance measuring sensor, or any other sensor capable of measuring distance; An information control system, wherein the object position determination unit determines the position information of the object based on the first measurement data acquired by the measurement sensor from the first position, the position coordinates of the first position, the second measurement data acquired by the measurement sensor from the second position, and the position coordinates of the second position.

6. 6. The information control system according to claim 5, The object position determination unit determines the position information of the object based on the first measurement data, the position coordinates of the first position, the second measurement data, the position coordinates of the second position, third measurement data measured by the measurement sensor from a third position, and the position coordinates of the third position.

7. 2. The information control system according to claim 1, the object position determination unit determines, based on the position coordinates of the first position, the relative orientation or absolute orientation of the object seen from the first position obtained from the first measurement data, the position coordinates of the second position, and the relative orientation or absolute orientation of the object seen from the second position obtained from the second measurement data, An information control system that determines the position information of the object by calculating position coordinates of a geometric intersection of a line passing through the object and the first position and a line passing through the object and the second position.

8. In the information control system according to claim 7, the measurement data acquisition unit acquires third measurement data obtained by measuring the object from a third position; the object position determination unit determines, based on the position coordinates of the first position, the relative orientation or absolute orientation of the object seen from the first position, the position coordinates of the second position, the relative orientation or absolute orientation of the object seen from the second position, the position coordinates of the third position, and the relative orientation or absolute orientation of the object seen from the third position obtained from the third measurement data, An information control system that determines the position information of the object by calculating position coordinates of a geometric intersection of a line passing through the object and the first position, a line passing through the object and the second position, and a line passing through the object and the third position.

9. 2. The information control system according to claim 1, The group of moving bodies is composed of a plurality of unmanned boats capable of navigating on water, An information control system, wherein the first measurement data and the second measurement data are acquired by the measurement sensor provided on the unmanned craft.

10. 10. The information control system according to claim 9, the plurality of unmanned craft include at least a first unmanned craft and a second unmanned craft; the first measurement data is acquired at the first position by the measurement sensor provided on the first unmanned craft; An information control system, wherein the second measurement data is acquired at the second position by the measurement sensor provided on the second unmanned craft.

11. 10. The information control system according to claim 9, the plurality of unmanned craft include at least a first unmanned craft; the first measurement data is acquired at the first position by the measurement sensor provided on the first unmanned craft; An information control system, wherein the second measurement data is acquired at the second position by the measurement sensor provided on the first unmanned craft that has moved from the first position to the second position.

12. 2. The information control system according to claim 1, The group of moving objects includes an unmanned boat capable of navigating on water and an aircraft capable of flying in the sky, the first measurement data is acquired by the measurement sensor provided on the unmanned watercraft; An information control system, wherein the second measurement data is acquired by the measurement sensor provided on the aircraft.

13. 2. The information control system according to claim 1, The group of moving bodies includes an unmanned craft capable of navigating on the surface of water and a submarine craft capable of navigating underwater, the first measurement data is acquired by the measurement sensor provided on the unmanned watercraft; An information control system, wherein the second measurement data is acquired by the measurement sensor provided on the submersible.

14. 2. The information control system according to claim 1, The group of moving objects includes an aircraft capable of flying in the sky and a submarine capable of navigating underwater, the first measurement data is acquired by the measurement sensor provided on the aircraft; the second measurement data is acquired by the measurement sensor provided in the submersible; Information and control systems.

15. 2. The information control system according to claim 1, triangulation to calculate a position of the object using a relative angle between the object and the second position as viewed from the first position and a relative angle between the object and the first position as viewed from the second position; A single survey to acquire a relative distance between the object and one of the moving bodies based on measurement data acquired from the measurement sensor provided on one of the moving bodies; A geometric calculation for calculating a position coordinate of a geometric intersection point between a straight line passing through the object and the first position and a straight line passing through the object and the second position; An information control system comprising a position calculation method determination unit that determines a method for calculating the position information of the object from any one of these combinations of methods.

16. 2. The information control system according to claim 1, An information control system comprising a recording unit that records at least any one of the position information of the object determined by the object position determination unit, the determination result, the measurement data obtained by measuring the object, and the operation history of the moving body in association with identification information of the object.

17. A group of moving bodies including a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data obtained by measuring an object with at least a portion thereof above water; A moving body operation control unit that controls the operation of the moving body; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination unit that determines position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside a determination result of whether the object corresponds to a predetermined monitoring target or not, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is present, the moving state, stationary state, moving direction, moving speed, history of past moving routes, or a predicted future route of the object, based on the position information; triangulation to calculate a position of the object using a relative angle between the object and the second position as viewed from the first position and a relative angle between the object and the first position as viewed from the second position; A single survey to acquire a relative distance between the object and one of the moving bodies based on measurement data acquired from the measurement sensor provided on one of the moving bodies; A geometric calculation for calculating a position coordinate of a geometric intersection point between a straight line passing through the object and the first position and a straight line passing through the object and the second position; The present invention further includes a position calculation method determination unit that determines a method for calculating the position information of the object from any one of the combinations of these methods, An information control system, wherein the position calculation method determination unit determines a method for calculating the position information of the object based on at least any of the following information: the measurement data obtained from the measurement sensor provided on the moving body, information regarding the calculation accuracy of the moving body's own position, information regarding the calculation accuracy of the relative distance between at least two of the moving body group, and information regarding the calculation accuracy of the relative orientation or absolute orientation of the moving body.

18. A group of moving bodies including a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data obtained by measuring an object with at least a portion thereof above water; A moving body operation control unit that controls the operation of the moving body; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination unit that determines position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside a determination result of whether the object corresponds to a predetermined monitoring target or not, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is present, the moving state, stationary state, moving direction, moving speed, history of past moving routes, or a predicted future route of the object, based on the position information; triangulation to calculate a position of the object using a relative angle between the object and the second position as viewed from the first position and a relative angle between the object and the first position as viewed from the second position; A single survey to acquire a relative distance between the object and one of the moving bodies based on measurement data acquired from the measurement sensor provided on one of the moving bodies; A geometric calculation for calculating a position coordinate of a geometric intersection point between a straight line passing through the object and the first position and a straight line passing through the object and the second position; The present invention further includes a position calculation method determination unit that determines a method for calculating the position information of the object from any one of the combinations of these methods, When the measurement data acquired from the measurement sensor provided on the moving body is image data acquired by an optical camera, The position calculation method determination unit determines a method for calculating the position information of the object from either the triangulation or the geometric calculation when it detects that the exposure of the image data is greater than an upper tolerance, the amount of light of the image data is less than a lower tolerance, or the field of view of the image data is shorter than a lower limit distance.

19. A group of moving bodies including a plurality of moving bodies each equipped with a measurement sensor capable of acquiring measurement data obtained by measuring an object with at least a portion thereof above water; A moving body operation control unit that controls the operation of the moving body; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination unit that determines position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside a determination result of whether the object corresponds to a predetermined monitoring target or not, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is present, the moving state, stationary state, moving direction, moving speed, history of past moving routes, or a predicted future route of the object, based on the position information; The mobile object operation control unit determines the first position and the second position so as to satisfy at least one of the following conditions: a relative distance from the object to the first position and a relative distance from the object to the second position are within a predetermined distance, a relative distance between the first position and the second position is within a predetermined distance, and an angle between the first position and the second position as seen from the object is within a predetermined angle range.

20. 20. The information control system according to claim 19, The mobile body operation control unit determines, based on the first position and the second position, a first mobile body to be moved to the first position and instructed to acquire the measurement data, and a second mobile body to be moved to the second position and instructed to acquire the measurement data.

21. A group of moving bodies including a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data obtained by measuring an object with at least a portion thereof above water; A moving body operation control unit that controls the operation of the moving body; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination unit that determines position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside a determination result of whether the object corresponds to a predetermined monitoring target or not, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is present, the moving state, stationary state, moving direction, moving speed, history of past moving routes, or a predicted future route of the object, based on the position information; The mobile object operation control unit instructs a first mobile object and a second mobile object that are present at the first position and the second position, respectively, to acquire the measurement data when the relative distance from the object to the first position and the relative distance from the object to the second position are within a predetermined distance, the relative distance between the first position and the second position is within a predetermined distance, and the angle between the first position and the second position as seen from the object is within a predetermined angle range.

22. A group of moving bodies including a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data obtained by measuring an object with at least a portion thereof above water; A moving body operation control unit that controls the operation of the moving body; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination unit that determines position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside a determination result of whether the object corresponds to a predetermined monitoring target or not, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is present, the moving state, stationary state, moving direction, moving speed, history of past moving routes, or a predicted future route of the object, based on the position information; If the object is determined to be a ship, An information control system that, based on the position information of the object determined by the object position determination unit, compares information about the object determined from the measurement data with information about the object obtained from the object via wireless communication by an external automatic ship identification system, and determines whether the object corresponds to a predetermined monitoring target based on the result of the comparison.

23. A group of moving bodies including a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data obtained by measuring an object with at least a portion thereof above water; A moving body operation control unit that controls the operation of the moving body; a measurement data acquisition unit that acquires first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination unit that determines position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data, or determines the validity of the determined position information; an information output unit that displays or transmits to the outside the position information of the object determined by the object position determination unit, or displays or transmits to the outside a determination result of whether the object corresponds to a predetermined monitoring target or not, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is present, the moving state, stationary state, moving direction, moving speed, history of past moving routes, or a predicted future route of the object, based on the position information; An information control system in which, when the mobile object operation control unit communicates with the target object via wireless communication or light emission, or issues a call or warning via voice or display device, it determines whether the target object corresponds to a predetermined monitoring target based on the target object's behavior in response to the communication, call, or warning.

24. A method for controlling a system for measuring an object using a group of moving bodies, the group being made up of a plurality of moving bodies each having a measurement sensor capable of acquiring measurement data obtained by measuring the object when at least a portion of the object is above water, comprising the steps of: The computer a moving object motion control step of controlling a motion of the moving object; a measurement data acquisition step of acquiring first measurement data obtained by measuring the object from a first position by at least one of the moving bodies of the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination step of determining position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data; an information output step of displaying or transmitting to the outside the position information of the object determined by the object position determination step, or displaying or transmitting to the outside a determination result of whether or not the object corresponds to a predetermined monitoring target, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is present, the moving state, stationary state, moving direction, moving speed, history of past moving routes, or predicted future routes of the object, which are determined based on the position information; A control method for performing the above.

25. A program usable in a system for measuring an object using a group of moving objects, the group being made up of a plurality of moving objects each having a measurement sensor capable of acquiring measurement data obtained by measuring the object when at least a portion of the object is above water, the program comprising: On the computer, A moving object operation control command for controlling an operation of the moving object; a measurement data acquisition command for acquiring first measurement data obtained by measuring the object from a first position by at least one of the moving bodies in the moving body group, and second measurement data obtained by measuring the object from a second position by the moving body or another moving body; an object position determination command for determining position information including at least one of a relative position coordinate, an absolute position coordinate, a relative distance, a relative orientation, and an absolute orientation of the object based on the first measurement data and the second measurement data; an information output command for displaying or transmitting to the outside the position information of the object determined by the object position determination command, or for displaying or transmitting to the outside a determination result of whether or not the object corresponds to a predetermined monitoring target, the type, shape, size, or orientation of the object, the relative distance, direction, or position coordinates of the position where the object is present, the moving state, stationary state, moving direction, moving speed, past moving route history, or future predicted route of the object, which is determined based on the position information; A program that executes the following.

Images