Operation system, control method and program

Abstract

According to the present invention, it is possible to make the system exhibit the performance required for operation while suppressing costs or minimizing the impact on economic activity in the vicinity of the site. [Solution] The present invention is an operation system comprising: a real aircraft behavior information acquisition unit that acquires real aircraft behavior information from a real aircraft; a simulated real aircraft that simulates a real aircraft in a virtual space based on the real aircraft behavior information; an unmanned aircraft group simulator that calculates the behavior state of a mixed unmanned aircraft group that is a mixture of unmanned aircraft and virtual aircraft generated in the virtual space; a control command determination unit that generates control commands for at least the real aircraft, etc.; a display unit that displays and outputs the behavior state of the generated mixed unmanned aircraft group; and a real aircraft increase / decrease determination unit that determines whether to increase the number of real aircraft deployed in the operation area by replacing at least one of the virtual aircraft with a real aircraft, or to decrease the number of real aircraft deployed in the operation area by replacing at least one of the real aircraft with a virtual aircraft.

Description

[Technical Field]

[0001] The present invention relates to an operation system, a control method, and a program. [Background technology]

[0002] In recent years, there has been consideration of using unmanned aircraft systems, including multiple unmanned vessels (hereinafter also referred to as "unmanned boats"), unmanned aircraft, unmanned vehicles, etc., for specific purposes in specified operational areas (marine areas, undersea areas, air areas, land areas, etc.), such as collecting environmental data, providing a communication environment, searching for suspicious vessels, etc., and inspecting and patrolling infrastructure facilities, etc. When such unmanned aircraft systems are to be used for the purposes described above, they are required to demonstrate the performance required for those purposes during actual operation.

[0003] Patent Document 1 discloses a technology for suppressing the effects of communication delays and information loss during communication when a mobile object is remotely controlled by a remote operator. Furthermore, the patent document discloses a remote control system including a mobile object to be remotely controlled by the remote operator and a remote operator terminal on the remote operator side, the remote operator terminal including a simulator that performs a real-time simulation of a group of objects including the mobile object in a predetermined space and is configured to display the results of the simulation, and in parallel with the execution of the simulation, the remote operator terminal receives object information indicating at least the positions of the group of objects and obtains corrected object information by compensating for delays in the received object information, and further discloses that the remote operator terminal corrects simulation object information, which is object information within the simulation, based on the corrected object information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-169715 Summary of the Invention [Problem to be solved by the invention]

[0005] However, as mentioned above, when operating multiple unmanned aircraft in this operation, in addition to the numerous unmanned aircraft to be deployed in the operation area, material resources are required, including replacement aircraft in case of abnormalities or breakdowns, replacement batteries and other consumable parts, etc. Furthermore, various human resources are required to operate the unmanned aircraft system (transportation and deployment to the site, control operation for mission execution, inspection, maintenance, and management of the aircraft), resulting in enormous costs for material and human resources. Furthermore, deploying a large number of unmanned aircraft (unmanned boats, etc.) in the operation area has the potential to interfere with the path of general ships navigating within and around the operation area, and if the entry of general ships into the operation area is restricted to avoid the risk of collision, there is a problem of impacting private economic activity.

[0006] Here, for example, when searching for an object in an operation area, the search performance required of the unmanned aerial vehicle system changes depending on the situation, such as the occurrence of signs of the object's appearance, past trends in the appearance of the object, etc. Therefore, it is desirable to deploy a relatively small number of aircraft to the site during peacetime when high search performance is not necessarily required, thereby reducing the costs required for material and human resources and minimizing the impact on surrounding economic activity, while, in situations where high search performance is required, increasing the number of aircraft deployed to the operation area to improve the search performance of the unmanned aerial vehicle system.

[0007] However, in situations where there are signs of an object appearing, it is necessary to quickly increase the number of drones and improve search performance, but in order to achieve the required search performance, it was difficult to quickly determine the operational requirements for the drone fleet, such as how many drones need to be deployed to which locations.In addition, there was room for consideration of a control method for deploying drones in order to quickly increase the number of drones.

[0008] Therefore, the present invention has been made in consideration of at least one of the above problems, and one object of the present invention is to provide a system or control method, etc., that can demonstrate the performance required for operation while reducing costs associated with physical and human resources or minimizing the impact on economic activity in the surrounding area of ​​the site. [Means for solving the problem]

[0009] According to the present invention, an operation system is obtained which comprises: a real aircraft behavior information acquisition unit which acquires real aircraft behavior information from a real aircraft; a simulated real aircraft which simulates a real aircraft in a virtual space based on the real aircraft behavior information; an unmanned aircraft group simulator which calculates the behavior state of a mixed unmanned aircraft group which is a mixture of unmanned aircraft and virtual aircraft generated in the virtual space; a control command determination unit which generates control commands for at least the real aircraft, etc.; a display unit which displays and outputs the behavior state of the generated mixed unmanned aircraft group; and a real aircraft increase / decrease determination unit which determines to increase the number of real aircraft deployed in the operation area by replacing at least one of the virtual aircraft with a real aircraft, or to decrease the number of real aircraft deployed in the operation area by replacing at least one of the real aircraft with a virtual aircraft. [Effects of the Invention]

[0010] According to the present invention, it is possible to achieve the performance required for operation while reducing costs associated with physical and human resources and minimizing the impact on economic activity in the vicinity of the site. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating the overall configuration of an operation system 1 according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram showing an example of an implementation image of an operation system 1 in a real space. [Figure 3] FIG. 10 is a diagram showing an example of a formation of a group made up of multiple unmanned boats 1010. [Figure 4] FIG. 2 is a functional block diagram showing the functional configuration of an unmanned watercraft 1010. [Figure 5]FIG. 2 is a diagram illustrating an example of a state transition of the operation system 1. [Figure 6] FIG. 2 is a diagram showing an example of stakeholders related to the operation system 1. [Figure 7] FIG. 10 is a diagram showing an example of the operation of the integrated environment management system 2000 in a search operation state. [Figure 8] 10 is a diagram showing an example of input / output information of the integrated environment management system 2000. FIG. [Figure 9] 10 is a diagram showing an example of input / output information between multiple functional units of the integrated environment management system 2000. FIG. [Figure 10] FIG. 2 is a functional block diagram showing the functional configuration of an integrated environment management system 2000. [Figure 11] FIG. 3 is a functional block diagram showing the functional configuration of an operational application control system 3000. [Figure 12] 10 is a diagram showing an example of user request information acquired by a user request information acquisition unit 2110. FIG. [Figure 13] 10 is a diagram showing an example of assumed condition parameters acquired by a simulation condition acquisition unit 2120. FIG. [Figure 14] FIG. 10 is a diagram showing an example of detailed state transitions in a search state determined by the control state transition determination unit 2630. [Figure 15] A figure showing a first example of display information of current behavior information of a mixed unmanned aerial vehicle group generated by the display information generation unit 3210. [Figure 16] A figure showing a second example of display information of current behavior information of a mixed unmanned aerial vehicle group generated by the display information generation unit 3210. [Figure 17] A figure showing a third example of display information of current behavior information of a mixed unmanned aerial vehicle group generated by the display information generation unit 3210. [Figure 18] 10A and 10B are diagrams showing examples of display switching tabs displayed on the display unit 3310. [Figure 19] FIG. 2 is a diagram showing an example of a higher-level control processing flow of the operation system 1. [Figure 20]FIG. 10 is a diagram showing an example of a control processing flow for acquiring advance information and real-time information by an information acquisition unit 2100. [Figure 21] FIG. 10 is a diagram showing an example of a control processing flow for determining a detection state and an operating state by the state determination unit 2500 and the like. [Figure 22] FIG. 10 is a diagram showing an example of a control processing flow when the group control command determiner 2600 controls the unmanned watercraft. [Figure 23] FIG. 10 is a diagram illustrating an example of a method for controlling replacement from a virtual machine to a real machine by a real machine increase / decrease control command generating unit 2640. [Figure 24] FIG. 20 is a diagram showing an example of a method for calculating control conditions when an actual machine for ensuring communication is additionally deployed by the actual machine increase / decrease control command generating unit 2640. [Figure 25] FIG. 26 is a diagram showing how the control command determination unit 2610 controls the movement of the real machine and the virtual machine so as to maintain the communication maintenance range. [Figure 26] FIG. 26 is a diagram showing how the control command determination unit 2610 controls the movement of the real and virtual ships so as to avoid interference with other ships. [Figure 27] FIG. 26 is a diagram showing how the control command determination unit 2610 controls the movement of the real machine and the virtual machine so as to avoid the wave area. [Figure 28] FIG. 20 is a diagram showing a first mode in which the actual machine increase / decrease control command generating unit 2640 performs actual machine addition control using an actual machine for ensuring communication. [Figure 29] FIG. 20 is a diagram showing a second mode in which the actual machine increase / decrease control command generating unit 2640 performs actual machine addition control using an actual machine for ensuring communication. [Figure 30] FIG. 10 is a diagram showing an example of an implementation in which the functional units of the converged environment management system 2000 and the operational application control system 3000 are implemented on a cloud server, a workstation, or the like. [Figure 31] FIG. 1 is a diagram illustrating an example of an implementation of a HILS connected to a virtual machine simulator. [Figure 32] FIG. 2 is a diagram illustrating an example of a hardware configuration diagram of a converged environment management system 2000 and an operational application control system 3000. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present invention will be described below with reference to the following embodiments. [Item 1] An operation system for controlling the operation of multiple unmanned aerial vehicles deployed in an operation area, an actual aircraft behavior information acquisition unit that acquires actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; an unmanned aircraft group simulator that calculates the behavior state of a mixed unmanned aircraft group that includes a simulated real aircraft that simulates the real aircraft in a virtual space based on the real aircraft behavior information and a virtual aircraft of the unmanned aircraft generated in the virtual space; a control command determination unit that generates a control command for the unmanned aerial vehicle including at least the real vehicle, the simulated real vehicle, or the virtual vehicle; A display unit that displays and outputs the behavior state of the mixed unmanned aerial vehicle group generated by the unmanned aerial vehicle group simulator; a real machine increase / decrease determination unit that determines whether to increase the number of the real machines deployed in the operational area by replacing at least one of the virtual machines with the real machine, or whether to decrease the number of the real machines deployed in the operational area by replacing at least one of the real machines with the virtual machine; An operating system comprising: [Item 2] In the operation system according to item 1, When the actual aircraft increase / decrease decision unit determines to increase the number of the actual aircraft deployed in the operation area, The control command determination unit generates an additional real machine control command for controlling movement of an additional real machine to replace the virtual machine, with the position of the virtual machine or a periphery of that position set as a movement target position. [Item 3] In the operation system according to item 1 or 2, When the movement of the additional real machine to replace the virtual machine to the position of the virtual machine or the vicinity of the position is completed in accordance with the additional real machine control command, The unmanned aerial vehicle simulator is an operation system that replaces the virtual vehicle in the virtual space with a simulated real vehicle that simulates the additional real vehicle. [Item 4] In the operation system according to any one of items 1 to 3, When the unmanned aerial vehicle simulator replaces the virtual vehicle in the virtual space with the simulated real vehicle, The behavior information of the simulated real machine includes at least one of information on a position, an orientation, a moving speed, a moving direction, a moving acceleration, a rotational angular velocity, a battery charge state, and a battery deterioration state; An operation system in which the behavior information of the simulated real aircraft is updated by the real aircraft behavior information acquired by the real aircraft behavior information acquisition unit, or is updated by the behavior state of the simulated real aircraft calculated by the unmanned aircraft group simulator based on the real aircraft behavior information. [Item 5] In the operation system according to any one of items 1 to 4, When the actual aircraft increase / decrease decision unit determines to reduce the number of the actual aircraft deployed in the operational area, The control command determination unit generates a recovered real machine control command for recovering the recovered real machine to be replaced with the virtual machine or moving the recovered real machine to a waiting area. [Item 6] In the operation system according to any one of items 1 to 5, When the actual aircraft increase / decrease decision unit determines to reduce the number of the actual aircraft deployed in the operational area, The unmanned aerial vehicle swarm simulator calculates the behavior state of the virtual vehicle to be replaced with the recovered real vehicle in the virtual space independently from the real vehicle behavior information of the recovered real vehicle. [Item 7] In the operation system according to any one of items 1 to 6, The actual machine increase / decrease decision unit decides to increase the number of actual machines when at least one of the following conditions is met: alert information is received from an external system; a predetermined time period has arrived; or a command requesting an increase in actual machines is received from a user. [Item 8] In the operation system according to any one of items 1 to 7, When searching for a predetermined object present in the operation area or a surrounding area of ​​the operation area using a measurement sensor mounted on the actual machine, The actual aircraft increase / decrease decision unit decides to increase the number of actual aircraft when at least one of the following occurs: the target object or a candidate object for the target object is detected using measurement data from the measurement sensor, or a value indicating the unmanned aircraft's search performance, mobility performance, battery performance, or communication performance has fallen below a predetermined value. [Item 9] In the operation system according to any one of items 1 to 8, The unit for determining whether to increase or decrease the number of actual aircraft decides to reduce the number of actual aircraft when at least one of the following conditions is met: alert information previously acquired from an external system has been lifted; an instruction to reduce the number of actual aircraft has been acquired from a user; or a detection flag indicating that an object or a candidate object for the object has been detected using measurement data from a measurement sensor mounted on the unmanned craft has been lifted. [Item 10] In the operation system according to any one of items 1 to 9, The actual aircraft increase / decrease decision unit decides to reduce the number of actual aircraft when at least one of the following occurs: the wave conditions in the operation area have deteriorated beyond a predetermined value, or it is predicted that future wave conditions in the operation area will deteriorate beyond a predetermined value. [Item 11] In the operation system according to any one of items 1 to 10, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, The actual aircraft increase / decrease decision unit determines the future relative distance between the other moving body and the unmanned aircraft based on current movement information or future planned movement information of other moving bodies obtained from an external system and the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator, and decides to reduce the number of actual aircraft if it determines that an interference state in which the relative distance will be less than a predetermined distance will occur in the future. [Item 12] In the operation system according to any one of items 1 to 11, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control commands for the drones, including the real drones and the virtual drones, so as to prevent an interference state from occurring in which the future relative distance between the drones in the mixed unmanned drone group, determined based on the future behavior state of the mixed unmanned drone group generated by the unmanned drone group simulator, is less than a predetermined collision avoidance safe distance. [Item 13] In the operation system according to any one of items 1 to 12, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control command for the unmanned aircraft, including the real aircraft and the virtual aircraft, so as to prevent an interference state from occurring in which the future relative distance between the other moving body and the unmanned aircraft, determined based on the current movement information or future movement schedule information of the other moving body obtained from an external system and the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator, is less than a predetermined distance. [Item 14] In the operation system according to any one of items 1 to 13, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control command for the unmanned aircraft, including the actual aircraft and the virtual aircraft, so as to prevent a wave area navigation state from occurring in which the relative distance on a horizontal plane between the typhoon, low pressure, rain cloud, or wave area and the unmanned aircraft is less than a predetermined distance, as determined based on current or future behavior information of the typhoon, low pressure, rain cloud, or wave area obtained from an external system and the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator. [Item 15] In the operation system according to any one of items 1 to 14, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control commands for the unmanned aircraft, including the real aircraft and the virtual aircraft, so as to prevent a communication failure state from occurring in which the future relative distance between the unmanned aircraft of the mixed unmanned aircraft group, determined based on the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator, exceeds a predetermined wireless communication distance between the unmanned aircraft. [Item 16] In the operation system according to any one of items 1 to 15, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control commands for the unmanned aircraft, including the actual aircraft and the virtual aircraft, so that the operational state of the mixed unmanned aircraft group determined based on the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator achieves a predetermined target for the operational state, or so that the determined operational state improves. [Item 17] In the operation system according to any one of items 1 to 16, When the target position of the additional real aircraft included in the additional real aircraft control command is outside the wireless communication range of the real aircraft already deployed in the operation area in the real space, The control command determination unit generates a second additional real machine control command to deploy a second additional real machine that relays wireless communication between the real machine and the additional real machine, with a second movement target position being a position that is within the wireless communication range from the real machine and also within the wireless communication range from the additional real machine. [Item 18] In the operation system according to any one of items 1 to 17, The control command determination unit after moving the second additional actual machine to the second movement target position in response to the second additional actual machine control command, moving the additional actual machine to the movement target position in response to the additional actual machine control command; Alternatively, an operation of moving the second additional actual machine to the second movement target position in accordance with the second additional actual machine control command and an operation of moving the additional actual machine to the movement target position in accordance with the additional actual machine control command are performed in parallel. [Item 19] In the operation system according to any one of items 1 to 18, during a period in which the additional real machine is moved to the position of the virtual machine or a peripheral position thereof in response to the additional real machine control command, the control command determination unit generates a control command to stop or almost stop the virtual machine; When the movement of the additional real aircraft to the position of the virtual aircraft or a peripheral position thereof is completed in response to the additional real aircraft control command, the unmanned aircraft swarm simulator rewrites the behavior state of the virtual aircraft in the virtual space with the real aircraft behavior information of the additional real aircraft that will replace the virtual aircraft. [Item 20] In the operation system according to any one of items 1 to 19, The additional real machine control command is a control command for matching the position, orientation, and speed of the additional real machine with the position, orientation, and speed of the virtual machine, and when the position, orientation, and speed of the virtual machine match or approximately match the position, orientation, and speed of the additional real machine, The unmanned aerial vehicle swarm simulator rewrites the behavior state of the virtual vehicle in the virtual space with the real vehicle behavior information of the additional real vehicle that replaces the virtual vehicle. [Item 21] In the operation system according to any one of items 1 to 20, during a movement period in which the additional real machine is moved to the position of the virtual machine or a peripheral position thereof in response to the additional real machine control command, the control command determination unit generates a control command to move the virtual machine in a predetermined movement pattern; The additional real aircraft control command is a control command to match the position, orientation, and speed of the virtual aircraft with the position, orientation, and speed of the additional real aircraft, and when the position, orientation, and speed of the virtual aircraft match or approximately match the position, orientation, and speed of the additional real aircraft, the unmanned aircraft swarm simulator rewrites the behavior state of the virtual aircraft in the virtual space with the real aircraft behavior information of the additional real aircraft that will replace the virtual aircraft. [Item 22] In the operation system according to any one of items 1 to 21, a state determination unit that determines whether an interference state will occur in which the current or future relative distance between the drones in the mixed unmanned aircraft group, or the relative distance between another moving object and the drone, will be equal to or less than a predetermined distance, based on the current or future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator; The display unit displays the determination result of whether or not the interference state has occurred. [Item 23] In the operation system according to any one of items 1 to 22, a state determination unit that determines whether a communication failure state will occur in which the current or future relative distance between the drones in the mixed unmanned aircraft group will be equal to or greater than a predetermined wireless communication distance between the drones, based on the current or future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator; The display unit displays the determination result of whether or not the communication failure state has occurred. [Item 24] In the operation system according to any one of items 1 to 23, a state determination unit that determines whether the current or future operational state of the mixed unmanned aerial vehicle group will achieve a preset goal related to the operational state based on the current or future behavior state of the mixed unmanned aerial vehicle group generated by the unmanned aerial vehicle group simulator; The display unit displays a determination result of whether the operational status achieves the preset target. [Item 25] In the operation system according to any one of items 1 to 24, The display unit displays and outputs the actual machine behavior information acquired from the actual machine or the actual machine measurement data measured by a measurement sensor mounted on the actual machine. [Item 26] A control method for controlling the operation of multiple unmanned aerial vehicles deployed in an operation area, comprising: The computer an actual aircraft behavior information acquisition step of acquiring actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; an unmanned aircraft group behavior calculation step for calculating the behavior state of a mixed unmanned aircraft group including a simulated real aircraft that simulates the real aircraft in a virtual space based on the real aircraft behavior information and a virtual aircraft of the unmanned aircraft generated in the virtual space; a real machine increase / decrease decision step for deciding to increase the number of the real machines deployed in the operational area by replacing at least one of the virtual machines with the real machine, or to decrease the number of the real machines deployed in the operational area by replacing at least one of the real machines with the virtual machine; a control command determination step of generating a control command for the unmanned aerial vehicle including at least the actual vehicle; a display step of displaying and outputting the behavior state of the mixed unmanned aerial vehicle group generated by the unmanned aerial vehicle group behavior calculation step; A control method comprising: [Item 27] A program available for an operation system that controls the operation of multiple unmanned aerial vehicles deployed in an operation area, On the computer, an actual aircraft behavior information acquisition command to acquire actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; an unmanned aircraft group behavior calculation command for calculating the behavior state of a mixed unmanned aircraft group including a simulated real aircraft that simulates the real aircraft in a virtual space based on the real aircraft behavior information and a virtual aircraft of the unmanned aircraft generated in the virtual space; a real machine increase / decrease decision command that decides to increase the number of the real machines deployed in the operational area by replacing at least one of the virtual machines with the real machine, or to decrease the number of the real machines deployed in the operational area by replacing at least one of the real machines with the virtual machine; a control command determination command for generating a control command for the unmanned aerial vehicle including at least the actual vehicle; A display command for displaying and outputting the behavior state of the mixed unmanned aerial vehicle group generated by the unmanned aerial vehicle group behavior calculation command; A program that executes.

[0013] First Embodiment Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, the embodiments described below are merely examples, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc. In addition, in the first embodiment, an example will be described in which an unmanned boat 1010 is used as an example of an unmanned aircraft, but the unmanned aircraft is not limited to this, and the present invention can be applied to any other unmanned mobile object, such as an unmanned aerial vehicle, an unmanned vehicle, or an unmanned submersible.

[0014] [A.Configuration] (A-1. Overall system configuration) First, the system configuration of an operation system 1 (hereinafter also referred to as "system 1") according to one embodiment of the present invention will be described with reference to FIGS.

[0015] (A-1-1. System Configuration Overview) FIG. 1 is an overall configuration diagram of an operation system 1 according to one embodiment of the present invention. As shown in FIG. 1, the operation system 1 includes an unmanned watercraft system 1000, an integrated environment management system 2000, and an operational application control system 3000. The unmanned watercraft system 1000 is equipped with measurement sensors capable of detecting an object 7000 and includes multiple unmanned watercraft 1010 (also referred to as "unmanned vessels") capable of navigating and moving on the sea. The integrated environment management system 2000 is also configured to be able to communicate with an external cooperative system 6000 and an external system 4000 via an internet line or the like, allowing for the input and output of information. The integrated environment management system 2000 can send control commands to the unmanned watercraft system 1000 deployed on the sea via a terrestrial base station 5200 and a communication satellite 5100, and can also receive behavior information and measurement data from the unmanned watercraft system 1000. Therefore, the integrated environment management system 2000 can remotely control the operation of the unmanned watercraft system 1000 or make it navigate autonomously or automatically, and perform operations such as searching for an object 7000 using measurement sensors in a predetermined area on the sea or underwater, collecting marine data, inspecting or patrolling offshore infrastructure, or providing a communication environment to the sea or underwater area using a communication device mounted on the unmanned watercraft 1010. Here, the predetermined area is any area that can be set by user input or can be set in the system in advance.

[0016] The unmanned boat system 1000 includes a single or multiple unmanned boats 1010. When the unmanned boat system 1000 is composed of multiple unmanned boats 1010, the multiple unmanned boats 1010 are connected to each other via wireless communication, allowing the configuration of a communication network. The unmanned boat 1010 also has the function of measuring targets 7000, including marine life such as submarines, underwater drones, divers, and whales moving underwater, ships on the sea, drifting objects, drifters, aircraft in the sky, water vapor, etc., using measurement sensors mounted on the boat (such as sonar or other acoustic sensors, optical cameras, IR cameras, laser sensors such as LiDAR, and radar sensors such as millimeter-wave sensors and microwave sensors).

[0017] The detection determination results and measurement data of the object 7000 detected by the unmanned watercraft system 1000, as well as various types of behavior information for each unmanned watercraft 1010 in the unmanned watercraft system 1000, are transmitted to the integrated environment management system 2000 via the communications satellite 5100 and the terrestrial base station 5200. The integrated environment management system 2000 determines operation commands for the unmanned watercraft system 1000 based on information acquired from the unmanned watercraft system 1000 and request information acquired in advance from users, etc. The generated information, such as the operation commands, is transmitted to a user terminal device 8000 and displayed and output to the user. In addition, intervention commands, such as operation commands, can be acquired from the user via the user terminal device 8000.

[0018] (A-1-2. Example of implementation of Operation System 1 in real space) Fig. 2 is a diagram showing an example of an implementation image of the operation system 1 in real space. In the example shown in Fig. 2, a terrestrial base station 5200 and an integrated environment management system 2000 are provided on the ground side shown in the upper right of the drawing. Also provided on the ground side are a collaborative system 6000, an external system 4000, and a user terminal device 8000, all of which are connected to the integrated environment management system 2000 via a network.

[0019] On the other hand, on the ocean side shown on the left side of the drawing, unmanned boat system 1000 is deployed to search for target object 7000 in the ocean. Unmanned boat system 1000 also has multiple groups (1000a, 1000b, 1000c) each consisting of a parent unit and multiple child units, and can communicate wirelessly between each group either directly or via communication satellite 5100.

[0020] In the example shown in Figure 2, the integrated environmental management system 2000 is shown to be implemented in a facility on land, but this is not limited to this. All or some of the functions implemented in the integrated environmental management system 2000 shown in this embodiment can also be installed on a coastal field base located in a coastal area on land (not shown) or on a manned mother ship at sea, and the unmanned boat system 1000 can be operated and managed at the coastal field base or manned mother ship.

[0021] In the configuration of the embodiment described above with reference to FIGS. 1 and 2, a non-terrestrial network using a communication satellite 5100 in a geosynchronous orbit or a low Earth orbit has been used as a communication network for transmitting and receiving information between the integrated environmental management system 2000 and the unmanned watercraft system 1000. However, the present invention is not limited to this. A non-terrestrial network using an unmanned air vehicle known as a High Altitude Platform Station (HAPS) can also be used. In this case, for example, an unmanned air vehicle that circles at an altitude of approximately 8 to 50 km can be used. Furthermore, a communication network for transmitting and receiving information between the integrated environmental management system 2000 and the unmanned watercraft 1010 can also be used, in which a terrestrial base station 5200 directly connects the unmanned watercraft 1010 via wireless communication, without going through the communication satellite 5100 or the HAPS. The terrestrial base station 5200 is not limited to a fixed base station, and may be a mobile base station.

[0022] (A-2. Unmanned Boat System 1000) Next, the system configuration of an unmanned boat system 1000 according to one embodiment of the present invention will be described with reference to FIGS.

[0023] (A-2-1. Composition of the group consisting of 1010 unmanned boats) Fig. 3 is a diagram showing an example of a formation of a group made up of multiple unmanned craft 1010. The example shown in Fig. 5 shows the formation of a group 1000a made up of multiple unmanned craft 1010 and the communication connection relationships between each unmanned craft 1010 when the unmanned craft system 1000 is made to perform a mission such as searching for an object.

[0024] 3, group 1000a is made up of parent device 1001 capable of wireless communication with communication satellite 5100, and child devices 1002 capable of direct or indirect wireless communication with parent device 1001. Parent device 1001 is connected to communication satellite 5100 by wireless communication, and has the function of aggregating information collected from multiple child devices 1002 and transmitting the information to communication satellite 5100, as well as transmitting information such as operation commands obtained from communication satellite 5100 and information generated by itself to each child device 1002, directly or indirectly.

[0025] 3, the group 1000a includes one parent device 1001 and multiple child devices 1002. The parent device 1001 and the multiple child devices 1002 are connected via wireless communication as indicated by solid lines, forming a wireless communication network among multiple unmanned boats 1010 at sea. The group 1000a shown in FIG. 3 includes a primary connected child device 10021 that is wirelessly connected to the parent device 1001, and a secondary connected child device 10022 that is wirelessly connected to the primary connected child device 10021.

[0026] 3, the slave devices 1002 forming a group are configured to include a primary connected slave device 10021 and a secondary connected slave device 10022, but the configuration of the group is not limited to this and may be configured only with a primary connected slave device 10021 that is directly connected wirelessly to the master device. As another example, the slave devices 1002 forming a group may include a tertiary connected slave device, a quaternary connected slave device, or a higher connected slave device.

[0027] Furthermore, the number of secondary connected slave devices 10022 wirelessly connected to the primary connected slave device 10021 is not limited to one, and by having multiple secondary connected slave devices 10022 wirelessly connected to the primary connected slave device 10021, it is possible to form a tree-structured communication network in which multiple unmanned watercraft 1010 branch out within the group 1000a. Furthermore, because there is an upper limit to the communication distance over which wireless communication is possible between each unmanned watercraft 1010, the position of at least one of the unmanned watercraft 1010 that communicates wirelessly with each other, for example, the parent device 1001 and the primary connected slave device 10021, and the primary connected slave device 10021 and the secondary connected slave device 10022, is controlled so that the relative distance between the unmanned watercraft 1010 is maintained within the upper limit of the relative distance over which wireless communication is possible.

[0028] Furthermore, if the relative distance between the unmanned craft 1010 becomes so great that the unmanned craft 1010 with which the communication partner is communicating moves outside the range of the wireless communication distance, wireless communication between them will no longer be possible and control commands from the integrated environment management system 2000 will no longer be able to be sent. Therefore, it is desirable for two unmanned craft 1010 that are connected to each other for communication to perform self-position control with a higher priority than other controls in order to maintain the relative distance with the communication partner within the range of the communication distance.

[0029] On the other hand, there is no need to maintain the relative distance within a predetermined range between other unmanned watercraft 1010 that do not communicate with each other wirelessly in order to maintain the communication connection described above. On the other hand, if the purpose of the unmanned watercraft system 1000 is to search for the target object 7000, the search can be performed more efficiently if the unmanned watercraft 1010 maintain an appropriate distance between them so that the measurement ranges of their measurement sensors do not overlap or only overlap moderately, rather than if the unmanned watercraft 1010 are too close and the measurement ranges of their measurement sensors overlap to a large extent. Therefore, the relative distance between the unmanned watercraft 1010 that do not communicate with each other is controlled with a relatively low priority, so that a predetermined search relative distance is maintained. This control to maintain the search relative distance can be based on the Boids algorithm, for example.

[0030] Furthermore, if the relative distance between the unmanned craft 1010 becomes too close and there is a possibility of a collision, position control can be performed to increase the relative distance with a relatively high priority in order to avoid a collision and prevent damage to the unmanned craft 1010.

[0031] As described above, control to maintain the relative distance between unmanned vessels 1010 that communicate with each other by wireless communication within the communication distance range, and avoidance control to avoid collision with other unmanned vessels that come within close range, are executed with a relatively high priority, while control to maintain the relative distance during search between unmanned vessels 1010 that do not communicate with each other by wireless communication can be executed with a relatively low priority.

[0032] (A-2-2. Configuration of Unmanned Vehicle 1010) Fig. 4 is a functional block diagram showing the functional configuration of the unmanned watercraft 1010. Fig. 6 illustrates the functional block diagram of the unmanned watercraft 1010, but the master unit 1001 and slave unit 1002 of the unmanned watercraft 1010 can be equipped with common functional units, with the exception of the configuration of the measurement unit 1100 and communication unit 1400. The unmanned watercraft 1010 is equipped with a measurement unit 1100, a vessel status determination unit 1200, a navigation unit 1300, a communication unit 1400, a determination unit 1500, a recording unit 1600, and a power supply unit 1700.

[0033] The measurement unit 1100 is a functional unit that acquires measurement data of an object 7000 that exists in the sea, on the sea, or in the air using a measurement sensor 1110. The measurement unit 1100 includes the measurement sensor 1110 and a measurement control unit 1120.

[0034] When searching for an object in an underwater area as the search target area, the measurement sensor 1110 may be, for example, a sonar or other sonic sensor that uses ultrasonic or other sound waves. When searching for an object in an above-sea or above-air area as the search target area, the measurement sensor 1110 may be, for example, a radar sensor that detects the object 7000 by emitting radio waves such as millimeter waves or microwaves and receiving reflected waves, a laser sensor such as LiDAR that acquires point cloud data by irradiating laser light and receiving the reflected laser light, or an electro-optical sensor that acquires optical image data, an infrared sensor (IR camera), or other optical camera.

[0035] Furthermore, in addition to the above-mentioned sensors, the measurement sensor 1110 may also be composed of a seawater condition measurement sensor that measures seawater conditions such as the salinity concentration, hydrogen ion exponent (pH), water temperature, seawater components, and density of seawater, or a sea state measurement sensor that measures seawater conditions such as ocean currents, tidal currents, wave height, wave period, and ocean or tidal current speed in the surrounding sea area, or a meteorological measurement sensor that measures meteorological conditions such as temperature, humidity, wind speed, solar radiation, air pressure, rainfall, other weather conditions, and air quality in the surrounding sea, or a marine ecology measurement sensor that measures the state of seaweed beds and plankton in the sea.

[0036] The measurement control unit 1120 operates a sensor attitude changing device that can change the attitude of the measurement sensor 1110 to control at least one of the attitude angles of the measurement sensor 1110 around three axes relative to the unmanned watercraft 1010. Furthermore, for example, if the measurement sensor is an electro-optical sensor, an infrared camera, or other optical camera, the measurement control unit 1120 can adjust the frame rate, shutter speed, etc. If the measurement sensor is a laser sensor, the measurement control unit 1120 can adjust the output of the irradiated laser. If the measurement sensor is a sonar or radar sensor, the measurement control unit 1120 can adjust the output of the emitted sound waves, millimeter waves, microwaves, etc. Furthermore, the measurement control unit 1120 can adjust the measurement sensitivity of the measurement sensor to a desired control amount. If the measurement sensor is an optoelectronic optical sensor, the measurement control unit 1120 can change the zoom amount or resolution of the optical sensor to a desired control amount.

[0037] Next, the unmanned watercraft state determination unit 1200 includes a navigation state determination unit 1210, an internal state determination unit 1220, and an external state determination unit 1230, and is a functional unit that determines the navigation state and internal and external states of the unmanned watercraft 1010. The navigation state determination unit 1210 determines the position (two-dimensional or three-dimensional), movement speed, heading, movement direction, movement acceleration / deceleration, turning speed, and other state quantities related to the navigation state of the unmanned watercraft. The internal state determination unit 1220 determines the remaining energy and fuel levels of the battery installed in the unmanned watercraft, the possible travel distance that can be calculated from the remaining energy and fuel levels, temporary abnormal states of equipment installed in the unmanned watercraft (temperature abnormality, communication abnormality, etc.), and equipment failure states.

[0038] In addition, the external state determination unit 1230 can determine communication quality conditions such as communication strength (dB value, etc.), communication speed, and communication delay of wireless communications with other unmanned boats 1010 in the same group of the unmanned boat system 1000, or wireless communications with the integrated environment management system 2000 via a communication satellite 5100 or a terrestrial base station 5200, or the oceanographic conditions around the unmanned boat system (wave height, wave speed, ocean current speed, ocean current direction, tidal current speed, tidal current direction), meteorological conditions (wind speed, wind direction, air pressure, temperature, humidity), weather conditions (fog, thunder, rainfall, snowfall, hail, graupel, cloudiness, etc.), seawater conditions (seawater temperature, seawater density, salinity, pH value, presence or absence of seaweed beds, etc.), sun-related information (sun position (altitude, direction, trajectory), backlighting, frontlighting, amount of solar radiation), and other conditions (lunar position (altitude, direction, trajectory, lunar age), ionospheric disturbances (solar flares, etc.)).

[0039] The method by which the navigation state determination unit 1210 determines the position, moving speed, moving direction, and acceleration / deceleration of the aircraft is not particularly limited, but for example, the position, moving speed, and moving direction of the aircraft at the current time can be determined using GNSS (Global Navigation Satellite System), GPS (Global Positioning System), RTK-GNSS (Real Time Kinematic - Global Navigation Satellite System), etc.

[0040] As another example of a method for determining the position, speed, direction, and acceleration / deceleration of the aircraft by the navigation state determination unit 1210, for example, when the seabed shape can be detected by the measurement sensor 1110, the position, speed, and direction of movement of the aircraft at the current time can be determined using SLAM (Simultaneous Localization And Mapping) technology based on the pre-recorded seabed shape and the seabed shape detected by the measurement sensor 1110.

[0041] Here, the self-position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information. Also, the acceleration / deceleration can be calculated based on the amount of change over time in the determined moving speed.

[0042] The method for measuring the aircraft's heading is to determine the aircraft's heading at the current time using, for example, a geomagnetic sensor, a GNSS compass, or SLAM technology using the seabed shape. The heading includes an attitude angle (orientation) in a planar view around at least the Z axis, and preferably may be attitude information around three axes: the X axis, the Y axis, and the Z axis. The turning speed can be calculated based on the amount of change over time in the determined heading information.

[0043] Next, the navigation unit 1300 includes a thrust generating unit 1310, an attitude control mechanism 1320, and a navigation control unit 1330, and is a functional unit that navigates the aircraft in any direction in accordance with operational commands received via the communication unit 1400. The thrust generating unit 1310 can be any means capable of generating thrust, and as an example, can be configured with a propeller driven by the power of an engine or an electric motor. The thrust generating unit 1310 can also be configured with a sail that receives wind to generate thrust, or with a wave glider that receives wave power to generate thrust.

[0044] The attitude control mechanism 1320 is composed of a rudder mounted on the airframe, a propeller attitude change mechanism that can change the attitude angle of the propeller (mainly the yaw angle around the Z axis), etc., and can control the nose direction (yaw angle) of the aircraft by changing these angles. In addition, a center of gravity position change mechanism that changes the position of a heavy object inside the aircraft using an actuator can also control the attitude angles of the aircraft, such as the roll angle around the X axis and the pitch angle around the Y axis.

[0045] The navigation control unit 1330 is a functional unit that controls the thrust generation unit 1310 and the attitude control mechanism 1320 to control the navigation operation of the aircraft. The navigation control unit 1330 has one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), an MPU, or a DSP), and is equipped with a processing unit that can access a memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more processing steps.

[0046] The processing unit includes a control module configured to control the navigation status of the aircraft. For example, the control module adjusts the aircraft's position on the sea surface, movement speed, movement acceleration / deceleration, heading, turning speed, and attitude angle around three axes. That is, the navigation control unit 1330 controls the navigation operation of the aircraft by causing the aircraft to perform various operations such as forward movement, reverse movement, acceleration, deceleration, and turning.

[0047] Next, the communication unit 1400 is a functional unit that communicates with other unmanned watercraft 1010 in the same group and with the integrated environment management system 2000. Here, the communication unit 1400 of the parent unit 1001 is equipped with both an inter-unmanned watercraft communication unit 1410 that can communicate wirelessly with other unmanned watercraft 1010 in the group, and an overall control communication unit 1420 that can communicate with the integrated environment management system 2000. On the other hand, the communication unit 1400 of the child unit 1002 is equipped with the inter-unmanned watercraft communication unit 1410 that can communicate wirelessly with other unmanned watercraft 1010 in the group, and does not need to be equipped with the overall control communication unit 1420.

[0048] The unmanned craft-to-unmanned craft communication unit 1410 is a functional unit that has a communication antenna used for a maritime wireless communication network and communicates with other unmanned crafts 1010 in the same group that are within the communication distance. The overall control communication unit 1420 has a satellite communication antenna that can communicate with the communication satellite 5100 or a communication antenna that can communicate directly with the terrestrial base station 5200, and communicates with the integrated environmental management system 2000 via the communication satellite 5100 or the terrestrial base station 5200. In addition to the communication units described above, the communication unit 1400 may also have a communication unit that has an AIS antenna or a VHF antenna and that communicates with external surveillance vessels and AIS base stations.

[0049] Next, the determination unit 1500 is a functional unit that performs data processing such as primary processing and data compression of the measurement data acquired by the measurement sensor 1110. For example, the determination unit 1500 can perform data processing of raw data (measurement data) after measurement acquired by the measurement sensor 1110, and perform primary processing to generate transmission data to be wirelessly transmitted from the unmanned watercraft system 1000 to the integrated environment management system 2000. Furthermore, in order to reduce the transmission load when wirelessly transmitting transmission data from the unmanned watercraft system 1000 to the integrated environment management system 2000, the determination unit 1500 can perform data compression processing to compress the raw data (measurement data) after measurement to generate transmission data.

[0050] Furthermore, by performing primary processing of the measurement data, the determination unit 1500 can interpret the state of the target object 7000, determining the presence or absence of a detected object, the size of the detected object, etc. The determination unit 1500 may also have a function to determine whether or not measurement data or transmission data needs to be sent from the unmanned watercraft system 1000 to the integrated environment management system 2000, or to select the data to be sent, depending on the interpretation results.

[0051] Next, the recording unit 1600 includes a measurement data recording unit 1610, a host device state recording unit 1620, and a determination information recording unit 1630. The measurement data recording unit 1610 records measurement data measured by the measurement unit 1100. The host device state recording unit 1620 records various state information related to the host device determined by the host device state determination unit 1200. Furthermore, the determination information recording unit 1630 records various determination information determined by the determination unit 1500.

[0052] Next, the power supply unit 1700 is a functional unit that supplies power to each functional unit of the unmanned watercraft 1010. The power supply unit 1700 includes a power storage device 1710, a power generation device 1720, and a power control unit 1730. The power storage device 1710 can be configured as a battery or other device with a power storage function. The power generation device 1720 can be configured as a solar panel that generates power using sunlight or a wave power generation device that generates power using wave power. The power control unit 1730 is a functional unit that controls the charging and discharging of the power storage device 1710 and also controls the power generation operation of the power generation device 1720.

[0053] (A-3. Operation System 1 State Transition and Stakeholders) Next, the state transitions and stakeholders of the operation system 1 will be explained using FIG. 5 and FIG.

[0054] (A-3-1. State transition of operation system 1) Fig. 5 is a diagram showing an example of state transitions of the operation system 1. The example shown in Fig. 5 shows state transitions of the entire system from when the operation system 1 determines the operational requirements for the actual operation in advance preparations before the actual operation, to when the operation performs a search for an object, etc., and ends the operation. In particular, the example shown in Fig. 5 shows state transitions when the unmanned watercraft system 1000 is operated for the purpose of searching for an object 7000.

[0055] In the example shown in Figure 5, the operation system 1 has various states: a pre-operation preparation state (status 100) before operation, an on-site preparation state immediately before operation (status 200), an actual operation state during operation (status 300), and a search end / cancellation state immediately before operation ends (status 400).

[0056] In the pre-operation preparation state (status 100), first, user requirements (for example, if the purpose of this operation is to search for the target object 7000, the search target object, search area, search period, and search target values ​​(search rate, detection probability, etc.)) are accepted from the user (status 101). Next, a demonstration test operation is partially performed using the actual vehicle to evaluate the performance of the unmanned boat system 1000, and the operational requirements for this operation (number of unmanned boats 1010, movement route pattern, etc.) are determined (status 102). Next, resources such as unmanned boats 1010 to be used for the operation are procured according to the determined operational requirements (status 103). Next, the resources are transported to the site (status 104). Next, the resources are installed on the coast near the offshore area of ​​this operation (status 105).

[0057] Next, in the on-site preparation state immediately before operation, first, the unmanned boat 1010 is lowered onto the sea (status 201). Next, the unmanned boat 1010 is deployed in the sea area for actual operation (status 202). Furthermore, a simulated real machine and a virtual machine that simulate the real machine are generated in the virtual space by simulation, and the behavior information of the real machine in the real space is synchronized with the behavior of the simulated real machine.

[0058] Next, the actual operation state (status 300) has four states: a search state (status 301) in which a search for an object is performed, a standby state (status 302) in which the unmanned vessel 1010 waits in the operation area, and a recovery charge state (status 303) in which the battery mounted on the unmanned vessel 1010 is recharged using a solar panel or the like when the SOC of the battery drops. After a target object is discovered, the unmanned vessel 1010 is made to perform an action such as tracking the target object, and the status transitions between these four states.

[0059] Next, the search end / cancellation state (status 400) has a return state (status 401) in which the unmanned vessel 1010 is returned to the recovery position, and a recovery state (status 402) in which the unmanned vessel is towed back to the shore or mother ship at the recovery position.

[0060] As shown in Fig. 5, the operation system 1 has an operational requirement determination operation status that checks whether the user's desired conditions can be met in the search for this operation and what the operational requirements are for meeting the desired conditions before entering the search operation state, which is the purpose of this operation. The operation system 1 also has a state transition management unit (not shown) that manages the state transitions of the system shown in Fig. 5.

[0061] (A-3-2. Stakeholders regarding Operation System 1) Fig. 6 is a diagram showing an example of stakeholders related to the operation system 1. As shown in Fig. 6, the operation system 1 has users who determine the operational requirements of the unmanned watercraft system 1000 by inputting and outputting information via the user interface unit 3300 of the operation application control system 3000.

[0062] The cooperative system 6000 also includes private monitoring organization facilities and surveillance boats. In this case, the cooperative system 6000 has workers at the private monitoring organization facilities and crew members on the surveillance boats, who work together to monitor nuisance behavior and the like in the marine area. In addition to private monitoring organizations, the cooperative system 6000 may also include marine research organizations that investigate marine life, private rescue organizations that search for and rescue people in distress, and marine infrastructure operators that patrol and inspect marine infrastructure.

[0063] The external system 4000 also includes a weather information system, a sea state information system, an MDA system, etc. that provide information on current or predicted future sea state conditions (wave height, wave speed, ocean current speed, ocean current direction, tidal current speed, tidal current direction) in the area where the unmanned boat system 1000 is deployed and its surrounding areas, meteorological conditions (wind speed, wind direction, air pressure, temperature, humidity), weather conditions (fog, thunder, rainfall, snowfall, hail, graupel, cloudiness, etc.), seawater conditions (seawater temperature, seawater density, salinity, pH value, presence or absence of seaweed beds, etc.), sun-related information (sun position (altitude, direction, trajectory), backlight, frontlight, amount of solar radiation), and other conditions (lunar position (altitude, direction, trajectory, lunar age), ionospheric disturbances (solar flares, etc.)). In addition, the external system 4000 may include an AIS (Automatic Identification System) that obtains ship information about ships via wireless communication from ships navigating the search area and its surrounding areas and manages this ship information, or it may include a municipal system or a fisheries information system that provides information on the operation schedules of ships and fishing boats in the search area and its surrounding areas and information on other scheduled events.

[0064] Furthermore, to actually operate unmanned watercraft system 1000 in a marine area, advance preparations before operation and on-site preparations immediately before operation are required, as shown in Fig. 5. For this reason, Fig. 6 shows the actors involved in each task required before unmanned watercraft system 1000 can be put into actual operation, including such advance preparation work.

[0065] As shown in Figure 6, stakeholders related to the operation system 1 include, in addition to the actors mentioned above, resource procurement personnel who handle the multiple unmanned vessels 1010 used in preliminary demonstration tests and actual operations, as well as the cargo and transport vehicles used for transportation, resource transportation personnel who transport the unmanned vessels 1010 and cargo using transport vehicles, coastal standby / vehicle deployment personnel who deploy the unmanned vessels 1010 at sea and manage the multiple unmanned vessels 1010 waiting on the coast, vehicle recovery personnel who recover the unmanned vessels 1010 from the sea, pre-operation preparation personnel who manage the overall work of pre-operation preparation, and marine operation vehicle operators who operate the unmanned vessels 1010 during operations at sea.

[0066] (A-4. Operation of the convergence environment management system 2000 and the operational application control system 3000, etc.) Next, an example of the operation of the integrated environment management system 2000 and the operational application control system 3000 will be described with reference to FIGS.

[0067] (A-4-1. Operation of the Integrated Environment Management System 2000 in the Search Operation State) 7 is a diagram showing an example of the operation of the integrated environment management system 2000 in a search operation state. In particular, the diagram shows an example of the results of a simulation of the behavior of a mixed unmanned aerial vehicle group when the mixed unmanned aerial vehicle group, which includes a real vehicle (parent vehicle) deployed in an operation area and whose operation is controlled by the integrated environment management system 2000, and multiple virtual vehicles (child vehicles) placed in an operation area in a virtual space generated by the integrated environment management system 2000, is deployed in the operation area of ​​the virtual space. The example shown in FIG. 7 particularly shows an example in which the parent vehicle in the mixed unmanned aerial vehicle group is configured as a real vehicle and the multiple child vehicles are configured as virtual vehicles.

[0068] 7 shows the positions and orientations of the real and virtual machines deployed in the operation area of ​​the virtual space at a given time, as well as their past movement history. In other words, the real machines are actually deployed in the operation area of ​​the real space, and their behavior is controlled based on real machine control commands sent from the integrated environment management system 2000. Behavior information of the real machines (position, orientation, etc.) is sent from the real machines to the integrated environment management system 2000, and the behavior information of the real machines is reflected in the operation area of ​​the virtual space generated by the integrated environment management system 2000.

[0069] Furthermore, the virtual machine is not deployed in the operation area of ​​the real space, and the behavior state (position, orientation, etc.) of the virtual machine in the virtual space is calculated based on the virtual machine model by the integrated environment management system 2000, and the behavior state of the virtual machine is reflected in the operation area of ​​the virtual space. At this time, the integrated environment management system 2000 calculates the behavior state of the virtual machine by simulation, taking into account the environmental state of the real space as understood from environmental information acquired from the external system 4000 and real machine behavior information acquired from the real machine, thereby being able to simulate the actual behavior of the virtual machine when it is operated in the real space.

[0070] The integrated environment management system 2000 generates control commands for both real and virtual machines so that the mixed group of real and virtual unmanned vehicles described above can meet constraints such as not getting too close to each other (not colliding) in the operating area of ​​the virtual space and maintaining a relative distance within the communication range, and can also meet goals such as target search performance included in the user's desired conditions.

[0071] (A-4-2. Input / output information of the convergence environment management system 2000) Fig. 8 is a diagram showing an example of input / output information of the integrated environment management system 2000. Fig. 8 particularly shows input information input to the integrated environment management system 2000 and output information output from the integrated environment management system 2000 in the pre-operation preparation state (status 100) and the search operation state (status 300) shown in Fig. 5.

[0072] As shown in Figure 8, the input information includes pre-acquired information input to the integrated environment management system 2000 in the pre-preparation state before operation (status 100), real-time processing information acquired during demonstration operation in the pre-preparation state before operation (status 100) or while the integrated environment management system 2000 is executing processing in the exploration operation state (status 300), which is the actual operation, and actual device replacement determination related information acquired during processing by the integrated environment management system 2000 in the exploration operation state (status 300), which is the actual operation.

[0073] The pre-acquired information includes, for example, user request information acquired from the user, assumed environment information assumed in the operation area where the actual operation will be performed, and assumed condition parameters including various assumed conditions for the actual operation. The real-time processing information is, for example, information acquired during processing execution by the integrated environment management system 2000, and includes, for example, a user intervention command acquired from the user, actual environment measurement data acquired from the external system 4000 and the external state determination unit 1230 of the actual aircraft, actual aircraft behavior information acquired from the navigation state determination unit 1210 of the actual aircraft, actual aircraft state information including internal state information of the actual aircraft such as SOC acquired from the internal state determination unit 1220 of the actual aircraft, and actual aircraft measurement data measured by the measurement unit 1100 of the actual aircraft. In addition, the actual aircraft replacement determination related information includes, for example, object warning information from the external system 4000, actual aircraft dispatch request information and actual aircraft recovery request information received from the user, etc.

[0074] 8, the output information is information output from the integrated environment management system 2000 in the advance preparation state before operation (status 100), and includes operational requirement information necessary for subsequent advance preparations and on-site preparations immediately before operation. Also, the output information is information output from the integrated environment management system 2000 in the search and operate state (status 300), and includes display data to be displayed on the display unit 3310 of the operational application control system 3000 and the user terminal device 8000, and control data for the actual devices deployed in the operation area in the real space.

[0075] The display data includes, for example, drone group behavior information showing the current behavior and future predicted behavior of a drone group that is a mixture of real and virtual drones, drone status information showing the internal status of each drone, such as SOC, and actual drone measurement data measured by the measurement unit 1100 of the actual drone.

[0076] (A-4-3. Information flow within the Integrated Environmental Management System 2000) Next, FIG. 9 is a diagram showing an example of input / output information between a plurality of functional units of the integrated environment management system 2000. In FIG.

[0077] As shown in Figure 9, the integrated environment management system 2000 includes an information acquisition unit 2100, a virtual space generation unit 2200, a real-time unmanned aerial vehicle swarm state simulator 2300, a future swarm control state simulator 2400, a state determination unit 2500, a swarm control command determination unit 2600, and an information output unit 2700.

[0078] First, the information acquisition unit 2100 acquires the pre-acquired information shown in Fig. 8 from the operation application control system 3000. Next, the information acquisition unit 2100 inputs the acquired user request information and assumed environment information to the virtual space generation unit 2200. The virtual space generation unit 2200 generates a virtual operation area that simulates the operation area in the virtual space, based on the operation area and assumed environment information included in the acquired user request information.

[0079] Next, after the actual vehicle is deployed in the real space, actual vehicle behavior information, actual vehicle status information, actual vehicle measurement data, etc. are input from the actual unmanned boat 1010 deployed in the operating area in the real space to the information acquisition unit 2100. In addition, actual environment measurement data measured by the external system 4000 or the actual vehicle is input to the information acquisition unit 2100. In addition, target object warning information is input to the information acquisition unit 2100 from the external system 4000.

[0080] Next, the virtual space generation unit 2200 updates the environmental state in the generated virtual space based on the acquired real-world environment measurement data. The updated environmental state in the virtual space is input to the real-time unmanned aerial vehicle swarm state simulator 2300 and the future swarm control state simulator 2400.

[0081] Next, real-time unmanned aerial vehicle swarm state simulator 2300 calculates the behavior state of all vehicles, including the virtual and actual vehicles, based on the actual vehicle behavior information and assumed condition parameters acquired from information acquisition unit 2100, the environmental state acquired from virtual space generation unit 2200, and further the virtual vehicle model information, and outputs the calculated behavior state information of all vehicles to state determination unit 2500, group control command determination unit 2600, and information output unit 2700. In addition, real-time unmanned aerial vehicle swarm state simulator 2300 updates the virtual vehicle model information based on the actual vehicle behavior information, and outputs the updated virtual vehicle model information to future group control state simulator 2400.

[0082] Here, virtual machine model information refers to model information that simulates a virtual machine in a simulation, and may be a model in which all functional units of the virtual machine are simulated in a virtual space, or it may be configured as a HILS (Hardware In the Loop Simulator) machine in which some functional units of the virtual machine are configured with actual hardware and other functional units are configured with virtual models to perform simulation.

[0083] Next, the future group control state simulator 2400 predicts and calculates the behavior state of all machines, including the virtual machine and the actual machine, at a future time based on the assumed condition parameters acquired from the information acquisition unit 2100, the environmental state acquired from the virtual space generation unit 2200, and further based on the updated virtual machine model information, and outputs the calculated future behavior prediction information of all machines to the state determination unit 2500, the group control command determination unit 2600, and the information output unit 2700.

[0084] Next, the status determination unit 2500 determines various operational conditions (object detection status, current operational status, future predicted operational status) of the unmanned boat system 1000 based on the aircraft increase / decrease determination conditions included in the user request information acquired from the information acquisition unit 2100, the actual aircraft measurement data acquired from the actual aircraft, the warning information acquired from the external system 4000, the behavior status information of all aircraft acquired from the real-time unmanned aircraft swarm status simulator 2300, and the future behavior prediction information of all aircraft acquired from the future swarm control status simulator 2400, and determines whether to increase or decrease the number of actual aircraft deployed in the operating area, and outputs the determined actual aircraft increase / decrease decision result to the swarm control command determination unit 2600.

[0085] Next, the group control command determination unit 2600 generates control commands for all aircraft, including virtual and real aircraft, based on the current behavior state information of all aircraft obtained from the real-time unmanned aircraft group state simulator 2300 and the future behavior state information of all aircraft obtained from the future group control state simulator 2400, and outputs the generated control commands to the information output unit 2700.

[0086] Furthermore, based on the result of the actual machine increase / decrease decision obtained from the state determination unit 2500, if an actual machine is to be increased, a virtual machine to replace the actual machine is selected, and on the other hand, if an actual machine is to be decreased, a real machine to be recovered (to be replaced with a virtual machine) is selected.Based on the result of the selection, a control command is generated for all machines including the virtual machine and the actual machine, and the generated control command is output to the information output unit 2700.

[0087] The information output unit 2700 outputs display data including the acquired behavior status information of all the aircraft to the operational application control system 3000. The information output unit 2700 also transmits control commands for the actual aircraft to the actual aircraft of the unmanned boat system 1000.

[0088] The operational application control system 3000 can also acquire the behavior information of the actual vehicle included in the display data directly from the actual vehicle of the unmanned watercraft system 1000, rather than from the information output unit 2700. The operational application control system 3000 can also acquire the measurement data of the actual vehicle included in the display data directly from the actual vehicle. The operational application control system 3000 can also acquire the environmental information of the operation area directly from the external system 4000, rather than from the information output unit 2700.

[0089] (A-4-3. Detailed functions of the Integrated Environmental Management System 2000) Next, detailed functions of the integrated environment management system 2000 will be described using Fig. 10. Fig. 10 is a functional block diagram showing the functional configuration of the integrated environment management system 2000. The integrated environment management system 2000 includes an information acquisition unit 2100, a virtual space generation unit 2200, a real-time unmanned aerial vehicle swarm state simulator 2300, a future swarm control state simulator 2400, a state determination unit 2500, a swarm control command determination unit 2600, and an information output unit 2700.

[0090] (A-4-3-1. Information acquisition section 2100) The information acquisition unit 2100 is a functional unit that acquires various information required for processing by each functional unit of the integrated environment management system 2000 (described later) from the user interface unit 3300 of the operational application control system 3000, the user terminal device 8000, the external system 4000, and the unmanned watercraft system 1000. The information acquisition unit 2100 includes a user request information acquisition unit 2110, a simulation condition acquisition unit 2120, an actual machine information acquisition unit 2130, an environmental information acquisition unit 2140, an external information acquisition unit 2150, and a user intervention information acquisition unit 2160.

[0091] The user request information acquisition unit 2110 is a functional unit that acquires user request information including desired conditions required for this operation from the user. Fig. 12 is a diagram showing an example of user request information acquired by the user request information acquisition unit 2110.

[0092] 12, the user request information includes, for example, request condition information, target information, and aircraft increase / decrease determination conditions. The request condition information includes information that can identify the object to be searched, information that can identify the operation area (information specifying the position, range, etc. of the operation area), operation period (date and time of search execution, start and end times, time period), etc.

[0093] The target information also includes information on preset targets related to the operational status of the unmanned watercraft system 1000. In the example shown in Fig. 12, the target information includes a target search rate related to a target value for the target search rate (also called coverage rate) indicating the ratio of the cumulative measurement area measured by the measurement sensor of the unmanned watercraft 1010 during the operation period to the operation area, and a target search performance related to a target value for the detection probability (also called target discovery probability, detection rate, or encounter rate) indicating the probability that the unmanned watercraft 1010 will detect the target object 7000 using the measurement sensor. Here, the target search rate may include an entire area target search rate, which is a target value related to the ratio of areas that have been moved or measured by the unmanned watercraft 1010 to the entire operation area to be searched, or a local area target search rate, which is a target value related to the ratio of areas that have been moved or measured by the unmanned watercraft 1010 to local areas that are part of the operation area to be searched.

[0094] 12 shows an example in which search rate, detection probability, and the like are set in advance as target information for actual operation of unmanned watercraft system 1000, but the target information is not limited to this and may include, for example, target values ​​related to the average or minimum charge amount, charge rate, or charge capacity of the power storage devices of multiple unmanned watercrafts (remaining charge (Wh), charge rate (%), power receiving capacity (Wh)), or target values ​​related to the number of unmanned watercrafts that have been swapped with waiting replacement vehicles due to insufficient charge amount, charge rate, or charge capacity of the power storage devices. Furthermore, the various types of target information described above may be set as minimum allowable values ​​(target values) to be exceeded, or as allowable ranges (target ranges) to be within.

[0095] In addition, the aircraft increase / decrease determination conditions include an aircraft number increase determination condition, which is a condition for determining that the number of actual aircraft will be increased, and an aircraft number decrease determination condition, which is a condition for determining that the number of actual aircraft will be decreased.

[0096] As shown in Figure 12, the conditions for determining whether to increase the number of aircraft can include, for example, at least one of the following: obtaining alert information regarding an object from the external system 4000; entering a specified time period in which increased alert is required; or obtaining a command to increase the number of actual aircraft from a user via the user input receiving unit 3320, the user terminal device 8000, etc.

[0097] As another example, the conditions for determining an increase in the number of aircraft may include, for example, at least one of the following: the detection of the object 7000 or a candidate object for the object 7000 using measurement data from a measurement sensor capable of measuring the object 7000 present in the operation area or its surrounding area; or a value indicating the search performance, movement performance, battery performance, or communication performance of the unmanned vessel 1010 having decreased below a predetermined value.

[0098] Also, as shown in Figure 12, the conditions for determining a reduction in the number of aircraft can include, for example, at least one of the following: alert information previously obtained from the external system 4000 has been lifted; an actual aircraft reduction request command has been received from a user via the user input receiving unit 3320 or the user terminal device 8000, requesting a reduction in the number of actual aircraft; or a detection flag that has detected the target object 7000 or a candidate object for the target object 7000 using the measurement data of the measurement sensor 1110 has been lifted.

[0099] As another example, the conditions for determining an increase in the number of aircraft may include, for example, at least one of the following: the wave conditions in the operating area have worsened beyond a predetermined value, or it has been predicted that future wave conditions in the operating area will worsen beyond a predetermined value.

[0100] As another example, the condition for determining an increase in the number of aircraft may include, for example, determining the future relative distance between other moving bodies and the unmanned aircraft based on current movement information or future planned movement information of other moving bodies, including other ships, obtained from the external system 4000, and the future behavior state of the mixed unmanned aircraft group generated by the future group control state simulator 2400 described below, and determining that an interference state in which the relative distance is less than a predetermined distance will occur in the future.

[0101] The simulation condition acquisition unit 2120 is a functional unit that acquires various information related to the simulation execution conditions when calculating the unmanned aerial vehicle swarm state by simulation in the real-time unmanned aerial vehicle swarm state simulator 2300 and the future swarm control state simulator 2400, which will be described later. Figure 13 is a diagram showing an example of assumed condition parameters acquired by the simulation condition acquisition unit 2120.

[0102] As shown in Fig. 13, the assumed condition parameters include assumed environmental conditions, aircraft conditions, assumed object conditions, environmental impact conditions, and other hypothetical conditions.

[0103] The expected environmental conditions are information on expected conditions of environmental information related to environmental disturbances expected in the operation area and operation time included in the user-requested condition information, and include meteorological conditions, sea conditions, etc. Meteorological conditions include, for example, wind speed, wind direction, rain, snow, cloudiness, fog, solar radiation, brightness, visibility, temperature, and other weather conditions, while sea conditions include, for example, wave height, and the speed, direction, and position of ocean and tidal currents.

[0104] The vehicle conditions are condition information related to the arrangement and vehicle performance of the multiple unmanned vehicles that make up the unmanned vehicle system 1000, and include, for example, the arrangement conditions of the multiple unmanned vehicles 1010 (random arrangement, uniform arrangement, density-changing arrangement, etc.), the movement conditions of each unmanned vehicle 1010 (movement speed, turning speed, movement direction, movement path, straight-line travel time, etc.), navigation performance conditions related to the navigation performance of the unmanned vehicle 1010 (maximum movement speed, maximum turning speed, maximum deceleration, power consumption per speed, etc.), and measurement conditions related to the measurement sensor 1110 mounted on the unmanned vehicle 1010. These include measurement sensor conditions (type of measurement sensor, measurable range, other measurement sensor performance, etc.), communication device conditions for the communication unit 1400 installed in the unmanned boat 1010 (communication distance, other communication performance, etc.), power storage device conditions for the power storage device 1710 such as a battery installed in the unmanned boat 1010 (charging capacity, output performance (W), charging performance (W), etc.), and power generation device conditions for the power generation device 1720 such as a solar panel or wave power generation device installed in the unmanned boat 1010 (power generation performance (W), etc.).

[0105] The target assumption conditions are assumed information regarding the target object 7000 that is the target of search, and include, for example, the presence probability indicating the probability that the target object exists in the operation area, the expected speed, expected course, and expected straight-line travel time of the target object.

[0106] The environmental influence conditions are information indicating the degree to which environmental disturbances affect the mobility performance of the unmanned watercraft 1010. For example, the mobility performance influence includes information on the degree to which ocean currents, wave height, wind speed, wind direction, water temperature, etc. affect the mobility performance of the unmanned watercraft 1010 using the navigation unit 1300. The environmental influence conditions also include a measurement performance influence that indicates the degree to which environmental disturbances affect the measurement performance of the measurement sensor 1110. The measurement performance influence includes information on the degree to which wave height, visibility, brightness, etc. affect the measurement performance of the measurement sensor 1110. The environmental influence conditions also include a communication performance influence that indicates the degree to which environmental disturbances affect the communication performance of the communication unit 1400. The communication performance influence includes information on the degree to which clouds, fog, wave height, etc. affect the communication performance of the communication unit 1400.

[0107] Other hypothetical conditions may also include information regarding hypothetical scenarios that may occur during operation, such as a scenario in which a large ship approaches the operating area of ​​the unmanned vehicle system 1000, or a scenario in which a submersible approaches the operating area.

[0108] The actual aircraft information acquisition unit 2130 is a functional unit that acquires aircraft behavior information and measurement data measured by the actual aircraft from the actual aircraft deployed in the operational area in real space. For example, the actual aircraft information acquisition unit 2130 can acquire actual aircraft behavior information determined by the actual aircraft navigation state determination unit 1210 and the internal state determination unit 1220 from the actual aircraft deployed in the operational area in real space. Here, the actual aircraft behavior information can include information such as the aircraft's position (two-dimensional or three-dimensional), movement speed, heading, movement direction, movement acceleration / deceleration, turning speed, other state quantities related to the navigation state, remaining energy and fuel in a battery installed in the aircraft, a possible travel distance that can be calculated from the remaining energy and fuel, temporary abnormal states of equipment installed in the aircraft (temperature abnormality, communication abnormality, etc.), and equipment failure states.

[0109] The actual machine information acquisition unit 2130 also has a function of acquiring actual machine measurement data measured by the actual machine measurement unit 1100 of the unmanned watercraft 1010. Furthermore, the actual equipment information acquisition unit 2130 may have the function of acquiring environmental measurement data regarding environmental information from the external condition determination unit 1230 of the actual equipment of the unmanned boat 1010, including at least any of the following in the area where the unmanned boat system 1000 is deployed and its surrounding areas: meteorological conditions (wind speed, wind direction, atmospheric pressure, temperature, humidity), weather conditions (fog, lightning, rainfall, snowfall, hail, graupel, cloudiness, etc.), sea state information (wave height, wave speed, ocean current speed, ocean current direction, tidal current speed, tidal current direction), other seawater conditions (seawater temperature, seawater density, salinity, pH value, presence or absence of seaweed beds, etc.), sun-related information (sun position (altitude, direction, trajectory), backlight, frontlight, amount of solar radiation), and other conditions (lunar position (altitude, direction, trajectory, age of the moon), ionospheric disturbances (solar flares, etc.)).

[0110] The environmental information acquisition unit 2140 is a functional unit that acquires information about the current environmental state or the predicted future environmental state in the operation area from an external system 4000, such as a weather information system or an MDA system. The environmental information acquisition unit 2140 has a function of acquiring environmental information from the external system 4000, including at least one of the following in the operation area and its surrounding areas: meteorological conditions (wind speed, wind direction, atmospheric pressure, temperature, humidity), weather conditions (fog, lightning, rainfall, snowfall, hail, graupel, cloudiness, etc.), oceanographic information (wave height, wave speed, ocean current speed, ocean current direction, tidal current speed, tidal current direction), other seawater conditions (seawater temperature, seawater density, salinity, pH value, presence or absence of seaweed beds, etc.), sun-related information (sun position (altitude, direction, trajectory), backlighting, frontlighting, solar radiation), and other conditions (lunar position (altitude, direction, trajectory, lunar age), ionospheric disturbances (solar flares, etc.)).

[0111] The external information acquisition unit 2150 has the function of acquiring, from an external system 4000 such as AIS, ship information regarding current ships and fishing boats sailing in the operation area and its surrounding areas, future operation schedule information, or information regarding other events currently taking place or scheduled to take place in the operation area and its surrounding areas.

[0112] Next, the user intervention information acquisition unit 2160 is a functional unit that acquires intervention information from the user when, in the search operation state (status 300), the integrated environment management system 2000 simulates and calculates in real time the behavior state of a mixed aircraft group that is a mixture of real and virtual aircraft in a virtual space, and generates control commands for the mixed aircraft group.

[0113] (A-4-3-2. Virtual space generation unit 2200) The virtual space generation unit 2200 is a functional unit that generates a virtual operation area within a virtual space that simulates the operation area based on the operation area included in the request information acquired by the user request information acquisition unit 2110 and the assumed environment information acquired by the simulation condition acquisition unit 2120.

[0114] The virtual space generation unit 2200 generates a virtual operation area in the virtual space with the same position and range as the operation area, based on, for example, operation area information (information on the area's position and range, etc.) included in the user request information.

[0115] Furthermore, the virtual space generation unit 2200 simulates an environment similar to the assumed environment information (for example, ocean currents, wave height, wind speed, wind direction, amount of solar radiation, brightness, visibility, weather, etc.) within the virtual operation area based on the assumed environment information. Furthermore, the virtual space generation unit 2200 can simulate events such as the approach of a large ship or a submarine within the virtual space based on the setting information for other virtual conditions shown in Fig. 13.

[0116] Furthermore, when current environmental information for the operation area or its surrounding area is acquired from the actual device information acquisition unit 2130 or the environmental information acquisition unit 2140, the current environmental state of the already generated virtual operation area can be updated based on that environmental information. Furthermore, when future environmental prediction information for the operation area or its surrounding area is acquired from the environmental information acquisition unit 2140, the future environmental state of the virtual operation area can be updated based on that environmental prediction information.

[0117] (A-4-3-3. Real-time Unmanned Aerial Vehicle Swarm Simulator 2300) The real-time unmanned aerial vehicle group state simulator 2300 is a functional unit that calculates the behavior state of a mixed unmanned aerial vehicle group that is a mixture of a simulated real vehicle that simulates a real vehicle in a virtual space and a virtual unmanned vehicle that is generated in a virtual space, based on real vehicle behavior information of the real vehicle acquired by the real vehicle information acquisition unit 2130. The real-time unmanned aerial vehicle group state simulator 2300 includes a virtual vehicle behavior calculation unit 2310, a real vehicle behavior calculation unit 2320, and a mixed vehicle group behavior calculation unit 2330.

[0118] The virtual aircraft behavior calculation unit 2310 is a functional unit that calculates virtual aircraft behavior calculation information relating to the aircraft behavior of the virtual aircraft deployed in a virtual operation area generated in a virtual space, based on virtual aircraft model information.

[0119] The virtual aircraft behavior calculation unit 2310 first generates or acquires virtual aircraft model information capable of simulating and calculating the behavior of the virtual aircraft. Here, the virtual aircraft model information is generated based on information on various condition parameters such as those shown in FIG. 13 acquired by the simulation condition acquisition unit 2120. For example, the virtual aircraft behavior calculation unit 2310 sets parameters that define the performance of the virtual aircraft model's navigation performance, measurement sensors, communication devices, power storage devices, and power generation devices, based on the assumed aircraft conditions. Furthermore, the virtual aircraft behavior calculation unit 2310 sets parameters that define the degree of influence that the environmental conditions in the virtual operation area have on the mobility performance, measurement performance, communication performance, power storage performance, and power generation performance of the virtual aircraft model, based on the environmental impact conditions.

[0120] The virtual aircraft behavior calculation unit 2310 calculates virtual aircraft behavior calculation information related to the aircraft behavior of the virtual aircraft based on operation commands for the virtual aircraft generated by the group control command determination unit 2500 (described below), the environmental state of the virtual operation area, and virtual aircraft model information. Here, the virtual aircraft behavior calculation information calculated by the virtual aircraft behavior calculation unit 2310 includes at least one of the position, orientation, movement speed, movement direction, rotational angular velocity, movement acceleration, rotational angular acceleration, and battery charge state or degradation state of each virtual aircraft.

[0121] Next, the actual aircraft behavior calculation unit 2320 generates or acquires actual aircraft model information of the simulated actual aircraft that simulates the actual aircraft in the virtual space. Here, the actual aircraft model information is generated based on performance information of each functional unit installed in the actual aircraft and information on various condition parameters such as those shown in FIG. 13 acquired by the simulation condition acquisition unit 2120. For example, the actual aircraft behavior calculation unit 2320 sets parameters that define the performance of the actual aircraft model, such as the navigation performance, measurement performance, communication device, power storage device, and power generation device, based on the performance information of each functional unit installed in the actual aircraft. Furthermore, the actual aircraft behavior calculation unit 2320 sets parameters that define the degree of influence of environmental conditions in the virtual operation area on the mobility performance, measurement performance, communication performance, power storage performance, and power generation performance of the actual aircraft model, based on the environmental impact conditions.

[0122] The actual aircraft behavior calculation unit 2320 calculates actual aircraft behavior calculation information regarding the aircraft behavior of a virtual actual aircraft that simulates the actual aircraft in a virtual space, based on the operation commands for the actual aircraft generated by the group control command determination unit 2500 described below, the environmental conditions of the virtual operation area, and actual aircraft model information.

[0123] Here, if the accuracy of the actual aircraft model is sufficiently high, the actual aircraft behavior calculation information calculated by the actual aircraft behavior calculation unit 2330 should be roughly consistent with the actual aircraft behavior information acquired by the actual aircraft behavior information acquisition unit 2210. If there is a large difference between the actual aircraft behavior calculation information and the actual aircraft behavior information, the accuracy of the actual aircraft model will be low, and in such a case, the actual aircraft behavior calculation unit 2320 has a function to correct the parameters of the actual aircraft model, and adjusts the parameters of the actual aircraft model according to the comparison result between the actual aircraft behavior calculation information and the actual aircraft behavior information.

[0124] Note that the actual machine behavior calculation unit 2320 obtains the comparison results between the actual machine behavior calculation information and the actual machine behavior information, and the parameter correction information of the actual machine model, which are information that should also be used to correct the parameters of the virtual machine model. Therefore, the virtual machine behavior calculation unit 2310 can obtain the comparison results between the actual machine behavior calculation information and the actual machine behavior information and the parameter correction information of the actual machine model from the actual machine behavior calculation unit 2320, correct the parameters of the virtual machine model based on this information, and calculate the virtual machine behavior calculation information using the corrected virtual machine model.

[0125] Next, the mixed aircraft group behavior calculation unit 2330 is a functional unit that generates behavior information of a mixed unmanned aircraft group that includes a mixture of real and virtual aircraft based on the real aircraft behavior information acquired by the real aircraft information acquisition unit 2130 and the virtual aircraft behavior calculation information calculated by the virtual aircraft behavior calculation unit 2310.

[0126] For example, the mixed aircraft group behavior calculation unit 2330 reflects the behavior of the simulated real aircraft in the virtual operation area based on information such as the position, orientation, movement speed, movement direction, rotational angular velocity, movement acceleration, rotational angular acceleration, battery charge state and degradation state of the real aircraft contained in the real aircraft behavior information, and further reflects the behavior of the virtual aircraft in the virtual operation area based on information such as the position, orientation, movement speed, movement direction, rotational angular velocity, movement acceleration, rotational angular acceleration, battery charge state and degradation state of the virtual aircraft contained in the virtual aircraft behavior calculation information, thereby reflecting the behavior of all aircraft in the unmanned aircraft group that is a mixture of simulated real aircraft and virtual aircraft within the virtual operation area.

[0127] (A-4-3-4. Future Array Control State Simulator 2400) The future group control state simulator 2400 is a functional unit that predicts and calculates the future behavior state of a mixed unmanned aircraft group that is a mixture of simulated real aircraft that simulate real aircraft in a virtual space and virtual aircraft that are unmanned aircraft generated in a virtual space. The future group control state simulator 2400 may also have a function that predicts the future movement state (movement path, movement speed, etc.) of other moving objects by time acceleration within the simulator based on the current movement state of other moving objects acquired from the external system 4000 or detected from the real aircraft. The future group control state simulator 2400 includes a virtual aircraft future behavior prediction unit 2410, a real aircraft future behavior prediction unit 2420, and a mixed aircraft group future behavior prediction unit 2430.

[0128] The virtual machine future behavior prediction unit 2410 is a functional unit that accelerates time in the virtual space and calculates virtual machine future behavior prediction information regarding the future behavior of the virtual machine deployed in the virtual operation area, based on the control commands generated by the control command determination unit 2510 (described later), virtual machine model information, and the future environmental state in the virtual operation area.

[0129] Next, the actual aircraft future behavior prediction unit 2420 is a functional unit that accelerates time in the virtual space and calculates actual aircraft future behavior prediction information regarding the future aircraft behavior of the simulated actual aircraft deployed in the virtual operation area, based on the future control commands generated by the future command determination unit 2620 (described later), actual aircraft model information of the simulated actual aircraft that simulates the actual aircraft, and the future environmental state in the virtual operation area.

[0130] Next, the mixed aircraft group future behavior prediction unit 2430 is a functional unit that accelerates time in the virtual space and generates future behavior information for a mixed unmanned aircraft group that is a mixture of simulated real aircraft and virtual aircraft, based on the virtual aircraft future behavior prediction information calculated by the virtual aircraft future behavior prediction unit 2410 and the actual aircraft future behavior prediction information calculated by the actual aircraft future behavior prediction unit 2420.

[0131] (A-4-3-5. Status determination unit 2500) The state determination unit 2500 is a functional unit that determines the search state of the object and the current and future operating states of the unmanned watercraft system 1000, and decides whether to increase or decrease the number of unmanned watercraft. The state determination unit 2500 includes an object detection state determination unit 2510, a current state determination unit 2520, a future state prediction determination unit 2530, and an unmanned watercraft increase / decrease determination unit 2540.

[0132] The object detection state determination unit 2510 is a functional unit that determines the detection state of an object, including the detection of a sign that the search object will appear and the initial detection of the object. For example, the object detection state determination unit 2510 determines that the object detection state is a sign that the object will appear when it receives warning information about the object from the external system 4000 or when it is a predetermined time period when it is necessary to strengthen warning.

[0133] Furthermore, the object detection state determination unit 2510 can detect the object 7000 and its candidate objects based on measurement data measured by the measurement sensor 1110 mounted on the actual unmanned watercraft. When the object detection state determination unit 2510 detects a candidate object for the object 7000, it can determine that the object is in a pre-detection state, and when the object 7000 is detected based on the measurement data, it can determine that the object is in an initial detection state.

[0134] Here, the object detection state determination unit 2510 detects nearby ships based on measurement data acquired from the actual device, compares it with AIS information acquired by the external information acquisition unit 2150, and when a ship disconnected from the AIS system is detected, it can determine that the ship is a candidate object for the search target. Furthermore, it can determine that the ship is a candidate object for the search target based on the type of ship detected based on the measurement data. Furthermore, it can determine that the ship is a candidate object for the search target based on the position and time period of the ship detected based on the measurement data.

[0135] Furthermore, the object detection state determination unit 2510 can determine whether the detection flag of the precursor detection state in which a candidate object for the object 7000 has been detected and the initial detection state in which the object 7000 has been detected has been cleared.

[0136] The current state determination unit 2520 is a functional unit that determines the current navigation state of the mixed unmanned vehicle group, including real and virtual vehicles. The current state determination unit 2520 determines the current relative distance between the unmanned vehicles in the mixed unmanned vehicle group based on, for example, the current behavior information (position, orientation, movement direction, movement speed, etc.) of each unmanned vehicle in the mixed unmanned vehicle group calculated by the real-time unmanned vehicle group state simulator 2300, and determines whether an interference state has occurred in which the relative distance is less than a predetermined collision avoidance safe distance.

[0137] In addition, the current state determination unit 2520 is not limited to determining the interference state between multiple unmanned vessels 1010, but also determines the current relative distance between other moving bodies and the unmanned vessel 1010 based on, for example, the current movement information or future planned movement information of other moving bodies obtained from the external system 4000 and the current behavior state of the mixed unmanned vessel group generated by the real-time unmanned vessel group state simulator 2300, and determines whether an interference state has occurred in which the current relative distance is less than a predetermined collision avoidance safe distance.

[0138] As another example, the current state determination unit 2520 determines the current relative distance between the unmanned vessels in the mixed unmanned vessel group based on the current behavior state of the mixed unmanned vessel group generated by the real-time unmanned vessel group state simulator 2300, and determines whether a communication failure state has occurred in which the relative distance is greater than the communication distance between the unmanned vessels shown in Figure 13.

[0139] As another example, the current state determination unit 2520 determines the operational state of the mixed unmanned aircraft group based on the current behavior state of the mixed unmanned aircraft group generated by the real-time unmanned aircraft group state simulator 2300, and determines whether the operational state satisfies the specified target information included in the user request information such as that shown in Figure 12.

[0140] As another example, the current state determination unit 2520 determines the relative distance on the horizontal plane between the typhoon, low pressure, rain cloud, or wave area and the unmanned vessel based on, for example, current or future behavior information of the typhoon, low pressure, rain cloud, or wave area acquired from the external system 4000 or the actual aircraft information acquisition unit 2130, and the current behavior state of the mixed unmanned aircraft group generated by the real-time unmanned aircraft group state simulator 2300, and determines whether a wave area navigation state in which the relative distance is less than a predetermined distance (i.e., a state in which the unmanned vessel is navigating a wave area where waves of a predetermined magnitude or greater are expected due to the influence of the typhoon, low pressure, or rain cloud) has occurred.

[0141] Next, the future state prediction determination unit 2530 is a functional unit that predicts and determines the future navigation state of the mixed unmanned vehicle group, which includes real and virtual vehicles. The future state prediction determination unit 2530 determines the future relative distance between the unmanned vehicles in the mixed unmanned vehicle group based on, for example, future behavior information (position, orientation, movement direction, movement speed, etc.) of each unmanned vehicle in the mixed unmanned vehicle group calculated by the future group control state simulator 2400, and determines whether an interference state will occur in the future, in which the relative distance will be less than a predetermined collision avoidance safe distance.

[0142] In addition, the future state prediction determination unit 2530 is not limited to determining interference states between multiple unmanned vessels 1010, but also determines the future relative distance between other moving bodies and the unmanned vessel 1010 based on, for example, current movement information or future planned movement information of other moving bodies obtained from the external system 4000 or detected from the actual vessel, or the future movement paths of other moving bodies predicted and calculated by the future group control state simulator 2400, and the future behavior state of the mixed unmanned vessel group generated by the future group control state simulator 2400, and determines whether an interference state will occur in the future in which the future relative distance will be less than a predetermined collision avoidance safe distance.

[0143] As another example, the future state prediction determination unit 2530 determines the future relative distance between the unmanned vessels in the mixed unmanned vessel group based on the future behavior state of the mixed unmanned vessel group generated by the future group control state simulator 2400, and determines whether a communication loss state will occur in the future in which the relative distance will be greater than the communication distance between the unmanned vessels shown in Figure 13.

[0144] As another example, the future state prediction determination unit 2530 determines the future operational state of the mixed unmanned aircraft group based on the future behavior state of the mixed unmanned aircraft group generated by the future group control state simulator 2400, and determines whether the operational state will satisfy the specified target information included in the user request information such as that shown in Figure 12 in the future.

[0145] As another example, the future state prediction determination unit 2530 determines the relative distance on the horizontal plane between the typhoon, low pressure, rain cloud, or wave area and the unmanned vessel based on, for example, current or future behavior information of the typhoon, low pressure, rain cloud, or wave area obtained from the external system 4000 and the future behavior state of the mixed unmanned vessel group generated by the future group control state simulator 2400, and determines whether a wave area navigation state in which the relative distance is less than a predetermined distance (i.e., a state in which the unmanned vessel is navigating a wave area where waves of a predetermined magnitude or greater are expected due to the influence of the typhoon, low pressure, or rain cloud) has occurred.

[0146] Next, the unmanned aircraft increase / decrease decision unit 2540 is a functional unit that decides to increase the number of real aircraft deployed in the virtual operation area in the virtual space by replacing at least one of the virtual aircraft deployed in the operation area with a real aircraft, or to decrease the number of real aircraft deployed in the operation area in the real space by replacing at least one of the real aircraft deployed in the operation area with a virtual aircraft.

[0147] The unmanned aircraft increase / decrease decision unit 2540 determines to increase the number of actual aircraft deployed in the operational area, for example, when the object detection status determination unit 2510 determines, in accordance with the aircraft number increase determination conditions shown in Figure 12, that alert information regarding an object has been obtained from the external system 4000, or that a specified time period has arrived in which increased alertness is necessary, or when a command requesting an increase in actual aircraft has been obtained from a user via the user input receiving unit 3320 or the user terminal device 8000, etc.

[0148] In addition, the unmanned aircraft increase / decrease decision unit 2540 decides to increase the number of actual aircraft to be deployed in the operating area, for example, when the object detection state determination unit 2510 determines that an object 7000 or a candidate object for the object 7000 has been detected based on measurement data from a measurement sensor capable of measuring the object 7000 present in the operating area or its surrounding area, according to the aircraft number increase determination conditions shown in Figure 12, or when it determines that a value indicating the search performance, mobility performance, battery performance, or communication performance of the unmanned boat 1010 has decreased below a predetermined value.

[0149] In addition, the unmanned aircraft increase / decrease decision unit 2540 decides to reduce the number of actual aircraft deployed in the operational area, for example, in accordance with the aircraft number reduction determination conditions shown in Figure 12, when alert information previously acquired from the external system 4000 is lifted, or when an actual aircraft reduction request command requesting a reduction in the number of actual aircraft is acquired from a user via the user input acceptance unit 3320 or the user terminal device 8000, or when a detection flag indicating that the target object 7000 or a candidate object for the target object 7000 has been detected using measurement data from the measurement sensor 1110 is lifted.

[0150] In addition, the unmanned aircraft increase / decrease decision unit 2540 decides to reduce the number of actual aircraft deployed in the operation area when it detects, for example, that the wave conditions in the operation area have deteriorated beyond a predetermined value, or when it predicts that the future wave conditions in the operation area will deteriorate beyond a predetermined value, in accordance with the aircraft number reduction judgment conditions shown in Figure 12.

[0151] In addition, the unmanned aircraft increase / decrease decision unit 2540 decides to reduce the number of actual aircraft deployed in the operational area when, for example, the future state prediction determination unit 2530 determines, in accordance with the aircraft number reduction determination conditions shown in Figure 12, that an interference state will occur in the future in which the future relative distance between another moving body and the unmanned aircraft will be less than a predetermined distance.

[0152] The unmanned vehicle increase / decrease decision unit 2540 may have a function to decide changes to future operational requirements in addition to increasing or decreasing the number of unmanned vehicles as described above. In this case, the future operational requirements to be changed include the number of parent vehicles, the number of child vehicles, the unmanned vehicle distribution pattern, movement speed, movement path pattern, straight-line travel time, search area order, and operation schedules such as replacement. For example, the unmanned vehicle increase / decrease decision unit 2540 can change the operational conditions when the current or future operational status of the mixed unmanned vehicle group does not achieve the target information.

[0153] (A-4-3-6. Group control command determination unit 2600) The group control command determiner 2600 is a functional unit that determines operation commands for unmanned aerial vehicles including all or at least any of real vehicles, simulated real vehicles that simulate real vehicles in virtual space, and virtual vehicles, based on behavior information of the mixed unmanned aerial vehicle group that includes a mixture of simulated real vehicles and virtual vehicles generated by the real-time unmanned aerial vehicle group state simulator 2300. The group control command determiner 2600 includes a control command determiner 2610, a future command determiner 2620, a control state transition determiner 2630, and a real vehicle increase / decrease control command generator 2640.

[0154] The control command determination unit 2610 has the function of generating control commands (including at least one of commands for position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration) for each unmanned vessel in the mixed unmanned vessel group, which includes real and virtual vessels. The control command determination unit 2610 can generate control commands for the unmanned vessel 1010, which includes real and virtual vessels, so as to prevent future interference between the unmanned vessels in the mixed unmanned vessel group, as determined by the current state determination unit 2520 and the future state prediction determination unit 2530, for example.

[0155] In addition, the control command determination unit 2610 can generate control commands for the unmanned craft 1010, including real and virtual craft, not only to prevent interference between multiple unmanned crafts 1010, but also to prevent interference between the unmanned craft 1010 and other future moving bodies as determined by the current state determination unit 2520 and the future state prediction determination unit 2530.

[0156] As another example, the control command determination unit 2610 can generate control commands for the unmanned craft 1010, including the actual craft and the virtual craft, so as to prevent a future communication failure state between the unmanned crafts, as determined by the current state determination unit 2520 or the future state prediction determination unit 2530, from occurring.

[0157] As another example, the control command determination unit 2610 can generate control commands for unmanned craft, including real and virtual craft, so that the future operational state of the mixed unmanned craft group determined by the current state determination unit 2520 or the future state prediction determination unit 2530 achieves the specified target information included in the user request information, or so that the determined future operational state improves (for example, the search rate improves).

[0158] As another example, the control command determination unit 2610 can generate control commands for unmanned aircraft, including real and virtual aircraft, so as to prevent future wave area navigation states determined by the current state determination unit 2520 and the future state prediction determination unit 2530 from occurring.

[0159] The future command determination unit 2620 is a functional unit that generates future control commands for unmanned craft including real and virtual craft after a predetermined time has elapsed, based on the future navigation state of the mixed unmanned craft group including real and virtual craft determined by the future state prediction determination unit 2530.

[0160] Here, the future control commands generated by the future command determination unit 2620 are input to the future group control state simulator 2400, which predicts and calculates the future behavior state of the mixed unmanned aerial vehicle group including real and virtual vehicles, and the future state prediction and determination unit 2530 determines the future navigation state of the mixed unmanned aerial vehicle group. Then, the future command determination unit 2620 again generates future control commands for the unmanned aerial vehicles including real and virtual vehicles after a further predetermined time has passed. In this way, the future command determination unit 2620, the future group control state simulator 2400, and the future state prediction and determination unit 2530 can predict and calculate future behavior states and operation states in a time-accelerated manner.

[0161] The control state transition determination unit 2630 is a functional unit that determines state transitions of the control state of the operation system 1. The control state transition determination unit 2630 can determine, for example, control state transitions of search (status 301), standby (status 302), recovery charge (status 303), and post-discovery action (status 304) in the search operating state. Furthermore, the control state transition determination unit 2630 can determine more detailed state transitions (child modes) in search (status 301).

[0162] Figure 14 is a diagram showing an example of detailed state transitions in the search state determined by the control state transition determination unit 2630. In the example shown in Figure 14, when the parent mode becomes search (status 301), the child mode state becomes search operate / integrated environment parallel processing (status 3011). Search operate / integrated environment parallel processing is a control state in which processing of a search operate using a real device in the real space and processing in the integrated environment that calculates the behavior state of a mixed unmanned vehicle group in the virtual space, which is a mixture of simulated real devices and virtual devices, are performed in parallel.

[0163] Next, in the search operation / integrated environment parallel processing (status 3011), if the unmanned vehicle increase / decrease decision unit 2540 decides to increase the number of real vehicles deployed in the operation area, the control state transitions to status 3012. In status 3012, a pre-preparation command is issued to replace the virtual vehicle in the virtual space with a real vehicle. Here, the pre-preparation command includes, for example, an add real vehicle control command to control the movement of an additional real vehicle to be added to replace the virtual vehicle, with the position of the virtual vehicle or a peripheral position thereof as the movement target position.

[0164] Next, the pre-preparation command in status 3012 is executed, and when the movement of the additional real machine to replace the virtual machine to the position of the virtual machine or its surrounding position is completed, the control state transitions to status 3013. In status 3013, a replacement process is performed to replace the virtual machine in the virtual space with a real machine. Here, in the replacement process, for example, real-time unmanned aerial vehicle state simulator 2300 performs a process to replace the virtual machine in the virtual space with a simulated real machine that simulates the additional real machine.

[0165] In the status 3013 replacement process, the behavior information of the simulated real machine includes at least one of the following information: position, orientation, movement speed, movement direction, movement acceleration, rotational angular velocity, rotational angular velocity, battery charge state, and battery degradation state.

[0166] Next, when the replacement process of status 3013 is completed, the control state transitions to status 3014. In status 3014, a post-processing command is issued after the virtual machine in the virtual space has been replaced with the real machine. Here, the behavior information of the replaced simulated real machine can be updated using real machine behavior information acquired by the real machine information acquisition unit 2130, or updated using the behavior state of the simulated real machine calculated by the real machine behavior calculation unit 2320 based on the real machine behavior information. Therefore, in the post-processing, for example, a method for updating the behavior information of the simulated real machine as described above is determined. When the post-processing of status 3014 is completed, the control state returns to status 3011.

[0167] In the search operation / integrated environment parallel processing (status 3011), if the unmanned vehicle increase / decrease decision unit 2540 decides to reduce the number of real vehicles deployed in the operation area, the control state transitions to status 3015. In status 3015, a pre-preparation command is issued to retrieve the real vehicles in the real space to replace them with virtual vehicles in the virtual space, or to move the real vehicles to a waiting area. Here, the pre-preparation command is a processing command that includes, for example, determining the recovery location of the real vehicle to be recovered, the waiting area that will be the destination of the recovery, and the selection of the real vehicle to be recovered. The pre-preparation command may also include a command to generate a movement route within a wireless communication range from the real vehicle (parent vehicle) remaining in the operation area so that the real vehicle to be replaced with a virtual vehicle can maintain wireless communication with the real vehicle (parent vehicle) remaining in the operation area when returning to the waiting area. Furthermore, when multiple real vehicles are to be returned, the pre-preparation command may also include a command to generate a return order in which the recovered real vehicles are returned in order, starting with the most distant from the real vehicle (parent vehicle) remaining in the operation area.

[0168] Next, when a pre-preparation command is executed in status 3015, the control state transitions to status 3016. In status 3016, a replacement process is performed to replace the real vehicle with a virtual vehicle in the virtual space. Here, in this replacement process, for example, the real-time unmanned aerial vehicle swarm state simulator 2300 generates the simulated real vehicle and the virtual vehicle to be recovered in the virtual space as different ships separately, and calculates the behavior state of the virtual vehicle independently from the simulated real vehicle. In other words, the simulated real vehicle and the virtual vehicle are generated at the same position in the virtual space. Furthermore, the initial values ​​of the calculated behavior state of the virtual vehicle (position, orientation, movement speed, movement direction, movement acceleration, rotational angular velocity, rotational angular velocity, battery charge state, battery degradation state, etc.) can be the behavior information of the simulated real vehicle.

[0169] Next, when the replacement process is completed in status 3016, the control state transitions to status 3017. In status 3017, a post-processing command is issued after the real machine has been replaced with a virtual machine in the virtual space. Here, the post-processing command is a real machine recovery control command to the real machine to recover the real machine that has been replaced with the virtual machine at a predetermined location or move it to a predetermined waiting area. When the post-processing in status 3017 is completed, the control state returns to status 3011.

[0170] The unmanned aircraft increase / decrease control command generation unit 2640 is a functional unit that generates an actual aircraft increase / decrease control command for increasing or decreasing the number of actual aircraft when the unmanned aircraft increase / decrease decision unit 2540 decides to increase or decrease the number of actual aircraft deployed in the operation area. That is, it is possible to generate a pre-preparation command (additional actual aircraft control command, etc.) in status 3012 shown in Fig. 14, a replacement processing command in status 3013, a post-processing command in status 3014, a pre-preparation command in status 3015, a replacement processing command in status 3016, and a post-processing command (recovered actual aircraft control command, etc.) in status 3017. Specific examples and multiple patterns of control commands generated by the actual aircraft increase / decrease control command generation unit 2640 will be described later.

[0171] (A-4-3-7. Information output unit 2700) The information output unit 2700 is a functional unit that transmits and outputs various information (such as the behavior status of the mixed unmanned aerial vehicle group) acquired, generated, or determined by each functional unit of the integrated environment management system 2000 to the outside, or displays and outputs to the user via the operational application control system 3000 described below. The information output unit 2700 includes a determination status output unit 2710, an unmanned aerial vehicle behavior information output unit 2720, a future behavior information output unit 2730, and an actual vehicle measurement data output unit 2740.

[0172] The judgment status output unit 2710 transmits and outputs at least one of the judgment results determined by the current status judgment unit 2520 or the future status prediction judgment unit 2530, such as whether an interference state has occurred in which the relative distance between unmanned aircraft at present or in the future is less than a predetermined distance, whether an interference state has occurred in which the relative distance between another moving body and an unmanned aircraft at present or in the future is less than a predetermined distance, whether a communication failure state has occurred in which the wireless communication distance between unmanned aircraft at present or in the future is greater than the wireless communication possible distance, or whether the operational status of the mixed unmanned aircraft at present or in the future will achieve the pre-set goal included in the user request information, to the outside, or displays and outputs to the user via the operational application control system 3000 described later.

[0173] The unmanned vessel behavior information output unit 2720 transmits and outputs the current behavior information (position, orientation, movement direction, movement speed, movement history, measurement range, etc.) of each unmanned vessel in the mixed unmanned vessel group calculated by the real-time unmanned vessel group state simulator 2300 to an external device, or displays and outputs it to the user via the operational application control system 3000 described below.

[0174] The future behavior information output unit 2730 transmits and outputs the future behavior information (position, orientation, movement direction, movement speed, movement history, measurement range, etc.) of each unmanned vessel in the mixed unmanned vessel group calculated by the future group control state simulator 2400 to an external device, or displays and outputs it to the user via the operational application control system 3000 described below.

[0175] The actual aircraft measurement data output unit 2740 transmits and outputs to the outside the actual aircraft behavior information acquired by the actual aircraft information acquisition unit 2130 from the actual aircraft deployed in the operational area in the real space, or the actual aircraft measurement data measured by the measurement sensors mounted on the actual aircraft, or displays and outputs to the user via the operational application control system 3000 described below. The actual aircraft behavior information can include information such as the aircraft's position (two-dimensional or three-dimensional), movement speed, heading, movement direction, movement acceleration / deceleration, turning speed, other state quantities related to the navigation state, remaining energy and fuel in the battery mounted on the aircraft, possible movement distance that can be calculated from the remaining energy and fuel, temporary abnormal states of equipment mounted on the aircraft (temperature abnormality, communication abnormality, etc.), and equipment failure state.

[0176] In addition, the information output unit 2700 may have a function to output a command to change future operational requirements determined by the unmanned aircraft increase / decrease decision unit 2540, in addition to the above-mentioned information.

[0177] (A-5. Detailed functions of the operation application control system 3000) Next, detailed functions of the operational application control system 3000 will be described with reference to Fig. 11. Fig. 11 is a functional block diagram showing the functional configuration of the operational application control system 3000. The operational application control system 3000 has an information recording unit 3100, a display information output control unit 3200, and a user interface unit 3300.

[0178] (A-5-1. Information recording unit 3100) The information recording unit 3100 is a functional unit that acquires various information from the integrated environment management system 2000, the actual unmanned watercraft 1010, etc., and records the various information. The information recording unit 3100 includes a judgment state recording unit 3110, an unmanned watercraft behavior information recording unit 3120, a future behavior information recording unit 3130, and an actual watercraft measurement data recording unit 3140.

[0179] The determination state recording unit 3110 is a functional unit that records information on the determination results of the current state determination unit 2520 and the future state prediction determination unit 2530 obtained from the determination state output unit 2710 of the integrated environment management system 2000 .

[0180] The unmanned vessel behavior information recording unit 3120 is a functional unit that records the current behavior information of each unmanned vessel in the mixed unmanned vessel group calculated by the real-time unmanned vessel group state simulator 2300, which is obtained from the unmanned vessel behavior information output unit 2720 of the integrated environment management system 2000. The unmanned vessel behavior information recording unit 3120 may also record actual vessel behavior information obtained from the actual unmanned vessel 1010.

[0181] The future behavior information recording unit 3130 is a functional unit that records future behavior information of each unmanned craft in the mixed unmanned craft group calculated by the future group control state simulator 2400, which is obtained from the future behavior information output unit 2730 of the integrated environment management system 2000.

[0182] The actual aircraft measurement data recording unit 3140 is a functional unit that records actual aircraft behavior information and actual aircraft measurement data acquired from the actual aircraft measurement data output unit 2740 of the integrated environment management system 2000. The actual aircraft measurement data recording unit 3140 may also record actual aircraft behavior information and actual aircraft measurement data acquired directly from the unmanned watercraft 1010.

[0183] (A-5-2. Display information output control unit 3200) The display information output control unit 3200 is a functional unit that generates display information for displaying various information recorded in the information recording unit 3100 on a display unit 3310 of a user interface unit 3300 (described later), and controls the display information in accordance with input information from a user input receiving unit 3320. The display information output control unit 3200 includes a display information generation unit 3210 and a display information control unit 3220.

[0184] The display information generation unit 3210 generates display information to be displayed on the display unit 3310 based on the various information recorded in the information recording unit 3100. Here, an example of the display information generated by the display information generation unit 3210 will be described with reference to Figs. 15 to 17.

[0185] FIG. 15 is a diagram showing a first example of display information for the current behavior information of the mixed unmanned vehicle group generated by the display information generation unit 3210. In the example shown in FIG. 15, the display information generation unit 3210 generates display information that allows the user to grasp the relative positions of each unmanned vehicle in the operation area by integrating a map image or an aerial image with the current behavior information based on the position information of each unmanned vehicle included in the current behavior information of the mixed unmanned vehicle group. The display information can be generated in a top view format, looking down on the operation area from directly above as shown in the upper diagram of FIG. 15, or a tilt view format, looking down on the operation area from diagonally above as shown in the lower diagram of FIG. 15. The behavior information shown in FIG. 15 includes the current position of each unmanned vehicle as well as its past movement history, based on the past position information of each unmanned vehicle recorded in the unmanned vehicle behavior information recording unit 3120. The behavior information for each unmanned vehicle indicates whether it is a real vehicle or a virtual vehicle using color or text.

[0186] Next, FIG. 16 is a diagram showing a second example of display information for the current behavior information of the mixed unmanned vehicle group generated by the display information generation unit 3210. In the example shown in FIG. 16, similar to FIG. 15, the display information generation unit 3210 generates display information that allows the user to grasp the relative positions of each unmanned vehicle in the operation area by integrating a map image or aerial image with the current behavior information based on the position information of each unmanned vehicle included in the current behavior information of the mixed unmanned vehicle group. However, in the example shown in FIG. 16, the display information includes information on the measurement range measured by each unmanned vehicle based on information on the type and measurable range of the measurement sensor mounted on the unmanned vehicle previously acquired by the simulation condition acquisition unit 2120, etc. Note that, similar to FIG. 15, the display information shown in FIG. 16 can be generated in the form of a top view (upper view) looking down on the operation area from directly above, or a tilt view (lower view) looking down on the operation area from diagonally above. Note that the behavior information of each unmanned vehicle indicates whether the unmanned vehicle is a real vehicle or a virtual vehicle using color or text.

[0187] Next, Fig. 17 is a diagram showing a third example of display information of the current behavior information of the mixed unmanned vehicle group generated by the display information generation unit 3210. In the example shown in Fig. 17, the measurement range of each unmanned vehicle integrated with the aerial image shown in Fig. 16 is displayed in the top view and tilt view formats at the top of the display screen. In addition, the actual vehicle measurement data (camera images of the area around the unmanned vehicle) recorded in the actual vehicle measurement data recording unit 3140 is displayed at the top right of the display screen.

[0188] Furthermore, detailed current behavior information of the unmanned craft selected with a cursor or the like on the display screen is displayed in the form of a dashboard or the like at the bottom of the screen shown in Fig. 17. In the example shown in Fig. 17, the behavior information displayed includes hull status (stopped), mission status (none), remaining battery charge, movement speed, attitude (machine direction, roll, pitch, yaw), thruster operation status (output %, rotation speed, voltage, current), PDU voltage, PDU current, rudder, battery temperature, etc.

[0189] (A-5-3. User interface unit 3300) The user interface unit 3300 is a functional unit that displays and outputs each piece of display information generated by the display information output control unit 3200, and also receives input information from the user. The user interface unit 3300 includes a display unit 3310 and a user input receiving unit 3320.

[0190] The display unit 3310 has a function of displaying and outputting each piece of display information generated by the display information output control unit 3200. For example, the display unit 3310 displays and outputs the display information shown in Fig. 15 to 17. However, without being limited to this, the display unit 3310 can also display and output information on the determination results of the current state determination unit 2520 and the future state prediction determination unit 2530 obtained from the determination state output unit 2710 of the integrated environment management system 2000, which are recorded in the information recording unit 3100, future behavior information of each unmanned watercraft in the mixed unmanned watercraft group calculated by the future group control state simulator 2400, and the like.

[0191] The user input receiving unit 3320 is a functional unit that receives user input for each piece of information displayed by the display unit 3310 or other user input. The user input information can be received by selecting a tab on the display screen of the display unit 3310, thereby changing the display information and display format.

[0192] Fig. 18 is a diagram showing an example of the display switching tabs displayed on the display unit 3310. As shown in Fig. 18, the user can select the desired display information and display format by selecting a display switching tab on the display screen of the display unit 3310. In Fig. 18, the real-time tab is selected, and the position, orientation, and movement history of each unmanned watercraft are displayed as current behavior information of the mixed unmanned watercraft.

[0193] When the patrol tab shown in Fig. 18 is selected, it is possible to display information showing the measurement range of each unmanned vessel as shown in Fig. 16. When the camera view tab is selected, it is possible to display the actual vessel measurement data (camera images around the unmanned vessel) recorded in the actual vessel measurement data recording unit 3140 shown in the upper right part of Fig. 17. When the future tab is selected, it is possible to display detailed information in a dashboard format as shown in Fig. 17.

[0194] Furthermore, the user input accepting unit 3320 can accept requests and commands from the user during the execution of operations of the mixed unmanned aerial vehicle group, such as requests to increase or decrease the number of actual aerial vehicles. Furthermore, the user input accepting unit 3320 can accept setting inputs such as target information and aircraft increase / decrease determination conditions included in user request information, or can accept input information that changes already set target information and aircraft increase / decrease determination conditions during the execution of operations of the mixed unmanned aerial vehicle group.

[0195] (A-6. Control flow of operation system 1) 19 to 24, the control flow of the operation system 1 will be described below. FIG. 19 is a diagram showing an example of the upper control processing flow of the operation system 1.

[0196] First, the information acquisition unit 2100 acquires the pre-settings and real-time information (step 101). The detailed processing of this step will be explained later with reference to FIG.

[0197] Next, the real-time unmanned aerial vehicle swarm state simulator 2300, the future swarm control state simulator 2400, and the state determination unit 2500 determine the detection state of the object and the current and future operation states (step 102). The detailed processing of this step will be explained later with reference to FIG. 21.

[0198] Next, the group control command decision unit 2600 and the information output unit 2700 decide on a control command and output it for transmission (step 103). The detailed processing of this step will be explained later with reference to Figs.

[0199] Next, the operational application control system 3000 generates and displays display information (step 104).

[0200] (A-6-1. Processing flow for obtaining advance information and real-time information) 20 is a diagram showing an example of a control processing flow for acquiring advance information and real-time information by the information acquisition unit 2100. In particular, FIG. 20 shows detailed processing of the processing of step 101 shown in FIG.

[0201] First, the user request information acquisition unit 2110 acquires user request information including desired conditions required by the user for the actual operation (step 201). In this step, the user request information acquired includes, for example, desired condition information, target information, and aircraft increase / decrease determination conditions as shown in Fig. 12 .

[0202] Next, assumed condition parameters are acquired by the simulation condition acquisition unit 2120 (step 202). In this step, various condition parameters assumed when operating the unmanned boat system 1000 in an operation area such as that shown in FIG. 13 are acquired.

[0203] Next, various types of actual aircraft information are acquired from the actual aircraft by the actual aircraft information acquisition unit 2130 (step 203). In this step, for example, actual aircraft behavior information determined by the actual aircraft navigation state determination unit 1210 and internal state determination unit 1220, actual aircraft measurement data measured by the actual aircraft measurement unit 1100, and external state information determined by the actual aircraft external state determination unit 1230 are acquired.

[0204] Next, the environmental information acquisition unit 2140 acquires environmental information about the operation area and its surrounding area from the external system 4000 (step 204). In this step, the environmental information acquired includes, for example, meteorological conditions, oceanographic information, other seawater conditions, sun-related information and other conditions, ionospheric disturbances (such as solar flares), and the like.

[0205] Next, the external information acquisition unit 2150 acquires information such as ship information in the operation area and its surrounding area from the external system 4000 (step 205). In this step, for example, ship information about ships and fishing boats around the operation area, information about future operation schedules, or information about other events currently taking place or scheduled in the operation area and its surrounding area are acquired.

[0206] (A-6-2. Control processing flow for determining detection status and operational status) 21 is a diagram showing an example of a control processing flow for determining the detected state and the operating state by the state determination unit 2500 etc. In particular, FIG. 21 shows detailed processing of the processing of step 102 shown in FIG.

[0207] First, the object detection state determination unit 2510 determines the object detection state, including the prediction of the appearance of the search object and the initial detection of the object (step 301). In this step, for example, when alarm information about the object is acquired from the external system 4000, or when a predetermined time period requiring increased alert has arrived, and a candidate object for the object 7000 is detected based on measurement data measured by the measurement sensor 1110 mounted on the actual unmanned watercraft, it can be determined that the object is in the prediction detection state. Furthermore, when the object 7000 is detected based on the measurement data, it can be determined that the object is in the initial detection state. Furthermore, it can be determined that the alarm information acquired from the external system 4000 has been canceled, or that the detection flags for the prediction detection state in which a candidate object for the object 7000 has been detected and the initial detection state in which the object 7000 has been detected have been canceled.

[0208] Next, the real-time unmanned aircraft group state simulator 2300 calculates the current behavior state of the mixed aircraft group, and based on the current behavior state of the mixed aircraft group, the current state determination unit 2520 determines the current operating state of the mixed aircraft group (step 302). In this step, it is possible to determine, for example, whether an interference state has occurred in which the current relative distance between unmanned vessels in the mixed unmanned vessel group, or the current relative distance between another moving body and the unmanned vessel 1010, is below a predetermined collision avoidance safe distance; whether a communication failure state has occurred in which the current relative distance between unmanned vessels in the mixed unmanned vessel group is above the communication distance between the unmanned vessels; whether the current operating state of the mixed unmanned vessel group satisfies the specified target information included in the user request information; whether a wave area navigation state has occurred in which the relative distance on the horizontal plane between a typhoon, low pressure system, or rain cloud and the unmanned vessel is below a predetermined distance (i.e., a state in which the unmanned vessel is navigating a wave area where waves of a predetermined magnitude or greater are expected due to the influence of a typhoon, low pressure system, or rain cloud); or whether a value indicating the unmanned vessel's search performance, mobility performance, battery performance, or communication performance has dropped below a predetermined value.

[0209] Next, the state determination unit 2500 determines the environmental state in the current operation area based on the environmental measurement data acquired from the actual machine information acquisition unit 2130 or the environmental information acquired from the environmental information acquisition unit 2140 (step 303). This step makes it possible to determine that the current wave state in the operation area has deteriorated below a predetermined value.

[0210] Next, the future behavior state of the mixed aircraft group is calculated by the future group control state simulator 2400, and based on the future behavior state of the mixed aircraft group, the future state prediction determination unit 2530 determines the future operating state of the mixed aircraft group (step 304). In this step, for example, it is determined whether a future interference state will occur in which the future relative distance between unmanned vessels in the mixed unmanned vessel group, or the future relative distance between other moving bodies and the unmanned vessel 1010, will be less than a specified collision avoidance safe distance; whether a future communication failure state will occur in which the future relative distance between unmanned vessels in the mixed unmanned vessel group will be greater than the communication distance between the unmanned vessels; whether the future operating state of the mixed unmanned vessel group satisfies the specified target information included in the user request information; whether a wave area navigation state will occur in the future in which the relative distance on the horizontal plane between a typhoon, low pressure system, or rain cloud and the unmanned vessel will be less than a specified distance (i.e., a state in which the unmanned vessel navigates in a wave area where waves of a specified magnitude or greater are expected due to the influence of a typhoon, low pressure system, or rain cloud); or whether values ​​indicating the unmanned vessel's future search performance, mobility performance, battery performance, or communication performance have fallen below a specified value.

[0211] Next, the state determination unit 2500 determines the future environmental state in the operation area based on the environmental measurement data acquired from the actual machine information acquisition unit 2130 or the environmental information acquired from the environmental information acquisition unit 2140 (step 305). This step makes it possible to determine that the future wave state in the operation area has deteriorated below a predetermined value.

[0212] (A-6-3. Control processing flow for unmanned boats) 22 is a diagram showing an example of a control processing flow when control of unmanned crafts is performed by the group control command determination unit 2600 etc. Fig. 22 particularly shows detailed processing of step 103 shown in Fig. 19.

[0213] First, the unmanned vehicle increase / decrease decision unit 2540 decides whether to increase or decrease the number of actual vehicles to be deployed in the operation area (step 401). In this step, if it is decided to increase the number of actual vehicles, the process proceeds to step 402; if it is decided that no increase or decrease in the number of actual vehicles is necessary, the process proceeds to step 405; and if it is decided to decrease the number of actual vehicles, the process proceeds to step 406.

[0214] Next, if it is decided to increase the number of real machines in step 401, the real machine increase / decrease control command generation unit 2640 calculates the conditions for switching from virtual machines to real machines (step 402). The details of this step will be described later.

[0215] Next, the actual machine increase / decrease control command generating unit 2640 controls the replacement of the virtual machine with the actual machine (step 403).

[0216] Next, the real machine increase / decrease control command generating unit 2640 performs a synchronization process to rewrite the behavior information of the virtual machine in the virtual space with the behavior information of the replaced real machine (step 404).

[0217] Next, if it is determined in step 401 that no increase or decrease in the number of real aircraft is necessary, the control command determination unit 2610 issues a control command to the mixed unmanned aircraft group including real aircraft and virtual aircraft (step 405). In this step, a control command is issued to control the behavior of the mixed unmanned aircraft group so as to prevent current or future interference between unmanned aircraft in the mixed unmanned aircraft group, interference between other moving bodies and the unmanned aircraft 1010, communication failure between unmanned aircraft, and navigation in a rough wave area, and so that the operational status of the mixed unmanned aircraft group achieves specified target information or improves the determined future operational status.

[0218] Next, if it is decided in step 401 to reduce the number of real machines, the real machine increase / decrease control command generation unit 2640 calculates conditions for switching from real machines to virtual machines (step 406). In this step, for example, the collection location of the real machine to be collected, the waiting area to be the destination of the movement, and the selection of the real machine to be collected are determined. In addition, a movement route within the wireless communication range of the real machine (parent machine) is calculated so that the real machine to be replaced with a virtual machine can return to the waiting area while maintaining wireless communication with the real machine (parent machine) remaining in the operation area.

[0219] Next, the real machine increase / decrease control command generator 2640 cancels the synchronization process of the behavior information between the real machine and the virtual machine to be replaced, and starts the process of calculating the behavior state of the virtual machine to be replaced with the real machine independently from the behavior information of the real machine (step 407).

[0220] Next, the actual machine increase / decrease control command generator 2640 returns the actual machine to be replaced with the virtual machine to the waiting area (step 408). In this step, the actual machine to be returned is moved so as to pass through the wireless communication range of the actual machine (parent machine) remaining in the operation area. Furthermore, when multiple actual machines are returned, the actual machines are returned in order starting from the one farthest from the actual machine (parent machine) remaining in the operation area.

[0221] (A-6-3-1. Control method for switching from virtual machine to real machine) Next, a detailed control method for calculating conditions for switching from a virtual machine to a real machine by the real machine increase / decrease control command generator 2640 shown in Fig. 22 will be described with reference to Fig. 23 and Fig. 24. Fig. 23 is a diagram showing an example of a method for controlling switching from a virtual machine to a real machine by the real machine increase / decrease control command generator 2640. Fig. 23 particularly shows detailed processing of step 402 shown in Fig. 22.

[0222] Steps 501 and 502 in the flowchart shown in FIG. 23 indicate the process of calculating the control conditions for maintaining wireless communication with the parent device (real device) in the operation area when the real device to be replaced with the virtual device is moved from the standby area to the position of the virtual device in the operation area, and steps 503 to 505 indicate the process of determining the command when replacing the real device with the virtual device.

[0223] First, the unit 2640 calculates the control conditions for each unmanned watercraft to utilize the units (child units) already deployed in the operation area to perform a relay operation to relay wireless communications between the parent unit (child unit) in the operation area and the replacement unit (step 501). The control conditions calculated in this step include, for example, the number of deployed units (child units) to be moved and the positions to which they should be moved when controlling the movement of the units (child units) already deployed in the operation area to a position between the parent unit (child unit) in the operation area and the replacement unit.

[0224] Next, the unit 2640 calculates the control conditions for each unmanned watercraft to newly dispatch a unit for ensuring communication that will perform a relay operation to relay wireless communications between the parent unit (unit) in the operation area and the replacement unit (step 502). The control conditions calculated in this step include, for example, the number of units for ensuring communication to be dispatched and the positions to which they should be moved, in a control to dispatch a unit for ensuring communication to a position between the parent unit (unit) in the operation area and the replacement unit.

[0225] Next, based on the control conditions calculated in steps 501 and 502, a determination is made as to whether additional deployment of actual devices for ensuring communication is necessary (step 503). If it is determined that additional deployment of actual devices for ensuring communication is necessary, the process proceeds to step 505. On the other hand, if it is determined that additional deployment of actual devices for ensuring communication is not necessary, the process proceeds to step 504.

[0226] Next, in step 503, if it is determined that additional deployment of a real device for ensuring communication is not necessary, the control conditions for performing a relay operation to relay wireless communication between the parent device (real device) in the operating area and the replacement real device are determined as an execution command (step 504) by utilizing the real device (child device) currently deployed.

[0227] Next, in step 503, if it is determined that additional deployment of a real device for ensuring communication is necessary, the control conditions for dispatching a new real device for ensuring communication that will perform relay operations to relay wireless communications between the parent device (real device) in the operating area and the replacement real device are determined as an execution command (step 505).

[0228] (A-6-3-2. Additional deployment control calculation method for actual aircraft for ensuring communications) Next, Fig. 24 is a diagram showing an example of a method for calculating control conditions when additionally deploying a real machine for ensuring communication by the real machine increase / decrease control command generator 2640. Fig. 24 particularly shows detailed processing of step 502 shown in Fig. 23.

[0229] First, the actual device increase / decrease control command generation unit 2640 calculates the deployment position of the actual device for ensuring communication (step 601). In this step, for example, the deployment position of the actual device for ensuring communication can be set to a position that belongs to both the communication area from the parent device (actual device) in the operation area and the communication area of ​​the replacement actual device.

[0230] Next, the actual aircraft increase / decrease control command generation unit 2640 calculates the number of actual aircraft to be deployed for ensuring communication (step 602). In this step, for example, when the actual aircraft for ensuring communication are deployed in a communication-enabled area from a parent aircraft (actual aircraft) in the operation area to relay communication, the number of actual aircraft for ensuring communication required for relaying communication to the replacement actual aircraft is calculated.

[0231] Here, in the processing of steps 601 and 602, there are two possible patterns: a first pattern in which the actual device for ensuring communication is replaced with the virtual device (i.e., dispatched to the location of the virtual device), and a second pattern in which the actual device for ensuring communication is dispatched to a location unrelated to the virtual device. In the processing of steps 601 and 602, the locations and number of actual devices for ensuring communication are calculated using either the first pattern or the second pattern. Specific examples of the first pattern and the second pattern will be described later.

[0232] Next, the actual device increase / decrease control command generating unit 2640 generates a movement command for the actual device for ensuring communication (step 603). In this step, for example, when deploying multiple actual devices for ensuring communication, the movement command is to move the actual devices for ensuring communication in order of the deployment position closest to the parent device, and further, the movement command is to make the movement route of the actual device for ensuring communication pass through the communication coverage areas of the parent device (actual device) in the operation area and other actual devices that have been deployed earlier.

[0233] Next, the actual machine increase / decrease control command generating unit 2640 generates a movement command for the actual machine that will replace the virtual machine (step 604). In this step, for example, the movement command is generated to move the replacement actual machine after the actual machine for ensuring communication arrives at the deployment position or in parallel with the movement of the actual machine for ensuring communication, and further, the movement command is generated so that the movement route of the replacement actual machine passes through the communication coverage area of ​​the parent machine (actual machine) in the operation area and the actual machine for ensuring communication that has been deployed earlier.

[0234] (A-7. Specific example of control by the control command determination unit 2610) Next, a specific example of control by the control command determination unit 2610 will be described with reference to FIGS.

[0235] (A-7-1. Communication Maintenance Control by the Control Command Determining Unit 2610) Fig. 25 is a diagram showing how the movement of the real machine and the virtual machine is controlled so as to maintain the communication maintenance range by the control command determination unit 2610. Fig. 25 particularly shows the behavioral states (position, orientation, movement history) of the real machine and the virtual machine in chronological order at times T1, T2, and T3.

[0236] At time T1 shown in the upper diagram of Figure 25, the real machine located in the center and the virtual machines in the periphery are controlled to operate in cooperation so that the search performance exceeds a predetermined target value, so as to prevent interference caused by the real machine becoming too close to the virtual machine.

[0237] 25 shows a situation where one of the virtual machines is instructed to move to a distant position at the bottom left at time T2. In such a case, at time T3 in the bottom diagram, the parent machine (real machine) that communicates with the virtual machine at the bottom left or another virtual machine is controlled to move closer to the virtual machine at the bottom left so that the virtual machine at the bottom left does not deviate from the communication range.

[0238] As shown in Figure 25, the control command determination unit 2610 generates control commands for each unmanned vessel in the mixed unmanned vessel group, including real and virtual vessels, so that the unmanned vessels in the mixed unmanned vessel group do not deviate from the communication range.

[0239] (A-7-2. Interference Avoidance Control with Other Ships by the Control Command Determination Unit 2610) Fig. 26 is a diagram showing how the movement of the real and virtual machines is controlled to avoid interference with other ships by the control command determination unit 2610. Fig. 26 particularly shows the behavioral states (position, orientation, movement history) of the real and virtual machines over time at times T1 and T2.

[0240] At time T1 shown in the upper diagram of Figure 26, based on the sailing schedule information of ships sailing in the surrounding area obtained from the external information acquisition unit 2150, a future interference state in which the relative distance between the ship and the unmanned boat will be less than a predetermined distance is predicted, and a movement command is issued to the actual machine and virtual machine that are predicted to be in an interference state in the future.

[0241] At time T2 shown in the lower diagram of FIG. 26, the actual machine and the virtual machine are moved in accordance with the movement command to avoid interference with the ship.

[0242] As shown in Fig. 26, the control command determination unit 2610 generates control commands for each unmanned vessel in the mixed unmanned vessel group so that any unmanned vessel in the mixed unmanned vessel group will not be in an interference state with other vessels at a future time. Note that in Fig. 26, the future interference state is determined based on the vessel's navigation schedule information, but this is not limiting; the future movement path of the vessel may also be predicted based on the vessel's current position and movement direction to determine the future interference state.

[0243] (A-7-3. Wave Area Avoidance Control by the Control Command Determination Unit 2610) Fig. 27 is a diagram showing how the movement of the real machine and the virtual machine is controlled to avoid the wave area by the control command determination unit 2610. Fig. 27 particularly shows the behavioral states (position, orientation, movement history) of the real machine and the virtual machine in chronological order at times T1 and T2.

[0244] At time T1 shown in the upper diagram of Figure 27, based on environmental information including current or future weather and sea conditions in the operation area and its surrounding areas obtained from the environmental information acquisition unit 2140, rough wave areas where wave conditions are expected to be rougher than specified 6 hours and 12 hours into the future are predicted, and movement commands are issued to the real and virtual aircraft to avoid the rough wave areas.

[0245] At time T2 shown in the lower diagram of FIG. 27, the real machine and the virtual machine are moved in accordance with the movement command, and the state after they have been moved to positions that avoid the rough wave area is shown.

[0246] As shown in Figure 27, the control command determination unit 2610 generates control commands for each unmanned boat in the mixed unmanned aircraft group so that the unmanned boat avoids rough wave areas where waves of a certain level or higher are expected due to the influence of a typhoon, low pressure, rain clouds, etc.

[0247] (A-8. Specific Example of Actual Machine Increase / Decrease Control by Actual Machine Increase / Decrease Control Command Generator 2640) Next, a specific example of control by the actual machine increase / decrease control command generating unit 2640 when the actual machines are increased will be described with reference to FIGS.

[0248] (A-8-1. First form of additional control of actual equipment for ensuring communications) Fig. 28 is a diagram showing a first mode of real machine addition control using a real machine for ensuring communication by the real machine increase / decrease control command generation unit 2640. Fig. 28 shows the time-series behavior states (position, orientation, movement history) of the real machine and the virtual machine at times T1, T2, T3, and T4.

[0249] At time T1 shown in the upper diagram of Figure 28, the actual vehicle (operational vehicle) is deployed near the center of the operational area, and the other unmanned vessels are virtual vehicles. The communication range between the actual vehicle and the vehicle is indicated by a dotted circle. In addition, multiple actual vehicles (standby vehicles) are waiting in the coastal area (standby area) at the upper right of the operational area, and preparations are being made for dispatching the actual vehicles, such as swapping them for virtual vehicles.

[0250] At time T2 shown in the second row of Fig. 28, a command is acquired to replace the virtual machine in the lower left of the operation area with a real machine. The real machine increase / decrease control command generator 2640 determines whether the virtual machine to be replaced and the real machine in the center are located within a communication range of each other. In the example shown in Fig. 28, the location of the virtual machine to replace the real machine is outside the wireless communication range of the real machine (operation machine), so it is determined that a real machine needs to be dispatched to ensure communication.

[0251] At time T3 shown in the third row of Fig. 28, an additional real machine control command is generated to move the real machine for ensuring communication to a position that is within the wireless communication range of the deployed real machine (operation machine) and within the wireless communication range of the virtual machine (the real machine replacement target), and that is the current location of the virtual machine as a movement target position, and the real machine for ensuring communication is moved in accordance with the additional real machine control command. Note that it is desirable to set a movement command such that the movement route of the real machine for ensuring communication passes through the communication areas of the real machine (operation machine) in the operation area and other real machines that have been deployed earlier.

[0252] At time T4 shown in the fourth row of Figure 28, the real machine has completed moving to the position of the virtual machine (the real machine replacement target), and the real-time unmanned aerial vehicle state simulator 2300 rewrites the behavior state of the virtual machine with the real machine behavior information of the replaced real machine, thereby performing a synchronization process of the behavior state information, thereby completing the process of replacing the virtual machine with the real machine. Here, it is desirable that the movement route of the replaced real machine be a movement command that passes through the communication area of ​​the real machine (operational machine) in the operation area and the real machine for ensuring communication that has been deployed earlier.

[0253] As described above, in the example shown in Figure 28, a control form is shown in which the actual device for ensuring communication is moved to the target movement position at time T3, and then the actual device is moved to the position of the virtual device (the object of real device replacement).However, this is not limited to this, and the operation of moving the actual device for ensuring communication to the target movement position at time T3 and the operation of moving the replacement actual device to the position of the virtual device (the object of real device replacement) at time T4 can also be performed in parallel.

[0254] During the period when control to replace the virtual machine with the actual machine is being executed (between times T2 and T4), the actual machine increase / decrease control command generation unit 2640 can temporarily stop or almost stop the operation of the virtual machine (the actual machine replacement target) or all unmanned boats, and when the movement of the actual machine to the position of the virtual machine (the actual machine replacement target) or its surrounding position is completed, perform synchronization processing of the behavior status information at time T4.

[0255] As another example, during the period when control to replace a virtual machine with a real machine is being executed (between times T2 and T4), the virtual machine (the real machine to be replaced) or all unmanned watercraft continue to operate, and the real machine increase / decrease control command generation unit 2640 generates a control command to match the position, orientation, and speed of the replaced real machine with the position, orientation, and speed of the virtual machine (the real machine to be replaced), and when the position, orientation, and speed of the replaced real machine match or approximately match the position, orientation, and speed of the virtual machine (the real machine to be replaced), synchronization processing of the behavior status information at time T4 can be performed.

[0256] As yet another example, during the period when control to replace a virtual machine with a real machine is being executed (between times T2 and T4), the real machine increase / decrease control command generation unit 2640 moves the virtual machine (the real machine replacement target) in a predetermined movement pattern, and further generates a control command to match the position, orientation, and speed of the replacement real machine with the position, orientation, and speed of the virtual machine (the real machine replacement target), and when the position, orientation, and speed of the replacement real machine match or approximately match the position, orientation, and speed of the virtual machine (the real machine replacement target), it can perform synchronization processing of the behavior state information at time T4.

[0257] (A-8-2. Second form of additional control of actual equipment for ensuring communications) Fig. 29 is a diagram showing a second mode of real machine addition control using a real machine for ensuring communication by the real machine increase / decrease control command generation unit 2640. Like Fig. 28, Fig. 29 shows the time-series behavior states (position, orientation, movement history) of the real machine and virtual machine at times T1, T2, T3, and T4.

[0258] The times T1 and T2 shown in the upper diagram of FIG. 29 indicate the same states as the times T1 and T2 in FIG.

[0259] At time T3 shown in the third row of Fig. 29, unlike the example shown in Fig. 28, an additional real device control command is generated to dispatch one or more real devices for ensuring communication to a location where no virtual device is currently located, and the real devices for ensuring communication are moved in accordance with the additional real device control command. In the example shown in Fig. 29, relay communication from the real device (operational device) is performed by dispatching two real devices for ensuring communication. Here, when deploying multiple real devices for ensuring communication, it is desirable to issue a movement command to move the real devices for ensuring communication in order of their deployment target position being closest to the real device (operational device), and further to issue a movement command such that the movement route of the real devices for ensuring communication passes through the communication-enabled areas of the real devices (operational device) in the operation area and other real devices that have been deployed earlier.

[0260] At time T4 shown in the fourth row of Figure 29, the real machine has completed moving to the position of the virtual machine (the real machine replacement target), and the real-time unmanned aerial vehicle state simulator 2300 rewrites the behavior state of the virtual machine with the real machine behavior information of the replaced real machine, thereby performing a synchronization process of the behavior state information, thereby completing the process of replacing the virtual machine with the real machine. Here, it is desirable that the movement route of the replaced real machine be a movement command that passes through the communication area of ​​the real machine (operational machine) in the operation area and the real machine for ensuring communication that has been deployed earlier.

[0261] (A-9. Implementation example of the Integrated Environment Management System 2000, etc.) Next, with reference to FIGS. 30 and 31, an implementation example in which the functional units of the converged environment management system 2000 and operational application control system 3000 described so far are implemented on a cloud server, a workstation, or the like will be described.

[0262] (A-9-1. Implementation example of the Integrated Environment Management System 2000) FIG. 30 is a diagram showing an example of an implementation in which the functional units of the converged environment management system 2000 and the operational application control system 3000 are implemented on a cloud server, a workstation, or the like.

[0263] The hardware shown in Figure 30 mainly includes a control station, a cloud server, a workstation, a HILS, communication equipment, and actual equipment. In the example shown in Figure 30, the functional units of the integrated environment management system 2000 and the operational application control system 3000 are implemented in the control station, cloud server, workstation, and HILS. The communication equipment and actual equipment in Figure 30 correspond to the communication satellite 5100, the terrestrial base station 5200, and the unmanned boat system 1000, respectively, shown in Figure 1 etc.

[0264] The control station in FIG. 30 includes a UI device that outputs display information to users of the integrated environment management system 2000 and the operational application control system 3000 and accepts user input information, and a web browser application running on the UI device. The web browser has a function for selecting information to be displayed from various display information on a website, and the user can select information to be displayed on the display screen of the UI device via the UI device. The UI device can also accept user request information (including desired condition information, target information, aircraft increase / decrease determination conditions, etc.) as shown in FIG. 12 and assumed condition parameters as shown in FIG. 13 from the user and transmit them to the website. Here, the UI device corresponds to the user interface unit 3300 shown in FIG. 11, etc., and the web browser has some of the functions of the display information output control unit 3200 shown in FIG. 11 (particularly, a display control function for selecting information to be displayed from various display information on a website).

[0265] The IoT Gateway implemented in the cloud server in Fig. 30 sends control commands to real machines deployed in the operation area via communication facilities and receives telemetry data (i.e., real machine behavior information, etc.) from the real machines. Furthermore, like the real machines, it sends control commands to a workstation implemented with a virtual machine simulator that calculates the behavior of a virtual machine (described later) by simulation, and receives telemetry data (i.e., virtual machine behavior calculation information, etc.) from the real machines. Here, the IoT Gateway corresponds to the function of the real machine information acquisition unit 2130 of the information acquisition unit 2100 shown in Fig. 10.

[0266] The telemetry data (real machine behavior information, virtual machine behavior calculation information, etc.) acquired by the IoT Gateway is analyzed by data analysis software to generate behavior information of a mixed machine group that is a mixture of real machines and virtual machines, and the analysis results (behavior information of the mixed machine group, etc.) are recorded in a data lake or DB. Here, the data analysis software corresponds to the function of the mixed machine group behavior calculation unit 2330. Furthermore, the data lake and DB correspond to the function of the information recording unit 3100.

[0267] The data analysis software also performs data analysis and statistical processing on the behavior information of the mixed vehicle group stored in the data lake to generate visualized data to display to users. The data analysis software generates display information that overlays the behavior information of the mixed vehicle group on aerial images or maps of the operation area, and display information that displays the behavior status and internal status of each unmanned vehicle, including real and virtual vehicles, in dashboard format, and uploads the generated visualized data to a website. Here, the data analysis software corresponds to the function of the display information generation unit 3210 of the display information output control unit 3200.

[0268] The website is also a functional unit that provides various data, including visualization data generated by the data analysis software, to the UI device. The website can record user request information and assumed condition parameters acquired from the UI device in a DB using a data storage API. The website is also a functional unit that reads various information, such as behavior information and navigation status of the mixed aircraft group recorded in the DB, and previously recorded user request information and assumed condition parameters, from the DB using a data acquisition API, generates display information based on the read information, and provides it to the UI device. Here, the website corresponds to the function of the display information generation unit 3210 of the display information output control unit 3200.

[0269] The group control software also has the function of using a data acquisition API to read various information from the DB, such as behavior information, operational status, operational requirements, previously recorded user request information, and assumed condition parameters of the mixed vehicle group, and, based on this information, estimates various performance characteristics of the unmanned vehicle, determines future interference between unmanned vehicle vehicles, determines the occurrence of communication disruption, determines whether the goal has been achieved, and determines operational commands for the unmanned vehicle. Based on the user request information and assumed condition parameters read from the DB, the group control software can obtain information on related group control scenarios and patterns from the container registry and determine operational commands for the unmanned vehicle. Here, the group control software corresponds to the functions of the state determination unit 2500 and the group control command determination unit 2600.

[0270] In addition, the command transmission API sends operational commands generated by the group control software to the actual devices and virtual device simulators via the IoT Gateway.

[0271] The virtual machine simulator is simulator software installed on a workstation, and simulates and calculates behavior calculation information, simulated measurement data, internal states, etc. of a virtual machine in a virtual space based on an operation command received via the IoT Gateway. Here, the virtual machine simulator can calculate behavior calculation information, simulated measurement data, internal states, etc. of a virtual machine using a virtual machine model in which all functional units constituting the virtual machine are simulated in a virtual space, but can also simulate and calculate behavior calculation information, simulated measurement data, internal states, etc. of a virtual machine using a HILS machine in which some functional units constituting the virtual machine are configured with actual hardware and the other functional units are configured with virtual models.

[0272] (A-9-2. HILS implementation example) Next, Fig. 31 is a diagram showing an example of an implementation of HILS connected to a virtual machine simulator, which is partly configured with hardware devices and partly configured with virtual models.

[0273] In the HILS implementation example shown in Figure 31, the onboard processing unit, hull behavior control device, and power control device shown on the left side of the figure are composed of hardware devices connected to each other via wired or wireless communication. Also, the communication device, sonar, camera, LiDAR, sensor simulation, hull behavior simulation, solar cell model, and battery model shown on the right side of the figure are composed of virtual models.

[0274] The power control device sends a current command for the solar cell and a current limit command for the battery to the HILS device, and the HILS device sends solar cell output voltage information calculated based on a solar cell model and battery SOC information calculated based on a battery model to the power control device.

[0275] The hull behavior device sends rudder angle commands for the attitude control mechanism 1320 and motor rotation speed commands for the thrust generation unit 1310 (propellers, etc.) to the HILS device. The hull behavior simulation unit in the HILS device calculates through simulation the hull behavior of the virtual aircraft when rudder angle commands and motor rotation speed commands are input, based on the virtual aircraft model, solar cell model, and battery model. Behavior calculation information related to hull behavior, including the calculated heading, position, angular velocity, and acceleration, is sent from the HILS device to the hull behavior device.

[0276] Furthermore, the sensor simulation unit in the HILS system generates virtual measurement data measured by each sensor based on sensor model information that simulates each sensor (communication device, sonar, camera, and LiDAR) and behavior calculation information calculated by the hull behavior simulation unit. This virtual measurement data is transmitted from the HILS system to the onboard processing device. Here, as a modified example, instead of using the sensor model information, the sensor simulation unit may use the actual sensor to generate simulated information (e.g., optical image information captured by a camera) of the information measured by the sensor.

[0277] The on-board processing device, which is a hardware device, receives simulated measurement data measured by each sensor from the HILS device. The on-board processing device also acquires behavior calculation information related to hull behavior and internal state information including solar cell output voltage information and battery SOC information from the hull behavior device and power control device. The on-board processing device transmits the generated behavior calculation information, simulated measurement data, internal state information, etc. of the virtual machine to the virtual machine simulator.

[0278] (A-10. Example of hardware configuration) 32 is a diagram showing an example of the hardware configuration of the integrated environment management system 2000 and the operational application control system 3000. Here, the integrated environment management system 2000 and the operational application control system 3000 according to the present invention are information processing devices such as a server device or a PC. As shown in the figure, the integrated environment management system 2000 includes an input device 100, an output device 200, a processing device 300, a main memory device 400, an auxiliary memory device 500, a communication device 600, and a bus 700 that electrically connects these devices.

[0279] The input device 100 can constitute the user input receiving unit 3320 of the user interface unit 3300, and is a device that allows a user to input information and instructions to the operational application control system 3000. Specifically, the input device 100 is, for example, a touch panel, a keyboard, a mouse, or an audio input device such as a microphone.

[0280] The output device 200 is a device that outputs various information generated by the operational application control system 3000, and can constitute the display unit 3310 of the user interface unit 3300. Specifically, the output device 200 can constitute the display unit 3310 using a display device for eyewear, AR, or VR, or it may also be a printer or a speaker.

[0281] The processing device 300 is, for example, a device that performs arithmetic processing. Specifically, the processing device 300 is, for example, a CPU, a microprocessor, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or other semiconductor devices capable of performing arithmetic processing.

[0282] The main storage device 400 is a memory device such as a RAM that temporarily stores various types of read information and a ROM that stores programs, application programs, and other various information executed by the processing device 300. The auxiliary storage device 500 is a non-volatile storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory that can store digital information.

[0283] The communication device 600 is a device that performs wireless or wired information communication with the outside.

[0284] The above-described embodiments are merely examples for facilitating understanding of the present invention, and are not intended to limit the present invention. The present invention can be modified and improved without departing from the spirit thereof, and it goes without saying that the present invention includes equivalents thereof.

[0285] [B. Effects of this embodiment] The above-described embodiment allows the performance required for operations to be achieved while reducing costs associated with physical and human resources or the impact on economic activity in the vicinity of the site. For example, by calculating the behavior of a mixed drone fleet consisting of real and virtual drones deployed in an operation area through simulation and confirming that a predetermined operational state can be maintained in real time, real drones can be dispatched to the operation area more quickly and efficiently when additional real drones are added to the operation area. This eliminates the need to constantly deploy a large number of drones in the operation area, allowing the performance required for operations to be achieved while reducing the number of real drones deployed. [Explanation of symbols]

[0286] 1...Operation system (system) 100...input device 200...output device 300...Processing device 400...Main storage device 500...Auxiliary storage device 600...Communication device 700...bus 1000...Unmanned boat system 1001...Base unit 1002...Sub unit 1010...Unmanned boat 10021... Primary connection slave unit 10022... Secondary connection slave unit 1100...Measuring unit 1110...Measuring sensor 1120...Measurement control unit 1200...Own aircraft state determination unit 1210...Navigation state determination unit 1220: Internal state determination unit 1230: External state determination unit 1300...Navigation section 1310...Thrust generation section 1320: Attitude control mechanism 1330: Navigation control unit 1400...Communication unit 1410...Unmanned vehicle communication unit 1420…General Control and Communications Department 1500…Judgment section 1600...Recording section 1610...Measurement data recording section 1620...Own aircraft status recording section 1630...Determination information recording section 1700...power supply unit 1710...electricity storage device 1720...power generating device 1730...power control unit 2000: Integrated Environmental Management System 2100: Information acquisition unit 2110: User request information acquisition unit 2120: Simulation condition acquisition unit 2130: Actual machine information acquisition unit 2140: Environmental information acquisition unit 2150: External information acquisition unit 2160: User intervention information acquisition unit 2200...Virtual space generation unit 2300...Real-time drone swarm simulator 2310... Virtual machine behavior calculation unit 2320... Actual machine behavior calculation unit 2330…Mixed aircraft group behavior calculation unit 2400...Future group control state simulator 2410... Virtual machine future behavior prediction unit 2420... Actual machine future behavior prediction unit 2430…Mixed Aircraft Fleet Future Behavior Prediction Department 2500...State determination unit 2510...Object detection state determination unit 2520: Current state determination unit 2530: Future state prediction determination unit 2540…Drone Increase / Decrease Decision Unit 2600...Group control command determination unit 2610...Control command determination unit 2620...Future command determination unit 2630...Control state transition determination unit 2640: Actual machine increase / decrease control command generation unit 2700...information output unit 2710...determination state output unit 2720...Unmanned boat behavior information output unit 2730...Future behavior information output unit 2740…Actual measurement data output section 3000...Operational Application Control System 3100: Information recording unit 3110: Judgment state recording unit 3120...Unmanned vessel behavior information recording unit 3130...Future behavior information recording unit 3140...Actual measurement data recording unit 3200: Display information output control unit 3210: Display information generation unit 3220...Display information control unit 3300...User interface section 3310: Display unit 3320: User input reception unit 4000…External system 5100: Communications satellite 5200: Ground base station 6000...Cooperative system 7000...Object 8000...User terminal device

Claims

1. An operation system for controlling the operation of multiple unmanned aerial vehicles deployed in an operation area, an actual aircraft behavior information acquisition unit that acquires actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; an unmanned aircraft group simulator that calculates the behavior state of a mixed unmanned aircraft group that includes a simulated real aircraft that simulates the real aircraft in a virtual space based on the real aircraft behavior information and a virtual aircraft of the unmanned aircraft generated in the virtual space; a control command determination unit that generates a control command for the unmanned aerial vehicle including at least the real vehicle, the simulated real vehicle, or the virtual vehicle; A display unit that displays and outputs the behavior state of the mixed unmanned aerial vehicle group generated by the unmanned aerial vehicle group simulator; a real machine increase / decrease determination unit that determines whether to increase the number of the real machines deployed in the operational area by replacing at least one of the virtual machines with the real machine, or whether to decrease the number of the real machines deployed in the operational area by replacing at least one of the real machines with the virtual machine; An operating system comprising:

2. 2. The operating system according to claim 1, When the actual aircraft increase / decrease decision unit determines to increase the number of the actual aircraft deployed in the operation area, The control command determination unit generates an additional real machine control command for controlling movement of an additional real machine to replace the virtual machine, with the position of the virtual machine or a periphery of that position set as a movement target position.

3. 3. The operation system according to claim 2, When the movement of the additional real machine to replace the virtual machine to the position of the virtual machine or the vicinity of the position is completed in accordance with the additional real machine control command, The unmanned aerial vehicle simulator is an operation system that replaces the virtual vehicle in the virtual space with a simulated real vehicle that simulates the additional real vehicle.

4. 4. The operation system according to claim 3, When the unmanned aerial vehicle simulator replaces the virtual vehicle in the virtual space with the simulated real vehicle, The behavior information of the simulated real machine includes at least one of information on a position, an orientation, a moving speed, a moving direction, a moving acceleration, a rotational angular velocity, a battery charge state, and a battery deterioration state; An operation system in which the behavior information of the simulated real aircraft is updated by the real aircraft behavior information acquired by the real aircraft behavior information acquisition unit, or is updated by the behavior state of the simulated real aircraft calculated by the unmanned aircraft group simulator based on the real aircraft behavior information.

5. 2. The operating system according to claim 1, When the actual aircraft increase / decrease decision unit determines to reduce the number of the actual aircraft deployed in the operational area, The control command determination unit generates a recovered real machine control command for recovering the recovered real machine to be replaced with the virtual machine or moving the recovered real machine to a waiting area.

6. 6. The operation system according to claim 5, When the actual aircraft increase / decrease decision unit determines to reduce the number of the actual aircraft deployed in the operational area, The unmanned aerial vehicle swarm simulator calculates the behavior state of the virtual vehicle to be replaced with the recovered real vehicle in the virtual space independently from the real vehicle behavior information of the recovered real vehicle.

7. 2. The operating system according to claim 1, The actual machine increase / decrease decision unit decides to increase the number of actual machines when at least one of the following conditions is met: alert information is received from an external system; a predetermined time period has arrived; or a command requesting an increase in actual machines is received from a user.

8. 2. The operating system according to claim 1, When searching for a predetermined object present in the operation area or a surrounding area of ​​the operation area using a measurement sensor mounted on the actual machine, The actual aircraft increase / decrease decision unit decides to increase the number of actual aircraft when at least one of the following occurs: the target object or a candidate object for the target object is detected using measurement data from the measurement sensor, or a value indicating the unmanned aircraft's search performance, mobility performance, battery performance, or communication performance has fallen below a predetermined value.

9. 2. The operating system according to claim 1, An operation system in which the actual aircraft increase / decrease decision unit decides to reduce the number of actual aircraft when at least one of the following conditions is met: alert information previously acquired from an external system has been lifted; a command requesting a reduction in actual aircraft has been acquired from a user; or a detection flag indicating that an object or a candidate object for the object has been detected using measurement data from a measurement sensor mounted on the unmanned aircraft has been lifted.

10. 2. The operating system according to claim 1, The actual aircraft increase / decrease decision unit decides to reduce the number of actual aircraft when at least one of the following occurs: the wave conditions in the operation area have deteriorated beyond a predetermined value, or it is predicted that future wave conditions in the operation area will deteriorate beyond a predetermined value.

11. 2. The operating system according to claim 1, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, The actual aircraft increase / decrease decision unit determines the future relative distance between the other moving body and the unmanned aircraft based on current movement information or future planned movement information of other moving bodies obtained from an external system and the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator, and decides to reduce the number of actual aircraft if it determines that an interference state in which the relative distance will be less than a predetermined distance will occur in the future.

12. 2. The operating system according to claim 1, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control commands for the drones, including the real drones and the virtual drones, so as to prevent an interference state from occurring in which the future relative distance between the drones in the mixed unmanned drone group, determined based on the future behavior state of the mixed unmanned drone group generated by the unmanned drone group simulator, is less than a predetermined collision avoidance safe distance.

13. 2. The operating system according to claim 1, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control command for the unmanned aircraft, including the real aircraft and the virtual aircraft, so as to prevent an interference state from occurring in which the future relative distance between the other moving body and the unmanned aircraft, determined based on the current movement information or future movement schedule information of the other moving body obtained from an external system and the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator, is less than a predetermined distance.

14. 2. The operating system according to claim 1, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control command for the unmanned aircraft, including the actual aircraft and the virtual aircraft, so as to prevent a wave area navigation state from occurring in which the relative distance on a horizontal plane between the typhoon, low pressure, rain cloud, or wave area and the unmanned aircraft is less than a predetermined distance, as determined based on current or future behavior information of the typhoon, low pressure, rain cloud, or wave area obtained from an external system and the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator.

15. 2. The operating system according to claim 1, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control commands for the unmanned aircraft, including the real aircraft and the virtual aircraft, so as to prevent a communication failure state from occurring in which the future relative distance between the unmanned aircraft of the mixed unmanned aircraft group, determined based on the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator, exceeds a predetermined wireless communication distance between the unmanned aircraft.

16. 2. The operating system according to claim 1, The unmanned aerial vehicle swarm simulator predicts and calculates a future behavior state of the mixed unmanned aerial vehicle swarm, An operation system in which the control command determination unit generates the control commands for the unmanned aircraft, including the actual aircraft and the virtual aircraft, so that the operational state of the mixed unmanned aircraft group determined based on the future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator achieves a predetermined target for the operational state, or so that the determined operational state improves.

17. 3. The operation system according to claim 2, When the target position of the additional real aircraft included in the additional real aircraft control command is outside the wireless communication range of the real aircraft already deployed in the operation area in the real space, The control command determination unit generates a second additional real machine control command to deploy a second additional real machine that relays wireless communication between the real machine and the additional real machine, with a second movement target position being a position that is within the wireless communication range from the real machine and also within the wireless communication range from the additional real machine.

18. 18. The operating system according to claim 17, The control command determination unit after moving the second additional actual machine to the second movement target position in response to the second additional actual machine control command, moving the additional actual machine to the movement target position in response to the additional actual machine control command; Alternatively, an operation of moving the second additional actual machine to the second movement target position in accordance with the second additional actual machine control command and an operation of moving the additional actual machine to the movement target position in accordance with the additional actual machine control command are performed in parallel.

19. 3. The operation system according to claim 2, during a period in which the additional real machine is moved to the position of the virtual machine or a peripheral position thereof in response to the additional real machine control command, the control command determination unit generates a control command to stop or almost stop the virtual machine; When the movement of the additional real aircraft to the position of the virtual aircraft or a peripheral position thereof is completed in response to the additional real aircraft control command, the unmanned aircraft swarm simulator rewrites the behavior state of the virtual aircraft in the virtual space with the real aircraft behavior information of the additional real aircraft that will replace the virtual aircraft.

20. 3. The operation system according to claim 2, The additional real machine control command is a control command for matching the position, orientation, and speed of the additional real machine with the position, orientation, and speed of the virtual machine, and when the position, orientation, and speed of the virtual machine match or approximately match the position, orientation, and speed of the additional real machine, The unmanned aerial vehicle swarm simulator rewrites the behavior state of the virtual vehicle in the virtual space with the real vehicle behavior information of the additional real vehicle that replaces the virtual vehicle.

21. 3. The operation system according to claim 2, during a movement period in which the additional real machine is moved to the position of the virtual machine or a peripheral position thereof in response to the additional real machine control command, the control command determination unit generates a control command to move the virtual machine in a predetermined movement pattern; The additional real aircraft control command is a control command to match the position, orientation, and speed of the virtual aircraft with the position, orientation, and speed of the additional real aircraft, and when the position, orientation, and speed of the virtual aircraft match or approximately match the position, orientation, and speed of the additional real aircraft, the unmanned aircraft swarm simulator rewrites the behavior state of the virtual aircraft in the virtual space with the real aircraft behavior information of the additional real aircraft that will replace the virtual aircraft.

22. 2. The operating system according to claim 1, a state determination unit that determines whether an interference state will occur in which the current or future relative distance between the drones in the mixed unmanned aircraft group, or the relative distance between another moving object and the drone, will be equal to or less than a predetermined distance, based on the current or future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator; The display unit displays the determination result of whether or not the interference state has occurred.

23. 2. The operating system according to claim 1, a state determination unit that determines whether a communication failure state will occur in which the current or future relative distance between the drones in the mixed unmanned aircraft group will be equal to or greater than a predetermined wireless communication distance between the drones, based on the current or future behavior state of the mixed unmanned aircraft group generated by the unmanned aircraft group simulator; The display unit displays the determination result of whether or not the communication failure state has occurred.

24. 2. The operating system according to claim 1, a state determination unit that determines whether the current or future operational state of the mixed unmanned aerial vehicle group will achieve a preset goal related to the operational state based on the current or future behavior state of the mixed unmanned aerial vehicle group generated by the unmanned aerial vehicle group simulator; The display unit displays a determination result of whether the operational status achieves the preset target.

25. 2. The operating system according to claim 1, The display unit displays and outputs the actual machine behavior information acquired from the actual machine or the actual machine measurement data measured by a measurement sensor mounted on the actual machine.

26. A control method for controlling the operation of multiple unmanned aerial vehicles deployed in an operation area, comprising: The computer an actual aircraft behavior information acquisition step of acquiring actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; an unmanned aircraft group behavior calculation step for calculating the behavior state of a mixed unmanned aircraft group including a simulated real aircraft that simulates the real aircraft in a virtual space based on the real aircraft behavior information and a virtual aircraft of the unmanned aircraft generated in the virtual space; a real machine increase / decrease decision step for deciding to increase the number of the real machines deployed in the operational area by replacing at least one of the virtual machines with the real machine, or to decrease the number of the real machines deployed in the operational area by replacing at least one of the real machines with the virtual machine; a control command determination step of generating a control command for the unmanned aerial vehicle including at least the actual vehicle; a display step of displaying and outputting the behavior state of the mixed unmanned aerial vehicle group generated by the unmanned aerial vehicle group behavior calculation step; A control method comprising:

27. A program available for an operation system that controls the operation of multiple unmanned aerial vehicles deployed in an operation area, On the computer, an actual aircraft behavior information acquisition command to acquire actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; an unmanned aircraft group behavior calculation command for calculating the behavior state of a mixed unmanned aircraft group including a simulated real aircraft that simulates the real aircraft in a virtual space based on the real aircraft behavior information and a virtual aircraft of the unmanned aircraft generated in the virtual space; a real machine increase / decrease decision command that decides to increase the number of the real machines deployed in the operational area by replacing at least one of the virtual machines with the real machine, or to decrease the number of the real machines deployed in the operational area by replacing at least one of the real machines with the virtual machine; a control command determination command for generating a control command for the unmanned aerial vehicle including at least the actual vehicle; A display command for displaying and outputting the behavior state of the mixed unmanned aerial vehicle group generated by the unmanned aerial vehicle group behavior calculation command; A program that executes.

Images