Operational state simulation system, operational state simulation method and program

Abstract

According to the present invention, it is possible to evaluate the performance and understand the operational requirements of a more reliable unmanned mobile system while reducing resources or the impact on the surrounding area of ​​the site. [Solution] The present invention is an operational state simulation system that includes a virtual space generation unit that generates a virtual operating area in a virtual space, a real aircraft behavior information acquisition unit that acquires real aircraft behavior information regarding aircraft behavior from a real aircraft in the real space, a virtual aircraft behavior calculation unit that calculates virtual aircraft behavior calculation information for a virtual aircraft deployed in the virtual operating area based on virtual aircraft model information, a mixed unmanned aircraft group behavior generation unit that generates behavior information for a mixed unmanned aircraft group that is a mixture of real aircraft and virtual aircraft based on the real aircraft behavior information and the virtual aircraft behavior calculation information, and an operational requirement determination unit that determines the operational requirements for the unmanned aircraft group so that the operational state estimate of the mixed unmanned aircraft group calculated based on the behavior information of the mixed unmanned aircraft group meets the set targets for the operational state.

Description

[Technical Field]

[0001] The present invention relates to an operational state simulation system, an operational state simulation method, and a program. [Background technology]

[0002] In recent years, the use of unmanned mobile systems, including multiple relatively small manned boats and unmanned vessels (hereinafter also referred to as "unmanned vessels"), for specific purposes such as collecting oceanographic data, providing a communications environment for sea and underwater areas, searching for suspicious vessels, and inspecting and patrolling offshore infrastructure, has been considered. When using such unmanned mobile systems for specific purposes, it is necessary to confirm in advance that the system can be used to demonstrate the minimum performance required for actual operation before actually operating the system. Therefore, generally, a demonstration test is conducted in advance in a location, environment, and for a period similar to that of actual operation, and the performance of the unmanned mobile system is evaluated.

[0003] However, in the above-mentioned demonstration tests, if the system to be tested includes a large number of unmanned vehicles, it is necessary to deploy a large number of unmanned vehicles at the site, which results in enormous costs for arranging the vehicles, replacing consumable parts, and the human resources required to operate the unmanned vehicle system (transporting and deploying them at the site, controlling and operating the mission, and inspecting, maintaining, and managing the vehicles).In addition, deploying a large number of unmanned vehicles (unmanned boats, etc.) at the test area (ocean area) where the demonstration test is conducted could interfere with the paths of general ships sailing within and around the test area, and if access by general ships is restricted within the test area to avoid the risk of collision, there is a problem of impacting private economic activity.

[0004] Therefore, technologies are being considered for simulating the operating state of an unmanned mobile system in a virtual space that simulates the actual operating environment, verifying the performance that the unmanned mobile system can demonstrate during actual operation, and determining operational requirements (number of vehicles, placement, group control method, etc.) for achieving target performance required during actual operation. For example, Patent Document 1 discloses a traffic flow simulation method and device for evaluating vehicle flow, which provides a simulation method and device that can treat vehicles with similar vehicle attributes as a group of vehicles. In particular, the patent document discloses a technology for performing a group of vehicles cooperative simulation using a traffic flow simulation means that performs a traffic flow simulation, a group of vehicles traveling generation and management means that generates and manages a plurality of vehicles as a group of vehicles based on the behavior of each vehicle, a group of vehicles traveling state storage means that stores the traveling state of the generated group of vehicles, a group of vehicles traveling state determination means that obtains group of vehicles display information that indicates the shape and traveling state of the group of vehicles that is necessary when determining the traveling state of the group of vehicles and displaying the traveling state as an image, and a group of vehicles display image generation means that displays an image of the behavior of each vehicle or the group of vehicles. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-81704 Summary of the Invention [Problem to be solved by the invention]

[0006] In a method for evaluating the operating state of an unmanned mobile system in a virtual space by simulating it, such as that described in Patent Document 1, it is necessary to assume situations and scenarios that may occur during actual operation, such as changes in the external environment and the approach of other mobile bodies, and set these in a simulation model in advance. However, in reality, it is difficult to assume all changes in the situation, such as the external environment and the approach of other mobile bodies, during actual operation, and therefore it is not easy to accurately evaluate the performance that the unmanned mobile system can exhibit during actual operation and the operational requirements that can achieve the required performance.

[0007] Therefore, before actually operating an unmanned mobile system, there is a need for a method to confirm in advance that the required performance for the actual operation can be achieved and the operational requirements for achieving the required performance, without requiring excessive physical or human resources or minimizing the impact on economic activity in the surrounding area, and to perform a performance evaluation of a highly reliable unmanned mobile system, or to confirm the operational requirements that can achieve the required performance.

[0008] Therefore, the present invention has been made in consideration of at least one of the above problems, and one of its objectives is to provide a system or control method, etc., that can evaluate the performance of a more reliable unmanned mobile system or confirm operational requirements that can achieve the required performance, without requiring excessive material and human resources or minimizing the impact on economic activity in the surrounding area of ​​the site. [Means for solving the problem]

[0009] According to the present invention, an operational state simulation system is provided that simulates the operational state in an operational area of ​​an unmanned aircraft group including a plurality of unmanned aircraft, the operational state simulation system comprising: a virtual space generation unit that generates a virtual operational area in a virtual space; a real aircraft behavior information acquisition unit that acquires actual aircraft behavior information regarding aircraft behavior from the actual unmanned aircraft deployed in the operational area in the real space; a virtual aircraft behavior calculation unit that calculates virtual aircraft behavior calculation information regarding aircraft behavior of the virtual aircraft of the unmanned aircraft deployed in the virtual operational area generated in the virtual space based on virtual aircraft model information; a mixed unmanned aircraft group behavior calculation unit that calculates behavior calculation information of a mixed unmanned aircraft group in which the actual aircraft and the virtual aircraft are mixed based on the actual aircraft behavior information and the virtual aircraft behavior calculation information; and an operational requirement determination unit that determines operational requirements or candidate operational requirements for the unmanned aircraft group so that the operational state estimated value of the mixed unmanned aircraft group calculated based on the behavior calculation information of the mixed unmanned aircraft group satisfies a previously set target related to the operational state. [Effects of the Invention]

[0010] According to the present invention, it is possible to evaluate the performance of a more reliable unmanned mobile system or to understand operational requirements that meet the required performance, without requiring excessive material and human resources or minimizing the impact on economic activity in the surrounding area. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating the overall configuration of an operational state simulation system 1 according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram showing an example of an implementation image of an operational state simulation system 1 in a real space. [Figure 3] FIG. 2 is a diagram showing an example of stakeholders related to the operational state simulation system 1. [Figure 4] FIG. 2 is a diagram showing an example of state transition of the operational state simulation system 1. [Figure 5] FIG. 10 is a diagram showing an example of a formation of a group made up of multiple unmanned boats 1010. [Figure 6] FIG. 2 is a functional block diagram showing the functional configuration of an unmanned watercraft 1010. [Figure 7] FIG. 10 is a diagram illustrating an example of the operation of the integrated environment management system 2000 in an operational requirement determination state. [Figure 8] FIG. 10 is a diagram showing an example of input / output information of the integrated environment management system 2000 in an operational requirement determination state. [Figure 9] 10 is a diagram showing an example of input / output information between functional units of the integrated environment management system 2000 in an operational requirement determination state. FIG. [Figure 10] FIG. 10 is a process diagram showing an example of a processing process of the converged environment management system 2000 etc. in an operational requirement determination state. [Figure 11] FIG. 2 is a functional block diagram showing the functional configuration of an integrated environment management system 2000. [Figure 12] FIG. 3 is a functional block diagram showing the functional configuration of an operational application control system 3000. [Figure 13] 10 is a diagram showing an example of user request information acquired by a user request information acquisition unit 2110. FIG. [Figure 14] 10 is a diagram showing an example of assumed condition parameters acquired by an assumed condition parameter acquisition unit 2130. FIG. [Figure 15] FIG. 10 is a diagram illustrating an example of an operation result from a past operation. [Figure 16] FIG. 10 is a diagram showing the change trend of detection probability for each item of operating conditions. [Figure 17] 10 is a diagram showing an example of operational requirement items determined by an operational requirement determination unit 2600. FIG. [Figure 18] 10 is a diagram showing an example of operational requirements that change over time within an operational period determined by the operational requirement determination unit 2600. FIG. [Figure 19] A figure showing an example of a display screen of behavior information of a mixed unmanned aerial vehicle group displayed on the display unit 3310. [Figure 20] FIG. 2 is a diagram showing an example of a higher-level control processing flow of the operational state simulation system 1. [Figure 21] FIG. 10 is a diagram showing an example of a control processing flow of presetting processing by an information acquisition unit 2100. [Figure 22] FIG. 10 is a diagram showing an example of a control processing flow of preparations before simulation execution by the virtual space generating unit 2300. [Figure 23] FIG. 10 is a diagram showing an example of a control processing flow of real-time processing during simulation execution by the unmanned watercraft behavior estimation unit 2400 and the like. [Figure 24] FIG. 10 is a diagram showing an example of a control processing flow for displaying information and receiving user input by the operational application control system 3000. [Figure 25] FIG. 10 is a diagram showing an example of a control processing flow of updating operational requirements and the like by the operational requirement determination unit 2600 and the like. [Figure 26] FIG. 10 is a diagram showing an example of a control processing flow of actual machine action determination processing by the operational requirement determination unit 2600. [Figure 27] FIG. 10 is a diagram showing an example of a control processing flow of operational requirement determination processing by an operational requirement determination unit 2600. [Figure 28] FIG. 2 is a diagram showing an example of a higher-level control processing flow of the operational state simulation system 1. [Figure 29] FIG. 10 is a diagram showing a decision control flow of the operational state simulation system 1 when determining future operational requirements during actual operation. [Figure 30] FIG. 10 is a diagram showing an example of an implementation in which the functional units of the converged environment management system 2000 and the operational application control system 3000 are implemented on a cloud server, a workstation, or the like. [Figure 31] FIG. 1 is a diagram illustrating an example of an implementation of a HILS connected to a virtual machine simulator. [Figure 32] FIG. 2 is a diagram illustrating an example of a hardware configuration diagram of a converged environment management system 2000 and an operational application control system 3000. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present invention will be described below with reference to the following embodiments. [Item 1] An operational state simulation system that simulates an operational state in an operational area of ​​a group of unmanned aerial vehicles including a plurality of unmanned aerial vehicles, a virtual space generation unit that generates a virtual operation area in a virtual space; an actual aircraft behavior information acquisition unit that acquires actual aircraft behavior information regarding aircraft behavior from the actual unmanned aircraft deployed in the operation area in real space; a virtual aircraft behavior calculation unit that calculates virtual aircraft behavior calculation information regarding aircraft behavior of the virtual aircraft of the unmanned aircraft deployed in the virtual operation area generated in the virtual space based on virtual aircraft model information; a mixed unmanned aircraft group behavior generation unit that generates behavior information of a mixed unmanned aircraft group in which the real aircraft and the virtual aircraft are mixed based on the real aircraft behavior information and the virtual aircraft behavior calculation information; An operational state simulation system comprising an operational requirement determination unit that determines operational requirements or candidate operational requirements for the unmanned aircraft group so that the operational state estimate value of the mixed unmanned aircraft group calculated based on the behavior information of the mixed unmanned aircraft group satisfies a pre-set target for the operational state. [Item 2] In the operational state simulation system according to item 1, The operational requirements of the unmanned aerial vehicle group determined by the operational requirements determination unit include: The number of deployed aircraft, which are the unmanned aircraft to be deployed in real space; Or, the number of replacement machines to be placed on standby to replace the deployment machine whose charge amount in the power storage device is below a predetermined value or whose installed functional unit has failed or become abnormal, Or, the movement path pattern, movement speed, deployment area, arrangement distribution pattern, or straight movement time of the deployment machine, Or, a schedule for replacing the developing machine with the replacement machine, Or, a recovery schedule for recovering the deployer; Or, a schedule of a recovery charging operation for charging the power storage device mounted on the deploying machine, An operational state simulation system including: [Item 3] In the operational state simulation system according to item 1 or 2, The operational requirements of the unmanned aerial vehicle group determined by the operational requirements determination unit are: An operational state simulation system in which at least one of the number of the deployed aircraft, the number of the replacement aircraft, the movement path pattern, the movement speed, the deployment area, the distribution pattern, or the straight-line travel time of the unmanned aircraft is specified to change over time during the operational period of the unmanned aircraft group. [Item 4] In the operational state simulation system according to any one of items 1 to 3, When the mission of the unmanned aerial vehicle group includes a search mission to search for an object using a measurement sensor mounted on the unmanned aerial vehicle, An operational state simulation system, wherein the set target includes a target value for a search rate indicating the proportion of the cumulative measurement area measured by the measurement sensor during an operation period, or a target value for a detection probability indicating the probability of the measurement sensor detecting the object. [Item 5] In the operational state simulation system according to any one of items 1 to 4, When the unmanned aerial vehicle operates using a power storage device mounted on the vehicle as a power source, An operational state simulation system, wherein the set target includes a target value for the average or minimum value of the charge amount, charge rate, or charge capacity of the storage device of the multiple unmanned aircraft, or a target value for the number of unmanned aircraft that have been replaced due to a lack of charge amount, charge rate, or charge capacity of the storage device. [Item 6] In the operational state simulation system according to any one of items 1 to 5, An operational state simulation system, wherein the set goals include a target value for operating time, which is the time during which the group of unmanned aircraft operates in the operational area. [Item 7] In the operational state simulation system according to any one of items 1 to 6, When the mission of the unmanned aerial vehicle group includes a wireless communication mission of connecting wireless communication with a communication unit mounted on the unmanned aerial vehicle to equipment present in the operation area or the surrounding area of ​​the operation area, An operational state simulation system, wherein the set goals include target values ​​for the communication quality of wireless communication between the equipment and the unmanned aircraft, or the area in which wireless communication with the unmanned aircraft is possible, or the time period in which wireless communication with the unmanned aircraft is possible. [Item 8] In the operational state simulation system according to any one of items 1 to 7, When the mission of the unmanned aerial vehicle group includes a measurement data acquisition mission in which measurement sensors mounted on the unmanned aerial vehicles measure the operation area or the area surrounding the operation area, The set target includes a target value for the data measurement time for measuring by the measurement sensor, the amount of measurement data obtained by measurement, or the size of the data measurement area for measuring. [Item 9] In the operational state simulation system according to any one of items 1 to 8, a real environment information acquisition unit that acquires environmental measurement data relating to environmental information including at least one of ocean currents, wave heights, wind speeds, wind directions, solar radiation, brightness, visibility, air temperatures, water temperatures, and weather in the operation area of ​​the real space; The virtual aircraft behavior calculation unit calculates the virtual aircraft behavior calculation information related to the aircraft behavior of the virtual aircraft based on the environmental measurement data and the virtual aircraft model information. [Item 10] In the operational state simulation system according to any one of items 1 to 9, a real environment information acquisition unit that acquires environmental measurement data relating to environmental information including at least one of ocean currents, wave heights, wind speeds, wind directions, solar radiation, brightness, visibility, and weather in the operation area of ​​the real space; the virtual space generation unit generates or updates a virtual environment state of the virtual operation area based on the environmental measurement data; The virtual aircraft behavior calculation unit calculates the virtual aircraft behavior calculation information related to the aircraft behavior of the virtual aircraft based on the virtual environment state and the virtual aircraft model information. [Item 11] In the operational state simulation system according to any one of items 1 to 10, an actual aircraft behavior calculation unit that calculates actual aircraft behavior calculation information regarding aircraft behavior of the actual aircraft based on actual aircraft model information that models the actual aircraft in the virtual space; An operational state simulation system, wherein the virtual aircraft behavior calculation unit calculates the virtual aircraft behavior calculation information regarding the aircraft behavior of the virtual aircraft based on a comparison result between the actual aircraft behavior calculation information calculated by the actual aircraft behavior calculation unit and the actual aircraft behavior information acquired by the actual aircraft behavior information acquisition unit. [Item 12] In the operational state simulation system according to any one of items 1 to 11, An operational state simulation system, wherein the virtual aircraft behavior calculation information calculated by the virtual aircraft behavior calculation unit includes at least one of the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of the virtual aircraft. [Item 13] In the operational state simulation system according to any one of items 1 to 12, a target achievement determination unit that determines whether the operational state estimation value of the mixed unmanned aerial vehicle group calculated based on the behavior calculation information of the mixed unmanned aerial vehicle group satisfies the set target; An operational state simulation system in which, when the goal achievement determination unit determines that the operational state estimation value does not satisfy the set goal, the operational requirement determination unit changes the operational requirements for the mixed unmanned aircraft group. [Item 14] In the operational state simulation system according to any one of items 1 to 13, a target achievement determination unit that determines whether the operational state estimation value of the mixed unmanned aerial vehicle group calculated based on the behavior calculation information of the mixed unmanned aerial vehicle group satisfies the set target; An operational state simulation system comprising: an operation command determination unit that determines operation commands including commands regarding at least one of the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of the actual or virtual aircraft included in the mixed unmanned aircraft group based on the judgment result of the goal achievement judgment unit. [Item 15] In the operational state simulation system according to any one of items 1 to 14, An operational state simulation system including an information output unit that transmits and outputs the operation command to the actual device, or displays and outputs information about the operation command. [Item 16] In the operational state simulation system according to any one of items 1 to 15, An operational state simulation system comprising an information output unit that transmits or displays information about the operational requirements or the candidate operational requirements determined by the operational requirement determination unit to an external device. [Item 17] In the operational state simulation system according to any one of items 1 to 16, An operational state simulation system comprising an information output unit that transmits or displays information regarding the behavior calculation information of the mixed unmanned aircraft group or information regarding the operational state estimation value to the outside. [Item 18] In the operational state simulation system according to any one of items 1 to 17, An operational state simulation system comprising an information output unit that displays and outputs the actual aircraft behavior information acquired from the actual aircraft or actual aircraft measurement data measured by a measurement sensor mounted on the actual aircraft. [Item 19] In the operational state simulation system according to any one of items 1 to 18, an environmental prediction data acquisition unit that acquires environmental prediction data after a predetermined time has elapsed in the operation area; the virtual space generation unit generates a predicted virtual environment state of the virtual operation area after the predetermined time has elapsed based on the environment prediction data; the virtual aircraft behavior calculation unit calculates the virtual aircraft behavior calculation information related to aircraft behavior of the virtual aircraft in the virtual operation area after the predetermined time has elapsed, based on the virtual environment predicted state and the virtual aircraft model information; The mixed unmanned aircraft group behavior calculation unit calculates the behavior calculation information of the mixed unmanned aircraft group after the predetermined time has elapsed based on the actual aircraft behavior information and the virtual aircraft behavior calculation information, The operational requirement determination unit is an operational state simulation system that determines the operational requirements or candidate operational requirements of the unmanned aircraft group so that the operational state estimation value of the mixed unmanned aircraft group after the predetermined time has elapsed, which is calculated based on the behavior calculation information of the mixed unmanned aircraft group, meets the set target regarding the operational state. [Item 20] In the operational state simulation system according to any one of items 1 to 19, An operational state simulation system including a user input receiving unit that receives user input information for changing the operational requirement or selecting a specific candidate from the operational requirement candidates. [Item 21] In the operational state simulation system according to any one of items 1 to 20, An operational state simulation system including a user input receiving unit that receives, from a user, user input information for setting or changing the set target. [Item 22] An operational state simulation method for simulating an operational state in an operational area of ​​a group of unmanned aerial vehicles including a plurality of unmanned aerial vehicles by simulation, The computer a virtual space generation step of generating a virtual operation area in the virtual space; an actual aircraft behavior information acquisition step of acquiring actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; a virtual aircraft behavior calculation step of calculating virtual aircraft behavior calculation information regarding aircraft behavior of the virtual aircraft of the unmanned aircraft deployed in the virtual operation area generated in the virtual space based on virtual aircraft model information; a mixed unmanned aircraft group behavior generation step of generating behavior information of a mixed unmanned aircraft group in which the real aircraft and the virtual aircraft are mixed based on the real aircraft behavior information and the virtual aircraft behavior calculation information; an operational requirement determination step of determining operational requirements or candidate operational requirements for the unmanned aerial vehicle group so that the operational state estimate of the mixed unmanned aerial vehicle group calculated based on the behavior information of the mixed unmanned aerial vehicle group satisfies a preset target related to the operational state; an information output step of transmitting or displaying information relating to the determined operational requirement or the candidate operational requirement to an external device; [Item 23] A program usable for an operational status simulation system that simulates an operational status in an operational area of ​​a group of unmanned aerial vehicles including a plurality of unmanned aerial vehicles, To the computer a virtual space generation command for generating a virtual operation area in the virtual space; an actual aircraft behavior information acquisition command to acquire actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; a virtual aircraft behavior calculation command that calculates virtual aircraft behavior calculation information regarding aircraft behavior of the virtual aircraft of the unmanned aircraft deployed in the virtual operation area generated in the virtual space based on virtual aircraft model information; A mixed unmanned aircraft group behavior generation command that generates behavior information of a mixed unmanned aircraft group in which the real aircraft and the virtual aircraft are mixed based on the real aircraft behavior information and the virtual aircraft behavior calculation information; An operational requirement determination command that determines operational requirements or candidate operational requirements for the unmanned aerial vehicle group so that the operational state estimate of the mixed unmanned aerial vehicle group calculated based on the behavior information of the mixed unmanned aerial vehicle group satisfies a preset target related to the operational state; A program that executes an information output command to transmit or display information about the determined operational requirement or candidate operational requirement to the outside.

[0013] First Embodiment Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, the embodiments described below are merely examples, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc. In addition, in the first embodiment, an example will be described in which an unmanned boat 1010 is used as an example of an unmanned mobile object, but the unmanned mobile object is not limited to this, and the present invention can be applied to any other unmanned mobile object, such as an unmanned aerial vehicle, an unmanned vehicle, or an unmanned submersible.

[0014] [A.Configuration] (A-1. Overall system configuration) First, the system configuration of an operational state simulation system 1 (hereinafter also referred to as "system 1") according to one embodiment of the present invention will be described with reference to FIGS.

[0015] (A-1-1. System Configuration Overview) FIG. 1 is an overall configuration diagram of an operational state simulation system 1 according to one embodiment of the present invention. As shown in FIG. 1, the operational state simulation system 1 includes an unmanned watercraft system 1000, an integrated environment management system 2000, and an operational application control system 3000. The unmanned watercraft system 1000 is equipped with measurement sensors capable of detecting an object 7000 and includes multiple unmanned watercraft 1010 (also referred to as "unmanned vessels") capable of navigating and moving on the sea. The integrated environment management system 2000 is also configured to be able to communicate with an external cooperative system 6000 and an external system 4000 via an internet line or the like, allowing for the input and output of information. The integrated environment management system 2000 can send control commands to the unmanned watercraft system 1000 deployed on the sea via a terrestrial base station 5200 and a communication satellite 5100, and can also receive behavior information and measurement data from the unmanned watercraft system 1000. Therefore, the integrated environment management system 2000 can remotely control the operation of the unmanned watercraft system 1000 or make it navigate autonomously or automatically, and perform operations such as searching for an object 7000 using measurement sensors in a predetermined area (first area) on the sea or underwater, collecting marine data, inspecting or patrolling offshore infrastructure, or providing a communication environment to the sea or underwater area using a communication device mounted on the unmanned watercraft 1010. Here, the predetermined area is any area that can be set by user input or can be set in the system in advance.

[0016] The unmanned boat system 1000 includes a single or multiple unmanned boats 1010. When the unmanned boat system 1000 is composed of multiple unmanned boats 1010, the multiple unmanned boats 1010 are connected to each other via wireless communication, allowing the configuration of a communication network. The unmanned boat 1010 also has the function of measuring targets 7000, including marine life such as submarines, underwater drones, divers, and whales moving underwater, ships on the sea, drifting objects, drifters, and aircraft in the sky, using measurement sensors mounted on the boat (such as sonar or other acoustic sensors, optical cameras, IR cameras, laser sensors such as LiDAR, and radar sensors such as millimeter-wave sensors and microwave sensors).

[0017] The detection determination results and measurement data of the object 7000 detected by the unmanned watercraft system 1000, as well as various types of behavior information for each unmanned watercraft 1010 in the unmanned watercraft system 1000, are transmitted to the integrated environment management system 2000 via the communications satellite 5100 and the terrestrial base station 5200. The integrated environment management system 2000 determines operation commands for the unmanned watercraft system 1000 based on information acquired from the unmanned watercraft system 1000 and request information acquired in advance from users, etc. The generated information, such as the operation commands, is transmitted to a user terminal device 8000 and displayed and output to the user. In addition, intervention commands, such as operation commands, can be acquired from the user via the user terminal device 8000.

[0018] (A-1-2. Example of real-world implementation of operational state simulation system 1) Fig. 2 is a diagram showing an example of an implementation image of the operational state simulation system 1 in real space. In the example shown in Fig. 2, a terrestrial base station 5200 and an integrated environment management system 2000 are provided on the ground side shown in the upper right of the drawing. Also provided on the ground side are a cooperative system 6000, an external system 4000, and a user terminal device 8000, all of which are connected to the integrated environment management system 2000 via a network.

[0019] On the other hand, on the ocean side shown on the left side of the drawing, unmanned boat system 1000 is deployed to search for target object 7000 in the ocean. Unmanned boat system 1000 also has multiple groups (1000a, 1000b, 1000c) each consisting of a parent unit and multiple child units, and can communicate wirelessly between each group either directly or via communication satellite 5100.

[0020] In the example shown in Figure 2, the integrated environmental management system 2000 is shown to be implemented in a facility on land, but this is not limited to this. All or some of the functions implemented in the integrated environmental management system 2000 shown in this embodiment can also be installed on a coastal field base located in a coastal area on land (not shown) or on a manned mother ship at sea, and the unmanned boat system 1000 can be operated and managed at the coastal field base or manned mother ship.

[0021] In the configuration of the embodiment described above with reference to FIGS. 1 and 2, a non-terrestrial network using a communication satellite 5100 in a geosynchronous orbit or a low Earth orbit has been used as a communication network for transmitting and receiving information between the integrated environmental management system 2000 and the unmanned watercraft system 1000. However, the present invention is not limited to this. A non-terrestrial network using an unmanned air vehicle known as a High Altitude Platform Station (HAPS) can also be used. In this case, for example, an unmanned air vehicle that circles at an altitude of approximately 8 to 50 km can be used. Furthermore, a communication network for transmitting and receiving information between the integrated environmental management system 2000 and the unmanned watercraft 1010 can also be used, in which a terrestrial base station 5200 directly connects the unmanned watercraft 1010 via wireless communication, without going through the communication satellite 5100 or the HAPS. The terrestrial base station 5200 is not limited to a fixed base station, and may be a mobile base station.

[0022] (A-1-3. Stakeholders regarding Operational State Simulation System 1) Fig. 3 is a diagram showing an example of stakeholders related to the operational state simulation system 1. As shown in Fig. 3, the operational state simulation system 1 has users who determine the operational requirements of the unmanned watercraft system 1000 by inputting and outputting information via the user interface unit 3300 of the operational application control system 3000.

[0023] The cooperative system 6000 also includes private monitoring organization facilities and surveillance boats. In this case, the cooperative system 6000 has workers at the private monitoring organization facilities and crew members on the surveillance boats, who work together to monitor nuisance behavior and the like in the marine area. In addition to private monitoring organizations, the cooperative system 6000 may also include marine research organizations that investigate marine life, private rescue organizations that search for and rescue people in distress, and marine infrastructure operators that patrol and inspect marine infrastructure.

[0024] The external system 4000 also includes a weather information system and an MDA system that provide information on oceanographic conditions (wave height, wave speed, ocean current speed, ocean current direction, tidal current speed, tidal current direction) in the area where the unmanned boat system 1000 is deployed and its surrounding areas, meteorological conditions (wind speed, wind direction, air pressure, temperature, humidity), weather conditions (fog, thunder, rainfall, snowfall, hail, graupel, cloudiness, etc.), seawater conditions (seawater temperature, seawater density, salinity, pH value, presence or absence of seaweed beds, etc.), sun-related information (sun position (altitude, direction, trajectory), backlight, frontlight, amount of solar radiation), and other conditions (lunar position (altitude, direction, trajectory, lunar age), ionospheric disturbances (solar flares, etc.)). In addition, the external system 4000 may be equipped with an AIS (Automatic Identification System) that acquires ship information about ships navigating the search area and its surrounding areas via wireless communication and manages this ship information.

[0025] Furthermore, in order to actually operate unmanned watercraft system 1000 in a marine area, it is necessary to carry out preparatory work before the actual operation to confirm whether unmanned watercraft system 1000 can achieve the required performance in a location, environment, and period similar to those of the actual operation, and to confirm operational requirements such as the number of vehicles necessary to achieve the performance required for the actual operation. For this reason, Figure 3 shows the actors involved in each task required before the actual operation of unmanned watercraft system 1000, including such preparatory work.

[0026] As shown in Figure 3, stakeholders in the operational state simulation system 1 include, in addition to the actors mentioned above, a resource procurement officer who handles the multiple unmanned vessels 1010 used in preliminary demonstration tests and actual operations, as well as the cargo and transport vehicles used for transportation, a resource transportation officer who transports the unmanned vessels 1010 and cargo using transport vehicles, a coastal standby / vehicle deployment officer who deploys the unmanned vessels 1010 at sea and manages the multiple unmanned vessels 1010 waiting on the coast, a vehicle recovery officer who recovers the unmanned vessels 1010 from the sea, a pre-operational preparation officer who manages all pre-operational preparation work, and a marine operation vehicle operator who operates the unmanned vessel 1010 during operations at sea.

[0027] (A-1-4. State transition of operational state simulation system 1) Next, we will explain the overall state transitions of the operational state simulation system 1, from determining the operational requirements for the actual operation in advance preparations before the actual operation, searching for an object during the actual operation, and ending the operation. Fig. 4 is a diagram showing an example of the state transitions of the operational state simulation system 1. In particular, the example shown in Fig. 4 shows the state transitions when the unmanned watercraft system 1000 is operated for the purpose of searching for an object 7000.

[0028] In the example shown in Figure 4, the operational state simulation system 1 has various states: a pre-operation preparation state before operation, an on-site preparation state immediately before operation, an actual operational operation state during operation, and a search end / cancellation state immediately before the end of operation.

[0029] In the pre-operation preparation state, first, user requirements are accepted from the user (status 101). In this status, if the purpose of the actual operation is to search for the target object 7000, the required conditions for the search (search target object, search area, search period, search target value (search rate, detection probability, etc.)) are accepted. Next, a demonstration test operation is partially carried out using an actual vehicle to evaluate the performance of the unmanned boat system 1000, and the operational requirements for the actual operation (number of unmanned boats 1010, movement route pattern, etc.) are determined (status 102). Next, resources such as unmanned boats 1010 to be used for the operation are procured according to the determined operational requirements (status 103). Next, the resources are transported to the site (status 104). Next, the resources are installed on the coast near the offshore area of ​​the actual operation (status 105).

[0030] Next, in the on-site preparation state immediately before operation, first, the unmanned vehicle 1010 is lowered onto the sea (status 201), and then the unmanned vehicle 1010 is deployed in the sea area for actual operation (status 202).

[0031] Next, the operational state has four states: a search state (status 301) in which a search for an object is performed, a standby state (status 302) in which the unmanned vessel 1010 waits in the operation area, and a recovery charge state (status 303) in which the battery mounted on the unmanned vessel 1010 is recharged using solar panels or the like when the SOC of the battery drops. After a target object is discovered, the unmanned vessel 1010 is made to perform an action such as tracking the target object, and the status transitions between these four states.

[0032] Next, the search end / cancellation state includes a return state (status 401) in which the unmanned watercraft 1010 is returned to the recovery position, and a recovery state (status 402) in which the unmanned watercraft is towed back to the shore or mother ship at the recovery position.

[0033] As shown in Fig. 4, the operational state simulation system 1 has an operational requirement determination operation status that checks whether the user's desired conditions can be met in the actual operation before entering the actual operational state, and what the operational requirements are for meeting the desired conditions. The operational state simulation system 1 also has a state transition management unit (not shown) that manages the state transitions of the system shown in Fig. 4 and Fig. 28 described later.

[0034] (A-2. Unmanned Boat System 1000) Next, the system configuration of an unmanned boat system 1000 according to one embodiment of the present invention will be described with reference to FIG.

[0035] (A-2-1. Composition of the group consisting of 1010 unmanned boats) Figure 5 is a diagram showing an example of the formation of a group made up of multiple unmanned craft 1010. The example shown in Figure 5 shows the formation of group 1000a made up of multiple unmanned craft 1010 and the communication connection relationships between each unmanned craft 1010 when the unmanned craft system 1000 is to perform a mission such as searching for an object.

[0036] 5, group 1000a is made up of parent device 1001 capable of wireless communication with communication satellite 5100, and child devices 1002 capable of direct or indirect wireless communication with parent device 1001. Parent device 1001 is connected to communication satellite 5100 by wireless communication, and has the function of aggregating information collected from multiple child devices 1002 and transmitting the information to communication satellite 5100, as well as transmitting information such as operation commands obtained from communication satellite 5100 and information generated by itself, directly or indirectly to each child device 1002.

[0037] 5 includes a single parent device 1001 and multiple child devices 1002. The parent device 1001 and the multiple child devices 1002 are connected via wireless communication as indicated by solid lines, forming a wireless communication network among multiple unmanned boats 1010 at sea. The group 1000a shown in FIG. 5 includes a primary connected child device 10021 that is wirelessly connected to the parent device 1001, and a secondary connected child device 10022 that is wirelessly connected to the primary connected child device 10021.

[0038] 5, the slave devices 1002 forming the group are configured to include a primary connected slave device 10021 and a secondary connected slave device 10022, but the group configuration is not limited to this and may be configured only with a primary connected slave device 10021 that is directly connected wirelessly to the master device. As another example, the slave devices 1002 forming the group may include a tertiary connected slave device, a quaternary connected slave device, or a higher connected slave device.

[0039] Furthermore, the number of secondary connected slave devices 10022 wirelessly connected to the primary connected slave device 10021 is not limited to one, and by having multiple secondary connected slave devices 10022 wirelessly connected to the primary connected slave device 10021, it is possible to form a tree-structured communication network in which multiple unmanned watercraft 1010 branch out within the group 1000a. Furthermore, because there is an upper limit to the communication distance over which wireless communication is possible between each unmanned watercraft 1010, the position of at least one of the unmanned watercraft 1010 that communicates wirelessly with each other, for example, the parent device 1001 and the primary connected slave device 10021, and the primary connected slave device 10021 and the secondary connected slave device 10022, is controlled so that the relative distance between the unmanned watercraft 1010 is maintained within the upper limit of the relative distance over which wireless communication is possible.

[0040] Furthermore, if the relative distance between the unmanned craft 1010 becomes so great that the unmanned craft 1010 with which the communication partner is communicating moves outside the range of the wireless communication distance, wireless communication between them will no longer be possible and control commands from the integrated environment management system 2000 will no longer be able to be sent. Therefore, it is desirable for two unmanned craft 1010 that are connected to each other for communication to perform self-position control with a higher priority than other controls in order to maintain the relative distance with the communication partner within the range of the communication distance.

[0041] On the other hand, there is no need to maintain the relative distance within a predetermined range between other unmanned watercraft 1010 that do not communicate with each other wirelessly in order to maintain the communication connection described above. On the other hand, if the purpose of the unmanned watercraft system 1000 is to search for the target object 7000, the search can be performed more efficiently if the unmanned watercraft 1010 maintain an appropriate distance between them so that the measurement ranges of their measurement sensors do not overlap or only overlap moderately, rather than if the unmanned watercraft 1010 are too close and the measurement ranges of their measurement sensors overlap to a large extent. Therefore, the relative distance between the unmanned watercraft 1010 that do not communicate with each other is controlled with a relatively low priority, so that a predetermined search relative distance is maintained. This control to maintain the search relative distance can be based on the Boids algorithm, for example.

[0042] Furthermore, if the relative distance between the unmanned craft 1010 becomes too close and there is a possibility of a collision, position control can be performed to increase the relative distance with a relatively high priority in order to avoid a collision and prevent damage to the unmanned craft 1010.

[0043] As described above, control to maintain the relative distance between unmanned vessels 1010 that communicate with each other by wireless communication within the communication distance range, and avoidance control to avoid collision with other unmanned vessels that come within close range, are executed with a relatively high priority, while control to maintain the relative distance during search between unmanned vessels 1010 that do not communicate with each other by wireless communication can be executed with a relatively low priority.

[0044] (A-2-3. Configuration of Unmanned Vehicle 1010) Figure 6 is a functional block diagram showing the functional configuration of the unmanned watercraft 1010. Note that while Figure 6 illustrates the functional block diagram of the unmanned watercraft 1010, the master unit 1001 and slave unit 1002 of the unmanned watercraft 1010 can be implemented with common functional units, with the exception of the configuration of the measurement unit 1100 and communication unit 1400. The unmanned watercraft 1010 is equipped with a measurement unit 1100, a vessel status determination unit 1200, a navigation unit 1300, a communication unit 1400, a determination unit 1500, a recording unit 1600, and a power supply unit 1700.

[0045] The measurement unit 1100 is a functional unit that acquires measurement data of an object 7000 that exists in the sea, on the sea, or in the air using a measurement sensor 1110. The measurement unit 1100 includes the measurement sensor 1110 and a measurement control unit 1120.

[0046] When searching for an object in an underwater area as the search target area, the measurement sensor 1110 may be, for example, a sonar or other sonic sensor that uses ultrasonic or other sound waves. When searching for an object in an above-sea or above-air area as the search target area, the measurement sensor 1110 may be, for example, a radar sensor that detects the object 7000 by emitting radio waves such as millimeter waves or microwaves and receiving reflected waves, a laser sensor such as LiDAR that acquires point cloud data by irradiating laser light and receiving the reflected laser light, or an electro-optical sensor that acquires optical image data, an infrared sensor (IR camera), or other optical camera.

[0047] Furthermore, in addition to the above-mentioned sensors, the measurement sensor 1110 may also be composed of a seawater condition measurement sensor that measures seawater conditions such as the salinity concentration, hydrogen ion exponent (pH), water temperature, seawater components, and density of seawater, or a sea state measurement sensor that measures sea state conditions such as ocean currents, tidal currents, wave height, wave period, and speed of ocean or tidal currents in the surrounding sea area, or a meteorological measurement sensor that measures meteorological conditions such as temperature, humidity, wind speed, solar radiation, air pressure, rainfall, other weather conditions, and air quality in the surrounding sea, or a marine ecology measurement sensor that measures the state of seaweed beds and plankton in the sea.

[0048] The measurement control unit 1120 operates a sensor attitude changing device that can change the attitude of the measurement sensor 1110 to control at least one of the attitude angles of the measurement sensor 1110 around three axes relative to the unmanned watercraft 1010. Furthermore, for example, if the measurement sensor is an electro-optical sensor, an infrared camera, or other optical camera, the measurement control unit 1120 can adjust the frame rate, shutter speed, etc. If the measurement sensor is a laser sensor, the measurement control unit 1120 can adjust the output of the irradiated laser. If the measurement sensor is a sonar or radar sensor, the measurement control unit 1120 can adjust the output of the emitted sound waves, millimeter waves, microwaves, etc. Furthermore, the measurement control unit 1120 can adjust the measurement sensitivity of the measurement sensor to a desired control amount. If the measurement sensor is an optoelectronic optical sensor, the measurement control unit 1120 can change the zoom amount or resolution of the optical sensor to a desired control amount.

[0049] Next, the unmanned watercraft state determination unit 1200 includes a navigation state determination unit 1210, an internal state determination unit 1220, and an external state determination unit 1230, and is a functional unit that determines the navigation state, internal state, and external state of the unmanned watercraft 1010. The navigation state determination unit 1210 determines the position (two-dimensional or three-dimensional), movement speed, heading, movement direction, movement acceleration / deceleration, turning speed, and other state quantities related to the navigation state of the unmanned watercraft. The internal state determination unit 1220 determines the remaining energy and fuel levels of the battery installed in the unmanned watercraft, the possible travel distance that can be calculated from the remaining energy and fuel levels, temporary abnormal states of equipment installed in the unmanned watercraft (temperature abnormality, communication abnormality, etc.), and equipment failure states.

[0050] In addition, the external condition determination unit 1230 can determine communication quality conditions such as communication strength (dB value, etc.), communication speed, and communication delay of wireless communications with other unmanned boats 1010 in the same group of the unmanned boat system 1000, or wireless communications with the integrated environment management system 2000 via a communication satellite 5100 or a terrestrial base station 5200, or the oceanographic conditions around the unmanned boat system (wave height, wave speed, ocean current speed, ocean current direction, tidal current speed, tidal current direction), meteorological conditions (wind speed, wind direction, air pressure, temperature, humidity), weather conditions (fog, thunder, rainfall, snowfall, hail, graupel, cloudiness, etc.), seawater conditions (seawater temperature, seawater density, salinity, pH value, presence or absence of seaweed beds, etc.), sun-related information (sun position (altitude, direction, trajectory), backlight, frontlight, amount of solar radiation), and other conditions (lunar position (altitude, direction, trajectory, lunar age), ionospheric disturbances (solar flares, etc.)).

[0051] The method by which the navigation state determination unit 1210 determines the position, moving speed, moving direction, and acceleration / deceleration of the aircraft is not particularly limited, but for example, the position, moving speed, and moving direction of the aircraft at the current time can be determined using GNSS (Global Navigation Satellite System), GPS (Global Positioning System), RTK-GNSS (Real Time Kinematic - Global Navigation Satellite System), etc.

[0052] As another example of a method for determining the position, speed, direction, and acceleration / deceleration of the aircraft by the navigation state determination unit 1210, for example, when the seabed shape can be detected by the measurement sensor 1110, the position, speed, and direction of movement of the aircraft at the current time can be determined using SLAM (Simultaneous Localization And Mapping) technology based on the pre-recorded seabed shape and the seabed shape detected by the measurement sensor 1110.

[0053] Here, the self-position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably includes three-dimensional coordinate information including altitude information. Also, the acceleration / deceleration can be calculated based on the amount of change over time in the determined moving speed.

[0054] The method for measuring the aircraft's heading is to determine the aircraft's heading at the current time using, for example, a geomagnetic sensor, a GNSS compass, or SLAM technology using the seabed shape. The heading includes an attitude angle (orientation) in a planar view around at least the Z axis, and preferably may be attitude information around three axes: the X axis, the Y axis, and the Z axis. The turning speed can be calculated based on the amount of change over time in the determined heading information.

[0055] Next, the navigation unit 1300 includes a thrust generating unit 1310, an attitude control mechanism 1320, and a navigation control unit 1330, and is a functional unit that navigates the aircraft in any direction in accordance with operational commands received via the communication unit 1400. The thrust generating unit 1310 can be any means capable of generating thrust, and as an example, can be configured with a propeller driven by the power of an engine or an electric motor. The thrust generating unit 1310 can also be configured with a sail that receives wind to generate thrust, or with a wave glider that receives wave power to generate thrust.

[0056] The attitude control mechanism 1320 is composed of a rudder mounted on the airframe, a propeller attitude change mechanism that can change the attitude angle of the propeller (mainly the yaw angle around the Z axis), etc., and can control the nose direction (yaw angle) of the aircraft by changing these angles. In addition, a center of gravity position change mechanism that changes the position of a heavy object inside the aircraft using an actuator can also control the attitude angles of the aircraft, such as the roll angle around the X axis and the pitch angle around the Y axis.

[0057] The navigation control unit 1330 is a functional unit that controls the thrust generation unit 1310 and the attitude control mechanism 1320 to control the navigation operation of the aircraft. The navigation control unit 1330 has one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), an MPU, or a DSP), and is equipped with a processing unit that can access a memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more processing steps.

[0058] The processing unit includes a control module configured to control the navigation status of the aircraft. For example, the control module adjusts the aircraft's position on the sea surface, movement speed, movement acceleration / deceleration, heading, turning speed, and attitude angle around three axes. That is, the navigation control unit 1330 controls the navigation operation of the aircraft by causing the aircraft to perform various operations such as forward movement, reverse movement, acceleration, deceleration, and turning.

[0059] Next, the communication unit 1400 is a functional unit that communicates with other unmanned watercraft 1010 in the same group and with the integrated environment management system 2000. Here, the communication unit 1400 of the parent unit 1001 is equipped with both an inter-unmanned watercraft communication unit 1410 that can communicate wirelessly with other unmanned watercraft 1010 in the group, and an overall control communication unit 1420 that can communicate with the integrated environment management system 2000. On the other hand, the communication unit 1400 of the child unit 1002 is equipped with the inter-unmanned watercraft communication unit 1410 that can communicate wirelessly with other unmanned watercraft 1010 in the group, and does not need to be equipped with the overall control communication unit 1420.

[0060] The unmanned craft-to-unmanned craft communication unit 1410 is a functional unit that has a communication antenna used for a maritime wireless communication network and communicates with other unmanned crafts 1010 in the same group that are within the communication distance. The overall control communication unit 1420 has a satellite communication antenna that can communicate with the communication satellite 5100 or a communication antenna that can communicate directly with the terrestrial base station 5200, and communicates with the integrated environmental management system 2000 via the communication satellite 5100 or the terrestrial base station 5200. In addition to the communication units described above, the communication unit 1400 may also have a communication unit that has an AIS antenna or a VHF antenna and that communicates with external surveillance vessels and AIS base stations.

[0061] Next, the determination unit 1500 is a functional unit that performs data processing such as primary processing and data compression of the measurement data acquired by the measurement sensor 1110. For example, the determination unit 1500 can perform data processing of raw data (measurement data) after measurement acquired by the measurement sensor 1110, and perform primary processing to generate transmission data to be wirelessly transmitted from the unmanned watercraft system 1000 to the integrated environment management system 2000. Furthermore, in order to reduce the transmission load when wirelessly transmitting transmission data from the unmanned watercraft system 1000 to the integrated environment management system 2000, the determination unit 1500 can perform data compression processing to compress the raw data (measurement data) after measurement to generate transmission data.

[0062] Furthermore, by performing primary processing of the measurement data, the determination unit 1500 can interpret the state of the target object 7000, determining the presence or absence of a detected object, the size of the detected object, etc. The determination unit 1500 may also have a function to determine whether or not measurement data or transmission data needs to be sent from the unmanned watercraft system 1000 to the integrated environment management system 2000, or to select the data to be sent, depending on the interpretation results.

[0063] Next, the recording unit 1600 includes a measurement data recording unit 1610, a host device state recording unit 1620, and a determination information recording unit 1630. The measurement data recording unit 1610 records measurement data measured by the measurement unit 1100. The host device state recording unit 1620 records various state information related to the host device determined by the host device state determination unit 1200. Furthermore, the determination information recording unit 1630 records various determination information determined by the determination unit 1500.

[0064] Next, the power supply unit 1700 is a functional unit that supplies power to each functional unit of the unmanned watercraft 1010. The power supply unit 1700 includes a power storage device 1710, a power generation device 1720, and a power control unit 1730. The power storage device 1710 can be configured as a battery or other device with a power storage function. The power generation device 1720 can be configured as a solar panel that generates power using sunlight or a wave power generation device that generates power using wave power. The power control unit 1730 is a functional unit that controls the charging and discharging of the power storage device 1710 and also controls the power generation operation of the power generation device 1720.

[0065] (A-3. Operation of the Integrated Environment Management System 2000, etc.) Next, an example of the operation of the integrated environment management system 2000 and the operational application control system 3000 will be described with reference to FIGS.

[0066] (A-3-1. Role of the Integrated Environment Management System 2000 in the Operational Requirements Decision State) 7 is a diagram showing an example of the operation of the integrated environment management system 2000 in an operational requirements determination state. In particular, FIG. 7 shows an example of how, when determining operational requirements in the operational requirements determination state of status 102 shown in FIG. 4, a mixed unmanned aerial vehicle group, which includes a real vehicle deployed in an operational area and whose operation is controlled by the integrated environment management system 2000, and a plurality of virtual vehicles placed in an operational area in a virtual space generated by the integrated environment management system 2000, is deployed in an operational area of ​​the virtual space, and the operational status of the mixed unmanned aerial vehicle group is simulated and calculated, performance of the operational status is evaluated, and the operational requirements for this operation are determined. In the example shown in FIG. 7, the parent vehicle of the mixed unmanned aerial vehicle group is configured as a real vehicle, and the child vehicle is configured as a virtual vehicle.

[0067] 7 shows the positions and orientations of the real and virtual machines deployed in the operation area of ​​the virtual space at a given time. In other words, the real machines are actually deployed in the operation area of ​​the real space, and the behavior of the real machines is controlled based on real machine control commands sent from the integrated environment management system 2000. Behavior information of the real machines (position, orientation, etc.) is sent from the real machines to the integrated environment management system 2000, and the behavior information of the real machines is reflected in the operation area of ​​the virtual space generated by the integrated environment management system 2000.

[0068] Furthermore, the virtual machine is not deployed in the operation area of ​​the real space, and the behavior state (position, orientation, etc.) of the virtual machine is calculated by the integrated environment management system 2000, and the behavior state of the virtual machine is reflected in the operation area of ​​the virtual space. At this time, the integrated environment management system 2000 calculates the behavior state of the virtual machine taking into consideration the environmental state of the real space grasped from behavior information acquired from the real machine and environmental information acquired from the external system 4000, thereby simulating the actual behavior of the virtual machine when it is operated in the real space.

[0069] The integrated environment management system 2000 determines operational requirements that enable a mixed group of real and virtual unmanned aircraft, such as those described above, to meet constraints such as not colliding with each other and maintaining relative distances within the communication range in the operational area of ​​the virtual space, while also meeting the user's desired conditions for this operation.

[0070] (A-3-2. Input / output information of the convergence environment management system 2000 in the operational requirements determination state) Fig. 8 is a diagram showing an example of input / output information of the integrated environment management system 2000 in the operational requirements determination state. Fig. 8 particularly shows input information input to the integrated environment management system 2000 and output information output from the integrated environment management system 2000 in the desired conditions reception state (status 101) and the operational requirements determination state (status 102) shown in Fig. 4.

[0071] As shown in Figure 8, the input information includes pre-acquired information input to the integrated environment management system 2000 in the desired conditions acceptance state (status 101) and real-time processing information acquired during processing execution of the integrated environment management system 2000 in the operational requirements determination state (status 102).

[0072] The pre-acquired information includes, for example, user request information acquired from the user, assumed environment information assumed in the operation area where the actual operation will be performed, and assumed condition parameters including various assumed conditions for the actual operation. The real-time processing information is, for example, information acquired during processing execution by the integrated environment management system 2000, and includes a user intervention command acquired from the user, actual environment measurement data acquired from the external system 4000 and the external state determination unit 1230 of the actual aircraft, actual aircraft behavior information acquired from the navigation state determination unit 1210 of the actual aircraft, actual aircraft state information including internal state information of the actual aircraft such as SOC acquired from the internal state determination unit 1220 of the actual aircraft, and actual aircraft measurement data measured by the measurement unit 1100 of the actual aircraft.

[0073] As shown in Figure 8, the output information is information output from the integrated environment management system 2000 in the operational requirements determination state (status 102), and includes operational requirements information necessary for preparing for operation such as subsequent status 103, display data to be displayed on the display unit 3310 of the operational application control system 3000 and the user terminal device 8000, and control data for the actual equipment deployed in the operational area in the real space.

[0074] The display data includes, for example, drone group behavior information showing the behavior of a group of drones that is a mixture of real and virtual drones, drone status information showing the internal status of each drone, such as SOC, and real drone measurement data measured by the measurement unit 1100 of the real drone.

[0075] (A-3-3. Example of operation of convergence environment management system 2000 in operational requirements determination state) Next, the operational process and information flow of the integrated environment management system 2000 in the operational requirements determination state will be described with reference to Figures 9 and 10. Figure 9 is a diagram showing an example of input / output information between functional units of the integrated environment management system 2000 in the operational requirements determination state. Figure 10 is a process diagram showing an example of a processing process of the integrated environment management system 2000 etc. in the operational requirements determination state.

[0076] As shown in Figure 9, the integrated environment management system 2000 includes an information acquisition unit 2100, a real environment information acquisition unit 2200, a virtual space generation unit 2300, an unmanned vessel behavior estimation unit 2400, a group control command determination unit 2500, an operational requirement determination unit 2600, and an information output unit 2700.

[0077] First, in a desired condition receiving state (status 101 ), pre-acquired information including user desired information, assumed environment information, assumed condition parameters, etc. is input from the operational application control system 3000 to the information acquiring unit 2100 .

[0078] Next, the information acquisition unit 2100 inputs the acquired request information and assumed environment information to the virtual space generation unit 2300 and the group control command determination unit 2500. The virtual space generation unit 2300 generates a virtual operation area that simulates the operation area in the virtual space, based on the operation area and operation time included in the acquired request information, as well as the assumed environment information.

[0079] Next, in the operational requirements determination state (status 102), after the actual aircraft is deployed in the real space, actual aircraft behavior information is input from the actual aircraft deployed in the operational area of ​​the real space to the real environment information acquisition unit 2200. In addition, actually measured environmental information is input from the external system 4000 and the actual aircraft to the real environment information acquisition unit 2200. The real environment information acquisition unit 2200 inputs the acquired actual aircraft behavior information to the unmanned watercraft behavior estimation unit 2400, and also inputs the acquired environmental information to the virtual space generation unit 2300.

[0080] Next, the virtual space generation unit 2300 updates the environmental state in the generated virtual space based on the acquired environmental information. The updated environmental state in the virtual space is input to the unmanned craft behavior estimator 2400 and the group control command determiner 2500.

[0081] Next, the unmanned watercraft behavior estimation unit 2400 calculates the behavior state of all aircraft, including the virtual aircraft and the actual aircraft, based on the actual aircraft behavior information acquired from the actual environment information acquisition unit 2200, the environmental state acquired from the virtual space generation unit 2300, and the assumed condition parameter information acquired from the information acquisition unit 2100, and also based on virtual aircraft model information that simulates the virtual aircraft, and outputs the calculated behavior state information of all aircraft to the group control command determination unit 2500.

[0082] Here, virtual machine model information refers to model information that simulates a virtual machine in a simulation, and may be a model in which all functional units of the virtual machine are simulated in a virtual space, or it may be configured as a HILS (Hardware In the Loop Simulator) machine in which some functional units of the virtual machine are configured with actual hardware and other functional units are configured with virtual models to perform simulation.

[0083] Next, the group control command determination unit 2500 generates control commands for all vehicles, including virtual vehicles and real vehicles, based on the behavior state information of all vehicles acquired from the unmanned watercraft behavior estimation unit 2400, the environmental state information acquired from the virtual space generation unit 2300, and the request information acquired from the information acquisition unit 2100, and outputs the generated control commands to the information output unit. The group control command determination unit 2500 also outputs the behavior state information of all vehicles to the information output unit 2700.

[0084] Furthermore, the group control command determination unit 2500 determines whether the current behavior state information of all aircraft satisfies the target information included in the request information acquired from the information acquisition unit 2100, and outputs the determination result to the operational requirement determination unit 2600. The operational requirement determination unit 2600 determines operational requirements based on the acquired determination result and the current behavior state information of all aircraft, and outputs the determined operational requirements to the information output unit 2700. Note that when the information acquisition unit 2100 acquires user intervention information from the user, the group control command determination unit 2500 and the operational requirement determination unit 2600 perform a determination process and a determination process of the operational requirements, respectively, based on the user intervention information.

[0085] The information output unit 2700 outputs display data including the acquired behavior status information of all aircraft and operational requirements to the operational application control system 3000. The information output unit 2700 also transmits the acquired control commands to the actual aircraft of the unmanned boat system 1000.

[0086] The operational application control system 3000 can also acquire the behavior information of the actual vehicle included in the display data directly from the actual vehicle of the unmanned watercraft system 1000, rather than from the information output unit 2700. The operational application control system 3000 can also acquire the measurement data of the actual vehicle included in the display data directly from the actual vehicle. The operational application control system 3000 can also acquire the environmental information of the operation area directly from the external system 4000, rather than from the information output unit 2700.

[0087] (A-4. Detailed functions of the Integrated Environmental Management System 2000) Next, detailed functions of the integrated environment management system 2000 will be described using Fig. 11. Fig. 11 is a functional block diagram showing the functional configuration of the integrated environment management system 2000. The integrated environment management system 2000 includes an information acquisition unit 2100, a real environment information acquisition unit 2200, a virtual space generation unit 2300, an unmanned vessel behavior estimation unit 2400, a group control command determination unit 2500, an operational requirements determination unit 2600, and an information output unit 2700.

[0088] (A-4-1. Information acquisition department 2100) The information acquisition unit 2100 is a functional unit that acquires various information required for processing by each functional unit of the integrated environment management system 2000 (described later) from the user interface unit 3300 of the operational application control system 3000 and the user terminal device 8000. The information acquisition unit 2100 includes a user request information acquisition unit 2110, an assumed environment information acquisition unit 2120, an assumed condition parameter acquisition unit 2130, and a user intervention information acquisition unit 2140.

[0089] The user request information acquisition unit 2110 is a functional unit that acquires user request information including the request conditions required for actual operation from the user in the request condition acceptance state (status 101) prior to the demonstration test operation that determines the performance evaluation and operational requirements for the unmanned watercraft system 1000. Figure 13 is a diagram showing an example of user request information acquired by the user request information acquisition unit 2110.

[0090] 13, the user request information includes, for example, requested condition information and goal information. The requested condition information includes information that can identify the object to be searched, information that can identify the operation area (information specifying the location, range, etc. of the operation area), operation period (date and time of search execution, start and end times, time period), etc.

[0091] The target information also includes a target search rate related to a target value for the search rate (also called coverage rate) indicating the ratio of the cumulative measurement area measured by the measurement sensor of the unmanned vessel 1010 during the operation period to the operation area, and a target search performance related to a target value for the detection probability (also called target discovery probability, detection rate, or encounter rate) indicating the probability that the unmanned vessel 1010 will detect the target object 7000 using the measurement sensor. Here, the target search rate may include an entire area target search rate, which is a target value related to the ratio of areas that the unmanned vessel 1010 has moved or measured to the entire operation area to be searched, or a local area target search rate, which is a target value related to the ratio of areas that the unmanned vessel 1010 has moved or measured to a local area that is part of the operation area to be searched.

[0092] Note that Figure 13 shows an example in which search rate, detection probability, etc. are set in advance as target information for the actual operation of the unmanned boat system 1000, but the target information is not limited to this and may include, for example, target values ​​related to the average or minimum value of the charge amount, charge rate, or charge capacity of the power storage devices of multiple unmanned boats (remaining charge (Wh), charge rate (%), power receiving capacity (Wh)), or target values ​​related to the number of unmanned boats that have been swapped with waiting replacement machines due to a lack of charge amount, charge rate, or charge capacity of the power storage devices.

[0093] The target information may also include a target value for the operating time, which is the time during which the unmanned watercraft system 1000 operates in the operating area. Here, operating time refers to the cumulative time during which the unmanned watercraft system 1000 operates normally in the operating area, excluding times when the unmanned watercraft system 1000 is unable to operate due to a dead battery or equipment failure.

[0094] 13 shows an example of target information when the purpose of the actual operation of the unmanned watercraft system 1000 is to search for the target object 7000, but different target information can be set depending on the intended use of the unmanned watercraft system 1000 in this operation. For example, if the mission of the unmanned watercraft system 1000 includes a wireless communication mission to connect wireless communication with the communication unit 1400 mounted on the unmanned watercraft to equipment present in the operation area or the area surrounding the operation area, the target information can include target values ​​related to the communication quality of wireless communication between the equipment and the unmanned watercraft, or the area in which wireless communication with the unmanned watercraft is possible, or the time period in which wireless communication with the unmanned watercraft is possible.

[0095] As another example, if the mission of the unmanned boat system 1000 includes a measurement data acquisition mission to measure the operation area or the area surrounding the operation area using a measurement sensor 1110 mounted on the unmanned boat, the target information may include target values ​​for the data measurement time for measurement by the measurement sensor 1110, the amount of measurement data obtained by the measurement, or the size of the data measurement area for measurement.

[0096] The above-mentioned various types of target information may be set as minimum allowable values ​​(target values) to be exceeded, or may be set as allowable ranges (target ranges) to be kept within.

[0097] The assumed environment information acquisition unit 2120 is a functional unit that acquires environmental information related to environmental disturbances expected during operation time in the operation area included in the desired condition information. The environmental information acquired by the assumed environment information acquisition unit 2120 includes environmental information including at least one of meteorological information (wind speed, wind direction, rain, snow, cloudiness, fog, solar radiation, brightness, visibility, temperature, other weather, etc.) and oceanographic information (wave height, speed, direction, position of ocean currents and tidal currents, etc.).

[0098] The assumed condition parameter acquisition unit 2130 is a functional unit that acquires various condition parameters assumed when operating the unmanned boat system 1000 in the operation area. Fig. 14 is a diagram showing an example of the assumed condition parameters acquired by the assumed condition parameter acquisition unit 2130.

[0099] As shown in Figure 14, the assumed condition parameters include, for example, assumed aircraft conditions for the unmanned vessel 1010, assumed conditions for the target object 7000, environmental impact conditions which are the degree to which environmental disturbances affect various performance characteristics of the unmanned vessel 1010, and other virtual conditions.

[0100] The assumed vehicle conditions also include, for example, the arrangement conditions of the multiple unmanned craft 1010 (random arrangement, uniform arrangement, density-changing arrangement, etc.), the movement conditions of each unmanned craft 1010 (movement speed, turning speed, movement direction, movement path, straight-line travel time, etc.), navigation performance conditions related to the navigation performance of the unmanned craft 1010 (maximum movement speed, maximum turning speed, maximum deceleration, power consumption per speed, etc.), and measurement sensor conditions related to the measurement sensors 1110 mounted on the unmanned craft 1010 (type of measurement sensor, measurement These include communication device conditions for the communication unit 1400 mounted on the unmanned boat 1010 (communication range, other communication performance, etc.), power storage device conditions for the power storage device 1710 such as a battery mounted on the unmanned boat 1010 (charging capacity, output performance (W), charging performance (W), etc.), and power generation device conditions for the power generation device 1720 such as a solar panel or wave power generation device mounted on the unmanned boat 1010 (power generation performance (W), etc.).

[0101] The target assumption conditions include, for example, search target assumption conditions, which are assumption information related to the target. The search target assumption conditions include, for example, the presence probability indicating the probability that the target exists in the operation area, the expected speed, expected path, expected straight-ahead time of the target, etc.

[0102] The environmental influence conditions also include, for example, a mobility performance influence indicating the degree to which environmental disturbances affect the mobility performance of the unmanned watercraft 1010. The mobility performance influence includes information regarding the degree to which ocean currents, wave height, wind speed, wind direction, water temperature, etc. affect the mobility performance of the unmanned watercraft 1010 using the navigation unit 1300. The environmental influence conditions also include a measurement performance influence indicating the degree to which environmental disturbances affect the measurement performance of the measurement sensor 1110. The measurement performance influence includes information regarding the degree to which wave height, visibility, brightness, etc. affect the measurement performance of the measurement sensor 1110. The environmental influence conditions also include a communication performance influence indicating the degree to which environmental disturbances affect the communication performance of the communication unit 1400. The communication performance influence includes information regarding the degree to which clouds, fog, wave height, etc. affect the communication performance of the communication unit 1400.

[0103] The environmental impact condition also includes a power storage performance impact indicating the degree to which environmental disturbances affect the power storage performance of the power storage device 1710. The power storage performance impact particularly includes information on the degree to which air temperature and water temperature affect the power storage performance of the power storage device 1710, such as a battery. The environmental impact condition also includes a power generation performance impact indicating the degree to which environmental disturbances affect the power generation performance of the power generation device 1720. If the power generation device 1720 is a solar panel, the power generation performance impact particularly includes a power generation performance impact indicating the degree to which solar radiation and air temperature affect the power generation performance of the solar panel, and if the power generation device 1720 is a wave power generation device, the power generation performance impact particularly includes a power generation performance impact indicating the degree to which wave height affects the power generation performance of the wave power generation device.

[0104] Other virtual conditions may also include information regarding hypothetical scenarios that may occur during operation, such as a scenario in which a large ship such as a regular sailing ship approaches the operating area of ​​the unmanned boat system 1000 (the expected movement path, movement speed, ship size, etc. of the large ship), or a scenario in which a submersible, which is the object of search, approaches the operating area (the expected movement path, movement speed, etc. of the submersible in the three-dimensional underwater space).

[0105] Next, the user intervention information acquisition unit 2140 is a functional unit that acquires intervention information from the user when the integrated environment management system 2000 is simulating in real time the behavior of a mixed machine group in which real machines and virtual machines are mixed in the virtual space in synchronization with the behavior of the real machines in the operational requirements determination state status (status 102). Specific examples of the intervention information acquired by the user intervention information acquisition unit 2140 will be described later.

[0106] (A-4-2. Real environment information acquisition unit 2200) The real environment information acquisition unit 2200 is a functional unit that acquires actual behavior information of the real machine and actual environmental information of the operation area from the real machine deployed in the operation area in the real space or from the external system 4000. The real environment information acquisition unit 2200 includes an actual machine behavior information acquisition unit 2210, an actual machine measurement data acquisition unit 2220, an environment measurement data acquisition unit 2230, and an environment prediction data acquisition unit 2240.

[0107] The actual aircraft behavior information acquisition unit 2210 is a functional unit that acquires actual aircraft behavior information determined by the actual aircraft navigation state determination unit 1210 from the actual aircraft deployed in the real-space operation area after the actual unmanned watercraft 1010 is deployed in the real-space operation area. Here, the actual aircraft behavior information includes the aircraft's position (two-dimensional or three-dimensional), movement speed, heading, movement direction, movement acceleration / deceleration, turning speed, and other state quantities related to the navigation state.

[0108] The actual machine measurement data acquisition unit 2220 is a functional unit that acquires actual machine measurement data measured by the actual machine measurement unit 1100 of the unmanned watercraft 1010.

[0109] The environmental measurement data acquisition unit 2230 is a functional unit that acquires environmental measurement data related to environmental information from the external system 4000 or the external condition determination unit 1230 of the actual unmanned boat 1010, including at least one of the following: meteorological conditions (wind speed, wind direction, atmospheric pressure, temperature, humidity), weather conditions (fog, thunder, rainfall, snowfall, hail, graupel, cloudiness, etc.), sea state information (wave height, wave speed, ocean current speed, ocean current direction, tidal current speed, tidal current direction), other seawater conditions (seawater temperature, seawater density, salinity, pH value, presence or absence of seaweed beds, etc.), sun-related information (sun position (altitude, direction, trajectory), backlight, frontlight, amount of solar radiation), and other conditions (lunar position (altitude, direction, trajectory, lunar age), ionospheric disturbances (solar flares, etc.)) in the area where the unmanned boat system 1000 is deployed and its surrounding areas.

[0110] The environmental prediction data acquisition unit 2240 is a functional unit that acquires environmental prediction data after a predetermined time has elapsed in a predetermined operation area in which the unmanned watercraft 1010 is operated. The environmental prediction data acquired by the environmental prediction data acquisition unit 2240 includes, similar to the environmental measurement data acquired by the environmental measurement data acquisition unit 2230, prediction data on environmental information including at least one of meteorological conditions, sea state information, other seawater conditions, sun-related information, and other conditions.

[0111] (A-4-3. Virtual space generation unit 2300) The virtual space generation unit 2300 is a functional unit that generates a virtual operation area within a virtual space that simulates the operation area based on the operation area included in the request information acquired by the user request information acquisition unit 2110 and the assumed environment information acquired by the assumed environment information acquisition unit 2120.

[0112] For example, the virtual space generation unit 2300 generates a virtual operation area in the virtual space with the same position and range as the operation area based on information about the operation area included in the request information (such as information about the area's position and range).

[0113] Furthermore, the virtual space generation unit 2300 simulates an environment similar to the assumed environment information (e.g., ocean currents, wave height, wind speed, wind direction, amount of solar radiation, brightness, visibility, weather, etc.) within the virtual operation area based on the assumed environment information. Furthermore, the virtual space generation unit 2300 can simulate events such as the approach of a large ship or a submarine within the virtual space based on the setting information for other virtual conditions shown in Fig. 14.

[0114] Furthermore, when the unmanned vessel behavior estimation unit 2400 described below calculates behavior calculation information for a mixed unmanned vessel group that includes both real and virtual vessels in real time, and the latest environmental measurement data is acquired by the environmental measurement data acquisition unit 2230, the environmental state of the already generated virtual operation area can be updated to match the latest environmental measurement data acquired by the environmental measurement data acquisition unit 2230.

[0115] In other words, when generating a virtual operation area, the virtual space generation unit 2300 sets the environmental state parameters of the virtual operation area to match the expected environmental information, and when executing a simulation to calculate the behavior of the mixed unmanned aircraft group in real time, it can update the environmental state parameters of the virtual operation area to match the latest environmental measurement data.

[0116] (A-4-4. Unmanned boat behavior estimation unit 2400) The unmanned watercraft behavior estimation unit 2400 is a functional unit that calculates the behavior (behavior calculation information) of a mixed unmanned watercraft group that is a mixture of virtual unmanned watercraft 1010 deployed in a virtual operation area generated in a virtual space and actual unmanned watercraft 1010 actually deployed in an operation area in real space. The unmanned watercraft behavior estimation unit 2400 includes a virtual watercraft behavior calculation unit 2410, a mixed watercraft group behavior generation unit 2420, and an actual watercraft behavior calculation unit 2430.

[0117] The virtual aircraft behavior calculation unit 2410 is a functional unit that calculates virtual aircraft behavior calculation information relating to the aircraft behavior of the virtual aircraft deployed in a virtual operation area generated in a virtual space, based on virtual aircraft model information.

[0118] The virtual aircraft behavior calculation unit 2410 first generates or acquires virtual aircraft model information capable of simulating and calculating the behavior of the virtual aircraft. Here, the virtual aircraft model information is generated based on information on various condition parameters such as those shown in FIG. 14 acquired by the assumed condition parameter acquisition unit 2130. For example, the virtual aircraft behavior calculation unit 2410 sets parameters that define the performance of the virtual aircraft model, such as the navigation performance, measurement sensor, communication device, power storage device, and power generation device, based on the assumed aircraft conditions. Furthermore, the virtual aircraft behavior calculation unit 2410 sets parameters that define the degree of influence that the environmental state in the virtual operation area has on the mobility performance, measurement performance, communication performance, power storage performance, and power generation performance of the virtual aircraft model, based on the environmental impact conditions.

[0119] The virtual aircraft behavior calculation unit 2410 calculates virtual aircraft behavior calculation information related to the aircraft behavior of the virtual aircraft based on operation commands for the virtual aircraft generated by the group control command determination unit 2500 (described below), the environmental state of the virtual operation area, and virtual aircraft model information. Here, the virtual aircraft behavior calculation information calculated by the virtual aircraft behavior calculation unit 2410 includes at least one of the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of each virtual aircraft.

[0120] The virtual aircraft behavior calculation unit 2410 may calculate virtual aircraft behavior calculation information regarding the aircraft behavior of the virtual aircraft based on the environmental measurement data acquired by the environmental measurement data acquisition unit 2230 and the virtual aircraft model information, without using information on the virtual environment state in the virtual operation area generated or updated by the virtual space generation unit 2300.

[0121] Next, the mixed aircraft group behavior generation unit 2420 is a functional unit that generates behavior information of a mixed unmanned aircraft group that includes a mixture of real and virtual aircraft based on the real aircraft behavior information acquired by the real aircraft behavior information acquisition unit 2210 and the virtual aircraft behavior calculation information calculated by the virtual aircraft behavior calculation unit 2410.

[0122] For example, the mixed aircraft group behavior generation unit 2420 reflects the behavior of the actual aircraft in the virtual operation area based on information such as the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of the actual aircraft contained in the actual aircraft behavior information, and further reflects the behavior of the virtual aircraft in the virtual operation area based on information such as the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of the virtual aircraft contained in the virtual aircraft behavior calculation information, thereby reflecting the behavior of all aircraft in the unmanned aircraft group that is a mixture of real and virtual aircraft within the virtual operation area.

[0123] Next, the actual aircraft behavior calculation unit 2430 first models the actual aircraft in a virtual space and generates or acquires actual aircraft model information that enables simulation calculation of the behavior of the actual aircraft. Here, the actual aircraft model information is generated based on performance information of each functional unit installed in the actual aircraft and information on various condition parameters such as those shown in FIG. 14 acquired by the assumed condition parameter acquisition unit 2130. For example, the actual aircraft behavior calculation unit 2430 sets parameters that define the performance of the actual aircraft model, such as the navigation performance, measurement sensor, communication device, power storage device, and power generation device, based on the performance information of each functional unit installed in the actual aircraft. Furthermore, the actual aircraft behavior calculation unit 2430 sets parameters that define the degree of influence of environmental conditions in the virtual operation area on the mobility performance, measurement performance, communication performance, power storage performance, and power generation performance of the actual aircraft model, based on the environmental impact conditions.

[0124] The actual aircraft behavior calculation unit 2430 calculates actual aircraft behavior calculation information regarding the aircraft behavior of the actual aircraft based on the operation commands for the actual aircraft generated by the group control command determination unit 2500 described below, the environmental conditions of the virtual operation area, and actual aircraft model information.

[0125] Here, if the accuracy of the actual aircraft model is sufficiently high, the actual aircraft behavior calculation information calculated by the actual aircraft behavior calculation unit 2430 should be roughly consistent with the actual aircraft behavior information acquired by the actual aircraft behavior information acquisition unit 2210. If there is a large difference between the actual aircraft behavior calculation information and the actual aircraft behavior information, the accuracy of the actual aircraft model will be low, and in such a case, the actual aircraft behavior calculation unit 2430 has a function to correct the parameters of the actual aircraft model, and adjusts the parameters of the actual aircraft model according to the comparison result between the actual aircraft behavior calculation information and the actual aircraft behavior information.

[0126] Note that the actual machine behavior calculation unit 2430 obtains the comparison results between the actual machine behavior calculation information and the actual machine behavior information, and the parameter correction information of the actual machine model, which are information that should also be used to correct the parameters of the virtual machine model. Therefore, the virtual machine behavior calculation unit 2410 can obtain the comparison results between the actual machine behavior calculation information and the actual machine behavior information and the parameter correction information of the actual machine model from the actual machine behavior calculation unit 2430, correct the parameters of the virtual machine model based on this information, and calculate the virtual machine behavior calculation information using the corrected virtual machine model.

[0127] (A-4-5. Group Control Command Determination Unit 2500) The group control command determiner 2500 is a functional unit that determines operation commands for the real and virtual vessels, based on behavior information of the mixed unmanned vessel group, which is a mixture of real and virtual vessels, generated by the mixed vessel group behavior generator 2420, so as to prevent interference between the operations of the unmanned vessels 1010 in the mixed unmanned vessel group and communication interruptions between the unmanned vessels 1010, and so as to achieve the target information included in the user request information. The group control command determiner 2500 includes an unmanned vessel performance estimator 2510, an interference prediction determiner 2520, a communication interruption prediction determiner 2530, a goal achievement state determiner 2540, and an operation command determiner 2550.

[0128] The unmanned boat performance estimation unit 2510 estimates various performance characteristics of each unmanned boat in the mixed unmanned boat group based on the aircraft expected conditions and environmental impact conditions acquired by the expected condition parameter acquisition unit 2130 and the behavior calculation information of the mixed unmanned boat group.

[0129] For example, the unmanned boat performance estimation unit 2510 can estimate the measurement area in which each unmanned boat in the mixed unmanned boat group has taken measurements from the past to the present, and the measurement range that can be measured at the present and future times, based on information on the measurement sensor conditions (type of measurement sensor, measurable range, etc.), the measurement performance impact, the environmental conditions of the virtual operation area (particularly, wave height, fog, rainfall, snowfall, etc.), and behavior calculation information of the mixed unmanned boat group (particularly, the position information of each unmanned boat).

[0130] As another example, the unmanned vessel performance estimation unit 2510 can estimate the communication area in which each unmanned vessel in the mixed unmanned vessel group has been able to communicate from the past to the present, and the communication range and communication quality in which communication is possible at present and in the future, based on information on communication device conditions (communication distance, communication performance, etc.), communication performance impact, environmental conditions in the virtual operation area (particularly, wave height, fog, rainfall, snowfall, etc.), and behavior calculation information for the mixed unmanned vessel group (particularly, position information for each unmanned vessel).

[0131] As another example, the unmanned vessel performance estimation unit 2510 can estimate the position at a future time and the predicted future movement path of each unmanned vessel in the mixed unmanned vessel group based on the unmanned vessel's movement conditions (movement speed, turning speed, movement direction, movement path, straight-line travel time), navigation performance conditions (maximum movement speed, maximum turning speed, maximum deceleration, power consumption per speed, etc.), movement performance influence, environmental conditions of the virtual operation area (particularly, sea conditions and weather conditions), and behavior calculation information of the mixed unmanned vessel group (particularly, the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of each unmanned vessel).

[0132] As another example, the unmanned vessel performance estimation unit 2510 can estimate the current amount of power generated by the power generation device, the amount of power consumed by the navigation unit 1300, and the amount of charge in the power storage device of each unmanned vessel in the mixed unmanned vessel group based on the power storage device conditions (charging capacity, output performance, charging performance, etc.), the power generation device conditions (power generation performance, etc.), the navigation performance conditions (power consumption per speed), the power storage performance influence degree, the power generation performance influence degree, the environmental conditions of the virtual operation area (particularly, air temperature, water temperature, solar radiation, wave height, etc.), and the behavior calculation information of the mixed unmanned vessel group (particularly, the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of each unmanned vessel), and can further predict the amount of charge in the power storage device at a future time.

[0133] The interference prediction determination unit 2520 can predict whether an interference state will occur in which the unmanned vessels in the mixed unmanned vessel group will come into contact with or approach too closely to each other at a future time, based on the future positions and predicted future movement paths of each unmanned vessel in the mixed unmanned vessel group calculated by the unmanned vessel performance estimation unit 2510. In addition, based on the future positions and predicted future movement paths of each unmanned vessel and approach scenario information for large vessels included in the assumed condition parameters, it can predict whether an interference state will occur in which the unmanned vessel and large vessel will come into contact with or approach too closely to each other.

[0134] If the interference prediction determination unit 2520 predicts that an interference state will occur at a future time, the operation command determination unit 2550, described later, generates an operation command (including at least one command for position, attitude, movement speed, rotational angular velocity, movement acceleration, or rotational angular acceleration) for the unmanned vessel predicted to experience an interference state so that the interference state will not occur at a future time, for example, by increasing the relative distance of the unmanned vessel predicted to experience an interference state at a future time.

[0135] The communication interruption prediction determination unit 2530 can predict whether a communication interruption state will occur at a future time in which an unmanned vessel 1010 in the mixed unmanned vessel group will be unable to communicate wirelessly with other unmanned vessels 1010, based on the communication range and communication quality that each unmanned vessel in the mixed unmanned vessel group can communicate at the present and future times calculated by the unmanned vessel performance estimation unit 2510.

[0136] If the communication interruption prediction determination unit 2530 predicts that a communication interruption state will occur at a future time, the operation command determination unit 2550, which will be described later, generates an operation command for the unmanned vessel so that a communication interruption state will not occur at a future time, for example, so that another unmanned vessel is placed within the communication range of the unmanned vessel where a communication interruption state is predicted to occur.

[0137] The target achievement state determination unit 2540 can calculate actual values ​​of the operational state from the past to the present or predicted values ​​of the future operational state as information on the operational state corresponding to the target information included in the user request information, based on the various performance estimation information calculated by the unmanned watercraft performance estimation unit 2510, and determine whether these actual values ​​or predicted values ​​satisfy the target information included in the user request information. Here, the target information may be set as a predetermined target value to be satisfied, or may be set as a predetermined target range to be satisfied.

[0138] Here, the operation state calculated by the target achievement state determination unit 2540 can include, for example, a target search rate and a detection probability. The actual values ​​of the target search rate and the detection probability can be calculated based on the measurement area where measurements have been taken up to the present, and the predicted values ​​of the target search rate and the detection probability can be calculated based on the measurement range that can be measured at a future time.

[0139] The operational state calculated by the target achievement state determination unit 2540 may include, for example, the charge amount, charge rate, or charge capacity of the power storage device, or the number of unmanned watercraft that have been swapped with a standby swapping vehicle due to a lack of charge amount, charge rate, or charge capacity of the power storage device.

[0140] The operational state calculated by the goal achievement state determination unit 2540 may also include, for example, an area or time period in which wireless communication with the unmanned vessel is possible, or the communication quality in that area or time period. These wireless communication areas, time periods, and communication quality can be calculated based on the communication area in which each unmanned vessel in the mixed unmanned vessel group has been able to communicate from the past to the present, as estimated by the unmanned vessel performance estimation unit 2510, and the communication range and communication quality in which communication is possible at the present and future times.

[0141] The operational state calculated by the target achievement state determination unit 2540 can also include, for example, the data measurement time taken by the measurement sensor 1110, the amount of measurement data obtained by the measurement, or the size of the data measurement area in which the measurement is performed. These operational states can be calculated based on the measurement area in which each unmanned vessel in the mixed unmanned vessel group has taken measurements from the past to the present, as estimated by the unmanned vessel performance estimation unit 2510, and the measurement range that can be measured at the present and future times.

[0142] If the target achievement state determination unit 2540 determines that the actual operating state value or the predicted future operating state value calculated by the unmanned boat performance estimation unit 2510 is in a target unachieved state that does not satisfy the target information included in the user request information, the operation command determination unit 2550 described below adjusts the operation commands including commands regarding at least one of the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of the actual or virtual aircraft included in the mixed unmanned aircraft group based on the determination result of the target achievement state determination unit 2540, and determines the operation commands so that the operating state changes from a target unachieved state to a target achieved state.

[0143] The operation command determination unit 2550 is a functional unit that determines operation commands for the real and virtual machines based on the behavior information of the mixed unmanned aircraft group, which is a mixture of real and virtual machines, generated by the mixed aircraft group behavior generation unit 2420, so that an interference state determined by the interference prediction determination unit 2520 does not occur, or a communication disruption state determined by the communication disruption prediction determination unit 2530 does not occur, or a target unachievement state determined by the target achievement state determination unit 2540 is resolved.

[0144] (A-4-6.Operational Requirements Determination Department 2600) The operational requirements determination unit 2600 has the function of determining operational requirements or candidate operational requirements for the unmanned aircraft group so that no interference state determined by the interference prediction determination unit 2520 occurs, no communication disruption state determined by the communication disruption prediction determination unit 2530 occurs, and the estimated operational state value of the mixed unmanned aircraft group calculated by the target achievement state determination unit 2540 based on the behavior calculation information of the mixed unmanned aircraft group satisfies the target information setting goal regarding the operational state included in the pre-set user request information.

[0145] For example, if the target achievement state determination unit 2540 determines that the operational state estimate (the actual operational state value up to the present or the predicted operational state value in the future) does not satisfy the target information included in the user request information, the operational requirements determination unit 2600 changes the operational requirements for the mixed unmanned aircraft group.

[0146] Here, an example of a method for determining operational requirements by the operational requirements determination unit 2600 will be described using Figures 15 and 16. Figure 15 is a diagram showing an example of operational results from a past operation. The example shown in Figure 15 particularly shows the operational conditions (number of unmanned boats 1010 to be deployed, distribution of the unmanned boats 1010, movement speed of the unmanned boat 1010, course of movement of the unmanned boat, and continuous straight-line movement time of the unmanned boat) collected when the unmanned boat system 1000 was previously operated, and the detection probability of an object (operational state) when operated under these operational conditions.

[0147] When the operational requirements are initially set in the operational requirements determination state (status 102) or when the operational requirements are determined shortly after the actual aircraft is deployed in the operational requirements determination state, the operational requirements determination unit 2600 can initially set the operational requirements based on information about the operational state during past operations as shown in Fig. 15. That is, if the target information included in the user request information is "detection probability 92% or more," the operational conditions that result in a detection probability of 92% or more can be selected from Fig. 15. For example, the operational conditions can be set to Case 5 (number of aircraft: 100, uniform deployment distribution, movement speed: 1 knot, movement path: random, straight-line time: 10 s).

[0148] Next, FIG. 16 is a diagram showing the change trend of the detection probability for each item of the operational conditions. The example shown in FIG. 16 particularly shows the trend of how the detection probability (operational state) of an object changes when each item of the operational conditions (number of deployed unmanned crafts 1010, distribution of the unmanned crafts 1010, movement speed of the unmanned crafts 1010, movement path of the unmanned craft, and continuous straight-line movement time of the unmanned craft) is changed. In the example shown in FIG. 16, the detection probability decreases when the number of crafts is small, and increases when the number of crafts is large. The detection probability decreases when the distribution is random, and increases when the distribution is uniform. Furthermore, the detection probability decreases when the movement speed is slow, and increases when the movement speed is fast. Furthermore, the detection probability decreases when the movement path is regular, and increases when the movement path is random. Furthermore, the detection probability decreases when the straight-line movement time is long, and increases when the straight-line movement time is short.

[0149] When the operational requirements determination unit 2600 determines that the operational state estimation value (the actual operational state value up to the present or the predicted operational state value in the future) does not satisfy the target information included in the user request information, it can change at least one of the items of the current operational conditions in a direction that increases the detection probability based on the information shown in Figure 16.

[0150] 17 is a diagram showing an example of operational requirement items determined by the operational requirement determination unit 2600. As shown in Fig. 17, the operational requirement items determined by the operational requirement determination unit 2600 include the number of deployed parent or child units to be deployed at the site, and the number of replacement units to be placed on standby to replace deployed parent or child units whose charge levels in their power storage devices are below a predetermined value or whose failures or abnormalities have occurred (this may include replacement units that are placed on standby for dispatch at coasts or bases).

[0151] In addition, the operational requirements may include the movement path pattern of the deployed aircraft, movement speed, deployment area, deployment distribution pattern, straight-line travel time, order of areas to be searched (or search priority areas), or a schedule for replacing the deployed aircraft with a replacement aircraft, a recovery schedule for recovering the deployed aircraft, a schedule for recovery charging operations to charge the storage device installed in the deployed aircraft, or an avoidance behavior pattern of the unmanned vessel to avoid other ships, etc. to avoid interference conditions (including the relative avoidance route in relation to the movement direction of the other ship, movement speed, movement stopping conditions, etc.).

[0152] Here, the operational requirement items determined by the operational requirement determination unit 2600 can be determined as fixed numerical values ​​for each item, or at least one of the above operational requirement items can be specified to change over time within the operation period. Figure 18 is a diagram showing an example of operational requirements that change over time within the operation period determined by the operational requirement determination unit 2600.

[0153] The example shown in Fig. 18 shows an example in which the operation period is three days and the items of the operation requirements change over time during the operation period. In particular, the example shown in Fig. 18 shows an example in which the number of deployed and swapped aircraft is increased during the nighttime hours, the movement speed is increased, and the route pattern is changed.

[0154] (A-4-7. Information output unit 2700) The information output unit 2700 is a functional unit that transmits and outputs information acquired, generated, or determined by each functional unit of the integrated environment management system 2000 to the outside, or displays and outputs the information to the user via the operation application control system 3000 described later. The information output unit 2700 includes an operation requirement information output unit 2710, an operation command output unit 2720, an unmanned watercraft behavior information output unit 2730, and an actual measurement data output unit 2740.

[0155] The operational requirement information output unit 2710 has a function of transmitting and outputting information on the operational requirements or operational requirement candidates determined by the operational requirement determination unit 2600 to the outside or displaying and outputting it via the operational application control system 3000. In this way, by outputting information on the determined operational requirements or operational requirement candidates to the outside or to the user, each advance preparation to be carried out before the actual operation shown in Fig. 4 can be carried out in accordance with the operational requirements in the actual operation.

[0156] The action command output unit 2720 has a function of transmitting and outputting the action command determined by the action command determination unit 2550 to the real machine, or displaying and outputting information related to the action command via the operational application control system 3000. In this way, the real machine deployed in the operational area in the real space can receive the action command transmitted by the action command output unit 2720 and control its own behavior in accordance with the action command.

[0157] The unmanned vessel behavior information output unit 2730 has the function of transmitting and outputting to the outside, or displaying and outputting via the operational application control system 3000, information regarding the behavior calculation information of the mixed unmanned vessel group and the estimated operating state value of the mixed unmanned vessel group calculated by the target achievement state determination unit 2540 based on the behavior calculation information of the mixed unmanned vessel group.

[0158] In addition, the unmanned boat behavior information output unit 2730 may have the function of transmitting and outputting the judgment results determined by the interference prediction judgment unit 2520, communication outage prediction judgment unit 2530, and goal achievement state judgment unit 2540 of the group control command decision unit 2500 to the outside, or displaying and outputting them via the operational application control system 3000.

[0159] The actual measurement data output unit 2740 has the function of transmitting and outputting the actual machine behavior information acquired from the actual machine, or the actual machine measurement data measured by the measurement sensor 1110 mounted on the actual machine, or the environmental measurement data to an external device for display output, or display output via the operational application control system 3000.

[0160] (A-5. Detailed functions of the operation application control system 3000) Next, detailed functions of the operational application control system 3000 will be described with reference to Fig. 12. Fig. 12 is a functional block diagram showing the functional configuration of the operational application control system 3000. The operational application control system 3000 has an information recording unit 3100, a display information output control unit 3200, and a user interface unit 3300.

[0161] (A-5-1. Information recording unit 3100) The information recording unit 3100 is a functional unit that acquires various information from the integrated environment management system 2000, the actual aircraft, the external system 4000, etc., and records the various information. The information recording unit 3100 includes an unmanned watercraft behavior calculation information recording unit 3110, an actual aircraft behavior measurement information recording unit 3120, an operational requirement information recording unit 3130, an unmanned watercraft operation command recording unit 3140, a judgment information recording unit 3150, an actual aircraft measurement data recording unit 3160, and an environmental measurement data recording unit 3170.

[0162] The unmanned vessel behavior calculation information recording unit 3110 is a functional unit that acquires and records information regarding behavior calculation information of the mixed unmanned vessel group and estimated operating status values ​​of the mixed unmanned vessel group from the unmanned vessel behavior information output unit 2730 described above.

[0163] The actual machine behavior measurement information recording unit 3120 is a functional unit that acquires and records actual machine behavior information acquired by the actual machine from the actual measurement data output unit 2740 or directly from the actual machine.

[0164] The operational requirement information recording unit 3130 is a functional unit that acquires information on the operational requirement or the candidate operational requirement determined by the operational requirement determination unit 2600 from the operational requirement information output unit 2710 and records it.

[0165] The unmanned watercraft operation command recording unit 3140 is a functional unit that acquires the operation command determined by the operation command determination unit 2550 from the operation command output unit 2720 and records it.

[0166] The judgment information recording unit 3150 is a functional unit that acquires and records information regarding the judgment results judged by the interference prediction judgment unit 2520, communication outage prediction judgment unit 2530, and goal achievement state judgment unit 2540 of the group control command determination unit 2500 from the unmanned boat behavior information output unit 2730.

[0167] The actual machine measurement data recording unit 3160 is a functional unit that acquires actual machine measurement data measured by the measurement sensor 1110 mounted on the actual machine from the actual measurement data output unit 2740 or directly from the actual machine, and records it.

[0168] The environmental measurement data recording unit 3170 is a functional unit that acquires and records environmental measurement data acquired by the external system 4000 or the external state determination unit 1230 of the actual device from the actual measurement data output unit 2740 or directly from the actual device or directly from the external system 4000.

[0169] (A-5-2. Display information output control unit 3200) The display information output control unit 3200 is a functional unit that generates display information for displaying various pieces of information recorded in the information recording unit 3100 on a user interface unit 3300 (described later). The display information output control unit 3200 includes a map display information generation unit 3210, an operational requirement display information generation unit 3220, an actual machine measurement data display information generation unit 3230, and a judgment information display information generation unit 3240.

[0170] The map display information generation unit 3210 is a functional unit that generates display information in map format showing the behavior status of each unmanned vessel in the mixed unmanned vessel group, based on the behavior calculation information of the mixed unmanned vessel group and the estimated operating status of the mixed unmanned vessel group recorded in the unmanned vessel behavior calculation information recording unit 3110, or the actual vessel behavior information acquired by the actual vessel behavior measurement information recording unit 3120. The map display information generation unit 3210 is also a functional unit that generates display information in map format showing the position targets, route targets, etc. included in the operation commands, based on the operation commands recorded in the unmanned vessel operation command recording unit 3140.

[0171] The operational requirement display information generation unit 3220 is a functional unit that generates display information showing operational requirements or operational requirement candidates based on information about the operational requirements or operational requirement candidates recorded in the operational requirement information recording unit 3130.

[0172] The actual machine measurement data display information generating unit 3230 is a functional unit that generates display information for the actual machine measurement data recorded in the actual machine measurement data recording unit 3160. The actual machine measurement data display information generating unit 3230 may have a function of generating display information showing the environmental measurement data recorded in the environmental measurement data recording unit 3170.

[0173] The judgment information display information generation unit 3240 is a functional unit that generates display information showing information regarding the judgment results determined by the interference prediction judgment unit 2520, communication outage prediction judgment unit 2530, and target achievement state judgment unit 2540 of the group control command determination unit 2500, which are recorded in the judgment information recording unit 3150.

[0174] (A-5-3. User interface unit 3300) The user interface unit 3300 is a functional unit that displays and outputs each piece of display information generated by the display information output control unit 3200, and also receives input information from the user. The user interface unit 3300 includes a display unit 3310 and a user input receiving unit 3320.

[0175] The display unit 3310 has a function of displaying and outputting each piece of display information generated by the display information output control unit 3200. For example, the display unit 3310 displays and outputs, in map format, behavior calculation information and actual aircraft behavior information of the mixed unmanned aircraft group generated by the map display information generation unit 3210, as well as each piece of information on operation commands including position targets and route targets.

[0176] The display unit 3310 also has a function of displaying and outputting each piece of display information generated by the operational requirement display information generation unit 3220. For example, the display unit 3310 displays and outputs information indicating operational requirements and operational requirement candidates as shown in Fig. 17 and Fig. 18.

[0177] The display unit 3310 also displays and outputs the display information of the actual machine measurement data generated by the actual machine measurement data display information generation unit 3230. The display unit 3310 also displays and outputs the display information of the judgment result generated by the judgment information display information generation unit 3240.

[0178] An example of the display information output and displayed by the display unit 3310 is shown in Fig. 19. Fig. 19 is a diagram showing an example of a display screen for behavior information of the mixed unmanned vehicle group displayed on the display unit 3310. The upper part of the screen shown in Fig. 19 displays, in map format, the current behavior information of each unmanned vehicle in the mixed unmanned vehicle group, which is a mixture of real and virtual vehicles, including the position and attitude, as well as past movement routes. It is desirable that the behavior information of each unmanned vehicle be displayed in such a way that it is possible to distinguish, by color or text display, whether the unmanned vehicle is a real vehicle or a virtual vehicle.

[0179] Furthermore, detailed behavior information of the unmanned watercraft (actual watercraft 201 in the example of Figure 19) selected with a cursor or the like on the display screen is displayed in the form of a dashboard or the like at the bottom of the screen. In the example shown in Figure 19, the behavior information displayed includes hull status (stopped), mission status (none), remaining battery charge, movement speed (0.08 m / s), attitude (machine direction, roll, pitch, yaw), thruster operation status (output %, rotation speed, voltage, current), PDU voltage, PDU current, rudder, battery temperature, etc.

[0180] The user input receiving unit 3320 is a functional unit that receives user input for each piece of information displayed by the display unit 3310 or other user input. User input information can also be received via operation buttons provided on the display screen of the display unit 3310.

[0181] The user input accepting unit 3320 can accept user input information for changing the operational requirements displayed on the display unit 3310 or for selecting a specific candidate from the displayed operational requirement candidates. In this way, by accepting user input for changing the operational requirements or selecting a candidate from the user, it is possible to execute an operation simulation of a mixed unmanned vehicle group with the operational requirements specified by the user, and to confirm whether the target detection probability and target search rate, which are the target information, can be achieved with these operational requirements.

[0182] In addition, the user input accepting unit 3320 can accept setting input of target information included in user request information, or can accept input information to change already set target information, etc. while executing an operation simulation of a mixed unmanned aircraft group.

[0183] (A-6. Control flow of operational state simulation system 1) 20 to 27, the control flow of the operational state simulation system 1 will be described below. FIG. 20 is a diagram showing an example of the upper control processing flow of the operational state simulation system 1.

[0184] First, the information acquisition unit 2100 etc. executes a pre-setting process (step 101). The detailed process of this step will be explained later with reference to FIG.

[0185] Next, the virtual space generation unit 2300 and the like execute a preparatory process for executing a behavior simulation of the mixed unmanned aerial vehicle group (step 102). The detailed process of this step will be explained later with reference to FIG.

[0186] Next, real-time processing during the behavior simulation of the mixed unmanned vehicle group is performed by the unmanned vehicle behavior estimation unit 2400 etc. (step 103). The detailed processing of this step will be explained later with reference to Figs. 23 to 26.

[0187] Next, the operational requirements determination unit 2600 executes a process to determine the operational requirements (step 104). The detailed process of this step will be explained later with reference to FIG.

[0188] (A-6-1. Control process flow for pre-setting process) 21 is a diagram showing an example of a control processing flow of the presetting processing by the information acquisition unit 2100. In particular, FIG. 21 shows detailed processing of the processing of step 101 shown in FIG.

[0189] First, the user request information acquisition unit 2110 acquires user request information including desired conditions required for this operation from the user (step 201). In this step, for example, the user request information including desired condition information and goal information as shown in FIG.

[0190] Next, the assumed environment information acquisition unit 2120 acquires assumed environment information (step 202). In this step, environmental information related to environmental disturbances expected during the operation time of the operation area included in the desired condition information of the user desired information is acquired.

[0191] Next, the assumed condition parameter acquisition unit 2130 acquires the assumed condition parameter setting unit (step 203). In this step, various condition parameters assumed when operating the unmanned boat system 1000 in an operation area such as that shown in Fig. 14 are acquired.

[0192] (A-6-2. Control process flow for preparation before simulation execution) 22 is a diagram showing an example of a control processing flow of preparations before simulation execution by the virtual space generation unit 2300 etc. In particular, FIG. 22 shows detailed processing of the processing of step 102 shown in FIG.

[0193] First, the virtual space generation unit 2300 generates a virtual operation area that simulates the operation area in the virtual space (step 301). In this step, the virtual operation area is generated based on, for example, the operation area included in the request information acquired by the user request information acquisition unit 2110 and further the assumed environment information acquired by the assumed environment information acquisition unit 2120.

[0194] Next, the actual unmanned watercraft 1010 is deployed in the operating area in the real space, and the actual watercraft behavior information acquisition unit 2210 acquires actual watercraft behavior information from the actual watercraft (step 302).

[0195] Next, based on the acquired information on the behavior of the actual craft, the virtual space generation unit 2300 generates an unmanned craft synchronized with the behavior of the actual craft in the virtual operation area (step 303).

[0196] Next, the virtual space generation unit 2300 generates a virtual machine within the virtual operation area (step 304).

[0197] By performing the above-described processing, a virtual operation area can be generated in a virtual space, and an unmanned boat and a virtual aircraft corresponding to the actual aircraft can be generated within the virtual operation area.

[0198] (A-6-3. Control process flow for real-time processing during simulation) 23 is a diagram showing an example of a control processing flow of real-time processing during simulation execution by the unmanned watercraft behavior estimation unit 2400 etc. In particular, Fig. 23 shows detailed processing of the processing of step 103 shown in Fig. 20.

[0199] First, the environmental measurement data acquisition unit 2230 acquires environmental measurement data relating to environmental information in the area where the unmanned boat system 1000 is deployed and its surrounding area from the external system 4000 or the external state determination unit 1230 of the actual aircraft (step 401).

[0200] Next, the actual aircraft behavior information acquisition unit 2210 acquires actual aircraft behavior information determined by the actual aircraft navigation state determination unit 1210 from the actual aircraft deployed in the operational area in the real space (step 402).

[0201] Next, the virtual space generation unit 2300 updates the environmental state of the virtual operation area based on the environmental measurement data acquired in step 401 (step 403).

[0202] Next, the virtual machine behavior calculation unit 2410 calculates the behavior of the virtual machine in the virtual operation area based on the virtual machine model information (step 404). In this step, the behavior of the virtual machine is calculated taking into account the environmental state of the updated virtual operation area. Also, in this step, it is possible to modify the parameters of the virtual machine model based on the comparison result between the actual machine behavior calculation information and the actual machine behavior information by the actual machine behavior calculation unit 2430, and calculate the virtual machine behavior calculation information using the modified virtual machine model.

[0203] Next, the mixed aircraft group behavior generation unit 2420 calculates behavior information of the mixed aircraft group, which is a mixture of real aircraft and virtual aircraft, in the virtual operation area based on the real aircraft behavior information and the virtual aircraft behavior calculation information (step 405).

[0204] Next, the operational application control system 3000 displays various information including the behavior of the mixed aircraft group and accepts user intervention information from the user (step 406). In this step, for example, input information to change already set target information while the operation simulation of the mixed unmanned aircraft group is being executed can be accepted. The detailed processing of this step will be explained later in FIG. 24.

[0205] Here, if it is determined in step 410 described below that the demonstration is incomplete and the processing of this step is repeated, in addition to the behavior of the mixed aircraft group, information on the operational requirements or operational requirement candidates determined by the operational requirement determination unit 2600, the operation commands determined by the operation command determination unit 2550, the operation state estimate of the mixed unmanned aircraft group calculated by the target achievement state determination unit 2540, the determination results determined by the interference prediction determination unit 2520, communication disruption prediction determination unit 2530, and target achievement state determination unit 2540 of the group control command determination unit 2500, actual aircraft behavior information acquired from the actual aircraft, actual aircraft measurement data or environmental measurement data measured by the measurement sensor 1110 mounted on the actual aircraft, etc. are displayed and output. Furthermore, if the processing of this step is repeated, user intervention information for changing the displayed operational requirements or selecting a specific candidate from the operational requirement candidates can be received.

[0206] Next, the target achievement state determination unit 2540 determines whether the actual operational state values ​​from the past to the present or the predicted operational state values ​​in the future calculated by the unmanned boat performance estimation unit 2510 satisfy the target information included in the user request information, based on the behavior calculation information of the mixed unmanned aircraft group calculated by the mixed aircraft group behavior estimation unit 2420 (step 407).

[0207] Next, the operational requirements determination unit 2600 determines or updates the operational requirements for the mixed unmanned aerial vehicle group (step 408). Furthermore, the next processing step to transition to is determined depending on whether the operational requirements for the mixed unmanned aerial vehicle group have been changed by updating them in this step. If there have been changes, the process transitions to step 409, and if there have been no changes, the process transitions to step 410. Note that detailed processing of this step will be explained later in FIG. 25.

[0208] Next, if it is determined in step 408 that the operational requirements have been changed, the operational requirements determination unit 2600 determines the action of the real machine in the real space (step 409). The detailed processing of this step will be explained later with reference to FIG.

[0209] Next, the operational requirements determination unit 2600 determines whether the pre-demonstration completion requirements (demonstration simulation implementation period, search implementation area, etc.) specified by the user for determining the operational requirements are met (step 410). Also, depending on whether the pre-demonstration completion requirements are met in this step, the next processing step to transition to is determined. If the demonstration is not complete, the process transitions to step 401 and the processing of this flowchart is repeated, and if the demonstration is complete, the processing of this flowchart is terminated.

[0210] (A-6-3-1. Control process flow for displaying information and accepting user input) 24 is a diagram showing an example of a control processing flow for displaying information and receiving user input by the operational application control system 3000. In particular, Fig. 24 shows detailed processing of step 406 shown in Fig. 23.

[0211] First, the display information output control unit 3200 generates display information such as behavior information of the mixed unmanned aerial vehicles in the virtual operation area in the virtual space (step 501).

[0212] Next, the user interface unit 3300 displays and outputs display information such as behavior information of the mixed unmanned aerial vehicle group (step 502). Note that in this step, if it is determined in the above-mentioned step 410 that the verification is incomplete and the processing of this step is repeated, in addition to the behavior information of the mixed unmanned aerial vehicle group, information on the operational requirements or candidate operational requirements determined by the operational requirement determination unit 2600, the operation command determined by the operation command determination unit 2550, the operation state estimate of the mixed unmanned aerial vehicle group calculated by the target achievement state determination unit 2540, the determination results determined by the interference prediction determination unit 2520, communication outage prediction determination unit 2530, and target achievement state determination unit 2540 of the group control command determination unit 2500, actual aircraft behavior information acquired from the actual aircraft, actual aircraft measurement data or environmental measurement data measured by the measurement sensor 1110 mounted on the actual aircraft, etc. are displayed and output.

[0213] Next, during the operation simulation of the mixed unmanned aerial vehicle group, the next processing step to transition to is determined depending on whether or not there is an intervention request input from the user via the user interface unit 3300, etc. (step 503). In this step, if there is an intervention request input from the user, the processing transitions to step 504, and on the other hand, if there is no intervention request input, the processing of this flowchart ends.

[0214] Next, if an intervention request is input in step 503, an input for changing the user request information previously acquired in step 201 is accepted (step 504). In this step, for example, input for changing the search object, operation area, operation time, and target information (target search rate, target detection probability, etc.) shown in Fig. 14 can be accepted.

[0215] Next, input for changing the assumed condition parameter settings previously acquired in step 203 is accepted (step 504). In this step, for example, input for changing the aircraft assumed conditions, the target assumed conditions, the environmental impact conditions, and other hypothetical conditions shown in Fig. 15 can be accepted. Furthermore, if it is determined in the above-mentioned step 410 that the verification is incomplete and the processing of this step is repeated, in this step, user intervention information for changing the operational requirements once determined or for selecting specific candidates from the operational requirement candidates can be accepted.

[0216] (A-6-3-2. Control process flow for updating operational requirements) 25 is a diagram showing an example of a control processing flow for updating operational requirements, etc., by the operational requirement determination unit 2600, etc. In particular, Fig. 25 shows detailed processing of the processing of step 408 shown in Fig. 23.

[0217] First, the target achievement state determination unit 2540 determines the next processing step to transition to depending on whether the actual value of the detection probability is within a predetermined range of a preset target detection probability (step 601). In this step, if it is determined that the detection probability is within the predetermined range, the processing transitions to step 602, and on the other hand, if it is determined that the detection probability is outside the predetermined range, the processing transitions to step 603.

[0218] Next, if it is determined in step 601 that the detection probability is within a predetermined range, it is determined that there is no change in the operational requirements (step 602), and the processing of this flowchart ends.

[0219] Next, if it is determined in step 601 that the detection probability is outside the predetermined range, the next processing step to transition to is determined depending on whether the detection probability is lower than the lower limit of the predetermined range (step 603). If it is determined in this step that the detection probability is lower than the lower limit of the predetermined range, the processing transitions to step 604, and if it is determined that the detection probability is lower than the lower limit of the predetermined range, the processing transitions to step 606.

[0220] Next, if it is determined in step 603 that the detection probability is lower than the lower limit of the predetermined range, the search performance of the unmanned watercraft system 1000 is determined to be insufficient (step 604), and the operational requirements for the mixed real and virtual unmanned watercraft group are changed to improve the search performance (step 605). In the process of changing the operational requirements in step 605, for example, based on the information shown in Fig. 16, at least one of the various parameters of the operational requirements (number of watercraft, deployment distribution pattern, movement speed, movement path pattern, straight-line travel time, etc.) is changed in a direction that improves the detection probability.

[0221] Next, if it is determined in step 603 that the detection probability is higher than the upper limit of the predetermined range, the search performance of the unmanned watercraft system 1000 is determined to be excessive (step 606), and the operational requirements for the mixed real and virtual unmanned watercraft group are changed so that the search performance decreases (step 607). In the process of changing the operational requirements in step 607, at least one of the various parameters of the operational requirements (particularly the number of aircraft, travel speed, etc.) is changed in a direction that decreases the detection probability, for example, based on the information shown in Figure 16. Here, since relaxing the operational requirements is required to reduce the equipment and human resources required for this operation, among the various parameters, reducing the number of aircraft, which is effective in reducing resources, or reducing the travel speed, which is effective in reducing power consumption, is selected as a measure to change the operational requirements.

[0222] (A-6-3-3. Control process flow of actual device action determination process) 26 is a diagram showing an example of a control processing flow of the actual machine action determination processing by the operational requirement determination unit 2600. In particular, Fig. 26 shows detailed processing of the processing of step 409 shown in Fig. 23 .

[0223] First, the operational requirement determination unit 2600 grasps the operational requirements after the change (step 701).

[0224] Next, the next processing step to transition to is determined according to the number of aircraft in the operational requirements after the change determined by the operational requirements determination unit 2600 (step 702). In this step, if the number of aircraft increases, the processing transitions to step 703, if the number of aircraft decreases, the processing transitions to step 708, and if there is no change in the number of aircraft, the processing of this flowchart ends.

[0225] Next, if the number of aircraft is increased in step 702, it is determined whether additional implementation is necessary (step 703). In this step, the accuracy of the behavior simulation of the mixed unmanned aircraft group is determined based on the results of comparing the actual aircraft behavior calculation information and the actual aircraft behavior information, and the operational requirements determination unit 2600 may automatically determine the accuracy based on this accuracy, or the user may input the determined result. Here, if the number of actual aircraft deployed in the real space is increased, the amount of actual measurement data, such as actual aircraft behavior information and environmental measurement data measured by the actual aircraft, increases, thereby improving the accuracy of the behavior simulation of the mixed unmanned aircraft group. Therefore, if the current accuracy of the behavior simulation of the mixed unmanned aircraft group is low, it is desirable to increase the number of actual aircraft.

[0226] Next, the next processing step to transition to is determined depending on whether or not a physical device is to be added (step 704). In this step, if a physical device is to be added, the processing transitions to step 705, whereas if a physical device is not to be added, the processing transitions to step 707.

[0227] Next, if an actual machine is to be added in step 704, an additional deployment command for the actual machine is generated (step 705). In this step, the deployment position, layout distribution pattern, movement path pattern, movement speed, and straight-line travel time of the actual machine to be added are also commanded.

[0228] Next, a real machine synchronized with the behavior of the newly deployed real machine is added to the virtual space (step 706). After this step, the processing of this flowchart ends.

[0229] Next, if a real machine is not added in step 704, a virtual machine is added to the virtual space (step 707). After this step, the processing of this flowchart ends.

[0230] Next, if the number of aircraft is to be reduced in step 702, it is determined whether or not reduction is necessary (step 708).

[0231] Next, the next processing step to transition to is determined depending on whether or not the physical machines are to be reduced (step 709). In this step, if the physical machines are to be reduced, the processing transitions to step 710, whereas if the physical machines are not to be reduced, the processing transitions to step 712.

[0232] Next, if the actual machines are to be reduced in step 709, a command to withdraw the actual machines is generated (step 710).

[0233] Next, the real machine corresponding to the collected real machine is deleted from the virtual space (step 711). After this step, the processing of this flowchart ends.

[0234] Next, if the number of real machines is not reduced in step 709, the virtual machine is deleted from the virtual space (step 712). After this step, the processing of this flowchart ends.

[0235] (A-6-4. Control process flow for operational requirements determination process) 27 is a diagram showing an example of a control processing flow of the operational requirement determination processing by the operational requirement determination unit 2600. In particular, FIG. 26 shows detailed processing of the processing of step 104 shown in FIG.

[0236] First, the type of operational requirement is determined (step 801). In this step, it is determined whether the operational requirement is of an operational requirement type in which each item of the operational requirement is specified by fixed information as shown in Fig. 17, or an operational requirement type in which at least one of the items of the operational requirement is specified as information that changes over time during the operation period as shown in Fig. 18. The operational requirement type may be determined based on preset information or on information input by the user.

[0237] Next, the next processing step to transition to is determined according to the determined operational requirement type (step 802). In this step, if the operational requirement type is a fixed type, the processing transitions to step 803, whereas if the operational requirement type is a time-series change type, the processing transitions to step 804.

[0238] Next, if the operational requirement type is a fixed type, the operational requirement or its candidate that can satisfy the target condition included in the user request information over the entire operation period is determined (step 803).

[0239] Furthermore, if the operational requirement type is a time-series change type, the operational requirement or its candidate that can satisfy the target condition included in the user request information is determined for each time period of the operational period by the operational state estimated value (step 804).

[0240] Next, the information output unit 2700 or the operational application control system 3000 transmits or displays the operational requirement or its candidate information (step 805).

[0241] (A-7. Function to determine future operational requirements during current operation) The function of determining future operational requirements while the unmanned watercraft system 1000 is currently in operation will be described below with reference to FIGS.

[0242] (A-7-1. Status of determining future operational requirements during current operation) Fig. 28 is a diagram showing an example of a higher-level control processing flow of the operational state simulation system 1. Fig. 4 shows four higher-level transition states when operating the unmanned watercraft system 1000 for the purpose of searching for the target object 7000: a pre-operation preparation state before operation, an on-site preparation state immediately before operation, an actual operational operation state during operation, and a search end / cancellation state immediately before the end of operation. However, Fig. 28 shows an example in which, in addition to these higher-level transition states, there is also a future operational change state during actual operation.

[0243] 28 shows a state in which future operational requirements at a future time (e.g., several hours later) are determined during actual operation in which a search for the target object 7000 is being performed, and resources are arranged to be sent to the site if additional resources are required. Therefore, the future operational change state during actual operation includes the following states: a performance evaluation of the unmanned watercraft system 1000 is performed during actual operation in which the actual vehicle is deployed, and future operational requirements at a specified future time (number of unmanned watercraft 1010, movement route pattern, etc.) are determined (status 501); a state in which resources such as unmanned watercraft 1010 to be used in the operation are procured according to the determined operational requirements (status 502); a state in which the resources are transported to the site (status 503); and a state in which resources are installed on the coast near the marine area of ​​the actual operation (status 504).

[0244] (A-7-2. Control flow for determining future operational requirements during current operation) Next, a control flow for determining future operational requirements at future times during actual operation will be described in the state of status 501 shown in Fig. 28. Fig. 29 is a diagram showing the determination control flow of the operational state simulation system 1 when determining future operational requirements during actual operation.

[0245] First, the real environment information acquisition unit 2200 and the unmanned vessel behavior estimation unit 2400 perform processing similar to steps 401 to 405 shown in Figure 23 to acquire information on the real environment and calculate current behavior information of the mixed unmanned vessel group (step 901).

[0246] Next, the environmental measurement data acquisition unit 2230 acquires environmental forecast information for a future time after a predetermined time has elapsed for which operational requirements are to be determined, or environmental forecast information from the present to a future time (step 902).

[0247] Next, the virtual space generation unit 2300 generates a predicted virtual environment state in the virtual space after a predetermined time has elapsed or from the present to a future time based on the environment forecast information (step 903).

[0248] Next, the virtual aircraft behavior calculation unit 2410 calculates, through simulation, virtual aircraft behavior calculation information regarding the aircraft behavior of the virtual aircraft in the virtual operation area after a predetermined time has elapsed or from the present to a future time, based on the virtual environment predicted state and the virtual aircraft model information, and the mixed aircraft group behavior generation unit 2420 calculates, through simulation, behavior information of the mixed unmanned aircraft group after a predetermined time has elapsed, based on the actual aircraft behavior information and the virtual aircraft behavior calculation information (step 904).

[0249] Next, the target achievement state determination unit 2540 determines whether the predicted operational state after a predetermined time has elapsed, calculated by the unmanned boat performance estimation unit 2510, satisfies the target information included in the user request information, based on the behavior information of the mixed unmanned aircraft group after a predetermined time has elapsed, calculated by the mixed aircraft group behavior generation unit 2420 (step 905).

[0250] Next, based on the determination result of the target achievement state determination unit 2540, the operational requirements determination unit 2600 determines or changes the operational requirements or candidate operational requirements for the mixed unmanned aerial vehicle group at a future time so that the predicted value of the operational state after a predetermined time has elapsed satisfies the target information (step 906). Note that the processing in this step can be the same as that in step 408. Furthermore, the determination of the operational requirements for the mixed unmanned aerial vehicle group at a future time in this step determines the next processing step to transition to depending on whether the operational requirements have been changed. If there have been changes, the processing transitions to step 907, and if there have not been changes, the processing transitions to step 908.

[0251] Next, if it is determined in step 906 that there is a change in the operational requirements, the operational requirements determination unit 2600 determines the action of the real machine in the real space at a future time (step 907). Note that the processing in this step can be the same as the processing in step 409.

[0252] Next, the operational requirement determination unit 2600 determines whether the simulation completion requirements (simulation period, search area, etc.) for determining the operational requirements at a future time specified by the user are satisfied (step 908). Also, depending on whether the simulation completion requirements are satisfied in this step, the next processing step to transition to is determined. If the simulation completion requirements are not satisfied, the process transitions to step 904 and the processing of this flowchart is repeated; if the simulation completion requirements are satisfied, the process transitions to step 909.

[0253] Next, in step 908, if the simulation completion requirements are met, the operational requirements determination unit 2600 determines the operational requirements at a future time, and the operational application control system 3000 displays and outputs the determined operational requirements (step 909).

[0254] Next, based on the changed operational requirements, command information for additional deployment or recovery of the actual aircraft is sent to the user terminal device of the on-site aircraft operator, aircraft deployment person, or aircraft recovery person (step 910).

[0255] (A-8. Implementation example of the Integrated Environment Management System 2000, etc.) Next, with reference to FIGS. 30 and 31, an implementation example in which the functional units of the converged environment management system 2000 and operational application control system 3000 described so far are implemented on a cloud server, a workstation, or the like will be described.

[0256] (A-8-1. Implementation example of the Integrated Environment Management System 2000) FIG. 30 is a diagram showing an example of an implementation in which the functional units of the converged environment management system 2000 and the operational application control system 3000 are implemented on a cloud server, a workstation, or the like.

[0257] The hardware shown in Figure 30 mainly includes a control station, a cloud server, a workstation, a HILS, communication equipment, and actual equipment. Here, the example shown in Figure 30 shows an example in which each functional unit of the integrated environment management system 2000 and the operational application control system 3000 is implemented in the control station, cloud server, workstation, and HILS. The communication equipment and actual equipment in Figure 30 correspond to the communication satellite 5100, the terrestrial base station 5200, and the unmanned boat system 1000, respectively, shown in Figure 1 etc.

[0258] The control station in FIG. 30 includes a UI device that outputs display information to users of the integrated environment management system 2000 and the operational application control system 3000 and accepts user input information, and a web browser application running on the UI device. The web browser has a function for selecting information to be displayed from various display information on a website, and the user can select information to be displayed on the display screen of the UI device via the UI device. The UI device can also accept user request information (including search objects, search areas, search time, and target information) as shown in FIG. 13 and assumed condition parameters as shown in FIG. 14 from the user and transmit them to the website. Here, the UI device corresponds to the user interface unit 3300 shown in FIG. 12, etc., and the web browser has some of the functions of the display information output control unit 3200 shown in FIG. 12 (particularly, a display control function for selecting information to be displayed from various display information on a website).

[0259] The IoT Gateway implemented in the cloud server in Fig. 30 sends control commands to real machines deployed in the operation area via communication facilities and receives telemetry data (i.e., real machine behavior information, etc.) from the real machines. Furthermore, like the real machines, it sends control commands to a workstation implemented with a virtual machine simulator that calculates the behavior of a virtual machine (described later) through simulation, and receives telemetry data (i.e., virtual machine behavior calculation information, etc.) from the real machines. Here, the IoT Gateway corresponds to the functions of part of the real environment information acquisition unit 2200 shown in Fig. 11 (such as the real machine behavior information acquisition unit 2210).

[0260] The telemetry data (real machine behavior information, virtual machine behavior calculation information, etc.) acquired by the IoT Gateway is analyzed by data analysis software to generate behavior information of a mixed machine group that includes a mixture of real machines and virtual machines, and the analysis results (behavior information of the mixed machine group, etc.) are recorded in a data lake or DB. Here, the data analysis software corresponds to the function of the mixed machine group behavior generation unit 2420. Furthermore, the data lake and DB correspond to the function of the information recording unit 3100.

[0261] The data analysis software also performs data analysis and statistical processing on the behavior information of the mixed vehicle group stored in the data lake to generate visualization data to be displayed to users. The data analysis software generates display information that overlays the behavior information of the mixed vehicle group on a map of the operation area, and display information that displays the behavior status and internal status of each unmanned vehicle, including real and virtual vehicles, in dashboard format, and uploads the generated visualization data to a website. Here, the data analysis software corresponds to the functions of the display information output control unit 3200, particularly the map display information generation unit 3210.

[0262] The website is also a functional unit that provides various data, including visualization data generated by the data analysis software, to the UI device. The website can record user request information and assumed condition parameters acquired from the UI device in a DB using a data storage API. The website is also a functional unit that reads various information, such as behavior information, operational status, operational requirements, and previously recorded user request information and assumed condition parameters of the mixed aircraft group recorded in the DB, from the DB using a data acquisition API, generates display information based on the read information, and provides the display information to the UI device. Here, the website corresponds to the functions of the operational requirement display information generation unit 3220, the actual aircraft measurement data display information generation unit 3230, and the judgment information display information generation unit 3240 of the display information output control unit 3200.

[0263] The group control software also has the function of using a data acquisition API to read various information from the DB, such as behavior information, operational status, operational requirements, previously recorded user request information, and assumed condition parameters of the mixed vehicle group, and, based on this information, estimates various performance characteristics of the unmanned vehicle, determines future interference between unmanned vehicle groups, determines whether communication disruption will occur, determines whether the goal has been achieved, determines operational commands for the unmanned vehicle, and determines operational requirements. Based on the user request information and assumed condition parameters read from the DB, the group control software can obtain information on related group control scenarios and patterns from the container registry and determine operational commands for the unmanned vehicle. Here, the group control software corresponds to the functions of the group control command determination unit 2500 and the operational requirement determination unit 2600.

[0264] The command transmission API also transmits operation commands generated by the group control software to the actual devices and virtual device simulators via the IoT Gateway. Here, the command transmission API corresponds to the function of the operation command output unit 2720.

[0265] The virtual machine simulator is simulator software installed on a workstation, and simulates and calculates behavior calculation information, simulated measurement data, internal states, etc. of a virtual machine in a virtual space based on an operation command received via the IoT Gateway. Here, the virtual machine simulator can calculate behavior calculation information, simulated measurement data, internal states, etc. of a virtual machine using a virtual machine model in which all functional units constituting the virtual machine are simulated in a virtual space, but can also simulate and calculate behavior calculation information, simulated measurement data, internal states, etc. of a virtual machine using a HILS machine in which some functional units constituting the virtual machine are configured with actual hardware and the other functional units are configured with virtual models.

[0266] (A-8-2. HILS implementation example) Next, Fig. 31 is a diagram showing an example of an implementation of HILS connected to a virtual machine simulator, which is partly configured with hardware devices and partly configured with virtual models.

[0267] In the HILS implementation example shown in Figure 31, the onboard processing unit, hull behavior control device, and power control device shown on the left side of the figure are composed of hardware devices connected to each other via wired or wireless communication. Also, the communication device, sonar, camera, LiDAR, sensor simulation, hull behavior simulation, solar cell model, and battery model shown on the right side of the figure are composed of virtual models.

[0268] The power control device sends a current command for the solar cell and a current limit command for the battery to the HILS device, and the HILS device sends solar cell output voltage information calculated based on a solar cell model and battery SoC information calculated based on a battery model to the power control device.

[0269] The hull behavior device sends rudder angle commands for the attitude control mechanism 1320 and motor rotation speed commands for the thrust generation unit 1310 (propellers, etc.) to the HILS device. The hull behavior simulation unit in the HILS device calculates through simulation the hull behavior of the virtual aircraft when the rudder angle command and motor rotation speed command are input, based on the virtual aircraft model, solar cell model, and battery model. Behavior calculation information related to hull behavior, including the calculated heading, position, angular velocity, and acceleration, is sent from the HILS device to the hull behavior device.

[0270] Furthermore, the sensor simulation unit in the HILS system generates virtual measurement data measured by each sensor based on sensor model information that simulates each sensor (communication device, sonar, camera, and LiDAR) and behavior calculation information calculated by the hull behavior simulation unit. This virtual measurement data is transmitted from the HILS system to the onboard processing device. Here, as a modified example, instead of using the sensor model information, the sensor simulation unit may use the actual sensor to generate simulated information (e.g., optical image information captured by a camera) of the information measured by the sensor.

[0271] The on-board processing device, which is a hardware device, receives simulated measurement data measured by each sensor from the HILS device. The on-board processing device also acquires behavior calculation information related to hull behavior and internal state information including solar cell output voltage information and battery SoC information from the hull behavior device and power control device. The on-board processing device transmits the generated behavior calculation information, simulated measurement data, internal state information, etc. of the virtual machine to the virtual machine simulator.

[0272] (A-9. Example of hardware configuration) 32 is a diagram showing an example of the hardware configuration of the integrated environment management system 2000 and the operational application control system 3000. Here, the integrated environment management system 2000 and the operational application control system 3000 according to the present invention are information processing devices such as a server device or a PC. As shown in the figure, the integrated environment management system 2000 includes an input device 100, an output device 200, a processing device 300, a main memory device 400, an auxiliary memory device 500, a communication device 600, and a bus 700 that electrically connects these devices.

[0273] The input device 100 can constitute the user input receiving unit 3320 of the user interface unit 3300, and is a device that allows a user to input information and instructions to the operational application control system 3000. Specifically, the input device 100 is, for example, a touch panel, a keyboard, a mouse, or an audio input device such as a microphone.

[0274] The output device 200 is a device that outputs various information generated by the operational application control system 3000, and can constitute the display unit 3310 of the user interface unit 3300. Specifically, the output device 200 can constitute the display unit 3310 using a display device for eyewear, AR, or VR, or it may also be a printer or a speaker.

[0275] The processing device 300 is, for example, a device that performs arithmetic processing. Specifically, the processing device 300 is, for example, a CPU, a microprocessor, a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or other semiconductor devices capable of performing arithmetic processing.

[0276] The main storage device 400 is a memory device such as a RAM that temporarily stores various types of read information and a ROM that stores programs, application programs, and other various information executed by the processing device 300. The auxiliary storage device 500 is a non-volatile storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory that can store digital information.

[0277] The communication device 600 is a device that performs wireless or wired information communication with the outside.

[0278] The above-described embodiments are merely examples for facilitating understanding of the present invention, and are not intended to limit the present invention. The present invention can be modified and improved without departing from the spirit thereof, and it goes without saying that the present invention includes equivalents thereof.

[0279] [B. Effects of this embodiment] The above-described embodiment makes it possible to perform a more reliable performance evaluation of an unmanned mobile system or to confirm operational requirements that can achieve the required performance, without requiring excessive physical and human resources or minimizing the impact on economic activity in the vicinity of the site. As an example, by calculating in a simulation the behavior of a mixed group of unmanned mobile systems, which includes a mixture of real and virtual systems deployed in an operation area, and confirming whether or not the specified performance can be achieved, or the operational requirements that can achieve the specified performance, it is possible to verify the performance of the system with the desired accuracy while limiting the number of real systems deployed at the site. [Explanation of symbols]

[0280] 1...Operational state simulation system (system) 100...input device 200...output device 300...Processing device 400...Main storage device 500...Auxiliary storage device 600...Communication device 700...bus 1000...Unmanned boat system 1001...Base unit 1002...Sub unit 1010...Unmanned boat 10021... Primary connection slave unit 10022... Secondary connection slave unit 1100...Measuring unit 1110...Measuring sensor 1120...Measurement control unit 1200...Own aircraft state determination unit 1210...Navigation state determination unit 1220: Internal state determination unit 1230: External state determination unit 1300...Navigation section 1310...Thrust generation section 1320: Attitude control mechanism 1330: Navigation control unit 1400...Communication unit 1410...Unmanned vehicle communication unit 1420…General Control and Communications Department 1500…Judgment section 1600...Recording section 1610...Measurement data recording section 1620...Own aircraft status recording section 1630...Determination information recording section 1700...power supply unit 1710...electricity storage device 1720...power generating device 1730...power control unit 2000: Integrated Environmental Management System 2100: Information acquisition unit 2110: User request information acquisition unit 2120: Assumed environment information acquisition unit 2130: Assumed condition parameter acquisition unit 2140...User intervention information acquisition unit 2200... Actual environment information acquisition unit 2210... Actual machine behavior information acquisition unit 2220...Actual measurement data acquisition unit 2230...Environment measurement data acquisition unit 2240…Environmental Prediction Data Acquisition Department 2300...Virtual space generation unit 2400...Unmanned vessel behavior estimation unit 2410...Virtual aircraft behavior calculation unit 2420... Mixed aircraft group behavior generation unit 2430... Actual aircraft behavior calculation unit 2500...Group control command determination unit 2510...Unmanned boat performance estimation unit 2520: Interference prediction determination unit 2530: Communication interruption prediction determination unit 2540...Goal achievement state determination unit 2550...Operation command determination unit 2600…Operation requirements determination department 2700...information output unit 2710...operational requirement information output unit 2720...Operation command output unit 2730...Unmanned boat behavior information output unit 2740…Measurement data output section 3000...Operational Application Control System 3100... Information recording unit 3110... Unmanned boat behavior calculation information recording unit 3120...Actual aircraft behavior measurement information recording unit 3130...Operational requirements information recording unit 3140: Unmanned boat operation command recording unit 3150: Judgment information recording unit 3160: Actual measurement data recording unit 3170: Environmental measurement data recording unit 3200: Display information output control unit 3210: Map display information generation unit 3220... Operational requirement display information generation unit 3230... Actual machine measurement data display information generation unit 3240…Judgment information display information generation unit 3300...User interface section 3310: Display unit 3320: User input reception unit 4000…External system 5100: Communications satellite 5200: Ground base station 6000...Cooperative system 7000...Object 8000...User terminal device

Claims

1. An operational state simulation system that simulates an operational state in an operational area of ​​a group of unmanned aerial vehicles including a plurality of unmanned aerial vehicles, a virtual space generation unit that generates a virtual operation area in a virtual space; an actual aircraft behavior information acquisition unit that acquires actual aircraft behavior information regarding aircraft behavior from the actual unmanned aircraft deployed in the operation area in real space; a virtual aircraft behavior calculation unit that calculates virtual aircraft behavior calculation information regarding aircraft behavior of the virtual aircraft of the unmanned aircraft deployed in the virtual operation area generated in the virtual space based on virtual aircraft model information; a mixed unmanned aircraft group behavior generation unit that generates behavior information of a mixed unmanned aircraft group in which the real aircraft and the virtual aircraft are mixed based on the real aircraft behavior information and the virtual aircraft behavior calculation information; An operational state simulation system comprising an operational requirement determination unit that determines operational requirements or candidate operational requirements for the unmanned aircraft group so that the operational state estimate value of the mixed unmanned aircraft group calculated based on the behavior information of the mixed unmanned aircraft group satisfies a pre-set target for the operational state.

2. 2. The operational state simulation system according to claim 1, The operational requirements of the unmanned aerial vehicle group determined by the operational requirements determination unit include: The number of deployed aircraft, which are the unmanned aircraft to be deployed in real space; Or, the number of replacement machines to be placed on standby to replace the deployment machine whose charge amount in the power storage device is below a predetermined value or whose installed functional unit has failed or become abnormal, Or, the movement path pattern, movement speed, deployment area, arrangement distribution pattern, or straight movement time of the deployment machine, Or, a schedule for replacing the developing machine with the replacement machine, Or, a recovery schedule for recovering the deployer; Or, a schedule of a recovery charging operation for charging the power storage device mounted on the deploying machine, or the drone's evasive maneuver patterns when avoiding other vessels; An operational state simulation system including:

3. 3. The operational state simulation system according to claim 2, The operational requirements of the unmanned aerial vehicle group determined by the operational requirements determination unit are: An operational state simulation system in which at least one of the number of the deployment aircraft, the number of the replacement aircraft, the movement path pattern, the movement speed, the deployment area, the distribution pattern, or the straight-line travel time of the unmanned aircraft is specified to change over time during the operational period of the unmanned aircraft group.

4. 2. The operational state simulation system according to claim 1, When the mission of the unmanned aerial vehicle group includes a search mission to search for an object using a measurement sensor mounted on the unmanned aerial vehicle, An operational state simulation system, wherein the set target includes a target value for a search rate indicating the proportion of the cumulative measurement area measured by the measurement sensor during an operation period, or a target value for a detection probability indicating the probability of the measurement sensor detecting the object.

5. 2. The operational state simulation system according to claim 1, When the unmanned aerial vehicle operates using a power storage device mounted on the vehicle as a power source, An operational state simulation system, wherein the set target includes a target value for the average or minimum value of the charge amount, charge rate, or charge capacity of the storage device of the multiple unmanned aircraft, or a target value for the number of unmanned aircraft that have been replaced due to a lack of charge amount, charge rate, or charge capacity of the storage device.

6. 2. The operational state simulation system according to claim 1, An operational state simulation system, wherein the set goals include a target value for operating time, which is the time during which the group of unmanned aircraft operates in the operational area.

7. 2. The operational state simulation system according to claim 1, When the mission of the unmanned aerial vehicle group includes a wireless communication mission of connecting wireless communication with a communication unit mounted on the unmanned aerial vehicle to equipment present in the operation area or the surrounding area of ​​the operation area, An operational state simulation system, wherein the set goals include target values ​​for the communication quality of wireless communication between the equipment and the unmanned aircraft, or the area in which wireless communication with the unmanned aircraft is possible, or the time period in which wireless communication with the unmanned aircraft is possible.

8. 2. The operational state simulation system according to claim 1, When the mission of the unmanned aerial vehicle group includes a measurement data acquisition mission in which measurement sensors mounted on the unmanned aerial vehicles measure the operation area or the area surrounding the operation area, The set target includes a target value for the data measurement time for measuring by the measurement sensor, the amount of measurement data obtained by measurement, or the size of the data measurement area for measuring.

9. 2. The operational state simulation system according to claim 1, a real environment information acquisition unit that acquires environmental measurement data relating to environmental information including at least one of ocean currents, wave heights, wind speeds, wind directions, solar radiation, brightness, visibility, air temperatures, water temperatures, and weather in the operation area of ​​the real space; The virtual aircraft behavior calculation unit calculates the virtual aircraft behavior calculation information related to the aircraft behavior of the virtual aircraft based on the environmental measurement data and the virtual aircraft model information.

10. 2. The operational state simulation system according to claim 1, a real environment information acquisition unit that acquires environmental measurement data relating to environmental information including at least one of ocean currents, wave heights, wind speeds, wind directions, solar radiation, brightness, visibility, and weather in the operation area of ​​the real space; the virtual space generation unit generates or updates a virtual environment state of the virtual operation area based on the environmental measurement data; The virtual aircraft behavior calculation unit calculates the virtual aircraft behavior calculation information related to the aircraft behavior of the virtual aircraft based on the virtual environment state and the virtual aircraft model information.

11. 2. The operational state simulation system according to claim 1, an actual aircraft behavior calculation unit that calculates actual aircraft behavior calculation information regarding aircraft behavior of the actual aircraft based on actual aircraft model information that models the actual aircraft in the virtual space; An operational state simulation system, wherein the virtual aircraft behavior calculation unit calculates the virtual aircraft behavior calculation information regarding the aircraft behavior of the virtual aircraft based on a comparison result between the actual aircraft behavior calculation information calculated by the actual aircraft behavior calculation unit and the actual aircraft behavior information acquired by the actual aircraft behavior information acquisition unit.

12. 2. The operational state simulation system according to claim 1, An operational state simulation system, wherein the virtual aircraft behavior calculation information calculated by the virtual aircraft behavior calculation unit includes at least one of the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of the virtual aircraft.

13. 2. The operational state simulation system according to claim 1, a target achievement determination unit that determines whether the operational state estimation value of the mixed unmanned aerial vehicle group calculated based on the behavior information of the mixed unmanned aerial vehicle group satisfies the set target; An operational state simulation system in which, when the goal achievement determination unit determines that the operational state estimation value does not satisfy the set goal, the operational requirement determination unit changes the operational requirements for the mixed unmanned aircraft group.

14. 2. The operational state simulation system according to claim 1, a target achievement determination unit that determines whether the operational state estimation value of the mixed unmanned aerial vehicle group calculated based on the behavior information of the mixed unmanned aerial vehicle group satisfies the set target; An operational state simulation system comprising: an operation command determination unit that determines operation commands including commands regarding at least one of the position, attitude, movement speed, rotational angular velocity, movement acceleration, and rotational angular acceleration of the actual or virtual aircraft included in the mixed unmanned aircraft group based on the judgment result of the goal achievement judgment unit.

15. 15. The operating state simulation system according to claim 14, An operational state simulation system including an information output unit that transmits and outputs the operation command to the actual device, or displays and outputs information about the operation command.

16. 2. The operational state simulation system according to claim 1, An operational state simulation system comprising an information output unit that transmits or displays information about the operational requirements or the candidate operational requirements determined by the operational requirement determination unit to an external device.

17. 2. The operational state simulation system according to claim 1, An operational state simulation system comprising an information output unit that transmits or displays information about the behavior of the mixed unmanned aircraft group or information about the operational state estimate value to the outside.

18. 2. The operational state simulation system according to claim 1, An operational state simulation system comprising an information output unit that displays and outputs the actual aircraft behavior information acquired from the actual aircraft or actual aircraft measurement data measured by a measurement sensor mounted on the actual aircraft.

19. 2. The operational state simulation system according to claim 1, an environmental prediction data acquisition unit that acquires environmental prediction data after a predetermined time has elapsed in the operation area; the virtual space generation unit generates a predicted virtual environment state of the virtual operation area after the predetermined time has elapsed based on the environment prediction data; the virtual aircraft behavior calculation unit calculates the virtual aircraft behavior calculation information related to aircraft behavior of the virtual aircraft in the virtual operation area after the predetermined time has elapsed, based on the virtual environment predicted state and the virtual aircraft model information; The mixed unmanned aircraft group behavior generation unit calculates the behavior information of the mixed unmanned aircraft group after the predetermined time has elapsed based on the actual aircraft behavior information and the virtual aircraft behavior calculation information, The operational requirements determination unit is an operational state simulation system that determines the operational requirements or candidate operational requirements of the unmanned aircraft group so that the operational state estimate value of the mixed unmanned aircraft group after the predetermined time has elapsed, which is calculated based on the behavior information of the mixed unmanned aircraft group, meets the set target regarding the operational state.

20. 2. The operational state simulation system according to claim 1, An operational state simulation system including a user input receiving unit that receives user input information for changing the operational requirement or selecting a specific candidate from the operational requirement candidates.

21. 2. The operational state simulation system according to claim 1, An operational state simulation system including a user input receiving unit that receives, from a user, user input information for setting or changing the set target.

22. An operational state simulation method for simulating an operational state in an operational area of ​​a group of unmanned aerial vehicles including a plurality of unmanned aerial vehicles by simulation, The computer a virtual space generation step of generating a virtual operation area in the virtual space; an actual aircraft behavior information acquisition step of acquiring actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; a virtual aircraft behavior calculation step of calculating virtual aircraft behavior calculation information regarding aircraft behavior of the virtual aircraft of the unmanned aircraft deployed in the virtual operation area generated in the virtual space based on virtual aircraft model information; a mixed unmanned aircraft group behavior generation step of generating behavior information of a mixed unmanned aircraft group including a mixture of the real aircraft and the virtual aircraft based on the real aircraft behavior information and the virtual aircraft behavior calculation information; an operational requirement determination step of determining operational requirements or candidate operational requirements for the unmanned aerial vehicle group so that the operational state estimate of the mixed unmanned aerial vehicle group calculated based on the behavior information of the mixed unmanned aerial vehicle group satisfies a preset target related to the operational state; an information output step of transmitting or displaying information relating to the determined operational requirement or the candidate operational requirement to the outside.

23. A program usable for an operational status simulation system that simulates an operational status in an operational area of ​​a group of unmanned aerial vehicles including a plurality of unmanned aerial vehicles, To the computer a virtual space generation command for generating a virtual operation area in the virtual space; an actual aircraft behavior information acquisition command to acquire actual aircraft behavior information regarding aircraft behavior from an actual unmanned aircraft deployed in the operation area in real space; a virtual aircraft behavior calculation command that calculates virtual aircraft behavior calculation information regarding aircraft behavior of the virtual aircraft of the unmanned aircraft deployed in the virtual operation area generated in the virtual space based on virtual aircraft model information; A mixed unmanned aircraft group behavior generation command that generates behavior information of a mixed unmanned aircraft group in which the real aircraft and the virtual aircraft are mixed based on the real aircraft behavior information and the virtual aircraft behavior calculation information; An operational requirement determination command that determines operational requirements or candidate operational requirements for the unmanned aerial vehicle group so that the operational state estimate of the mixed unmanned aerial vehicle group calculated based on the behavior information of the mixed unmanned aerial vehicle group satisfies a preset target related to the operational state; A program that executes an information output command to transmit or display information about the determined operational requirement or candidate operational requirement to the outside.

Images