Information processing system, information processing device, and information processing method

Abstract

The present technology relates to an information processing system, an information processing device, and an information processing method that make it possible for a mobile body to appropriately execute a mission. An information processing system according to the present invention comprises a mission control unit that creates an execution plan for a mission that includes photography of a subject from a mobile body on the basis of the results of a user having executed a simulation of the mission at an information processing terminal. The present technology can be applied, for example, to a system that controls the execution of a mission by a drone.

Description

Information processing system, information processing device, and information processing method

[0001] The present technology relates to an information processing system, an information processing device, and an information processing method, and more particularly to an information processing system, an information processing device, and an information processing method that enable a mobile object to appropriately execute a mission.

[0002] In recent years, the commercialization of inspection work using drones has progressed in various fields (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2019-159755

[0004] However, there is a lack of past experience with drone-based inspections, and workflows have not yet been established. As a result, execution methods (e.g., flight routes and inspection methods) vary from operator to operator, and each operator is still searching for the optimal execution method. For example, execution methods are often determined on-site based on the object being inspected, and inspection work is not always carried out appropriately.

[0005] This technology was developed in light of these circumstances, and enables mobile objects such as drones to properly carry out missions such as inspection work.

[0006] An information processing system according to one aspect of the present technology includes a mission control unit that creates an execution plan for a mission including photographing an object from a moving body based on the results of a simulation of the mission performed by a user on an information processing terminal.

[0007] In an information processing method according to one aspect of the present technology, an information processing system creates an execution plan for a mission including photographing an object from a moving body, based on a result of a simulation of the mission performed by a user on an information processing terminal.

[0008] According to one aspect of the present technology, an information processing device includes a mission control unit that creates an execution plan for a mission including photographing an object from a moving body based on a result of a simulation of the mission performed by a user on an information processing terminal.

[0009] In one aspect of the present technology, an execution plan for a mission including photographing an object from a moving body is created based on a result of a simulation of the mission performed by a user on an information processing terminal.

[0010] 1 is a block diagram showing an embodiment of a drone control system to which the present technology is applied; FIG. 2 is a flowchart for explaining processing of the drone control system; FIG. 3 is a sequence diagram for explaining details of a setting process; FIG. 4 is a sequence diagram for explaining details of a simulation process; FIG. 5 is a schematic diagram showing an example of a simulated object; FIG. 6 is a schematic diagram showing an example screen of a simulated mission game; FIG. 7 is a schematic diagram showing an example screen of a simulated mission game; FIG. 8 is a sequence diagram for explaining details of a learning process; FIG. 9 is a sequence diagram for explaining details of a mission execution process; FIG. 10 is a block diagram showing a first modified example of the drone control system; FIG. 11 is a block diagram showing a second modified example of the drone control system; FIG. 12 is a block diagram showing an example configuration of a computer;

[0011] Hereinafter, embodiments of the present technology will be described. The description will be made in the following order: 1. Embodiment 2. Modification 3. Other

[0012] <<1. Embodiment>> An embodiment of the present technology will be described with reference to FIGS. 1 to 9 .

[0013] <Configuration Example of Drone Control System 1> First, a configuration example of a drone control system 1 to which the present technology is applied will be described with reference to Fig. 1. The drone control system 1 is a system that, for example, creates an execution plan for executing a given mission and controls the operation of the drone 2 in accordance with the execution plan to execute the mission.

[0014] Here, the content of the mission is not particularly limited as long as the drone 2 can execute it autonomously. For example, the mission may be photographing, inspecting, surveying, etc. of a target object such as a specified area or object. For example, the mission may be creating, manufacturing, constructing, repairing, maintaining, etc. of a target object.

[0015] The target object is not particularly limited, and examples thereof include structures such as buildings, towers, bridges, dams, and roads, land, forests, and fields.

[0016] The drone control system 1 includes a cloud system 11, an administrator terminal 12, user terminals 13-1 to 13-n, and a control terminal 14. The cloud system 11, the administrator terminal 12, the user terminals 13-1 to 13-n, and the control terminal 14 are connected via a network (not shown) such as the Internet.

[0017] Hereinafter, when there is no need to distinguish between the user terminals 13-1 to 13-n, they will be simply referred to as user terminals 13.

[0018] The cloud system 11 is an information processing system including one or more information processing devices (e.g., servers, etc.). The cloud system 11 includes an information processing unit 21, a mission DB (database) 22, a condition DB (database) 23, and an execution result DB (database) 24.

[0019] The information processing unit 21 controls the simulation of a mission by the user terminal 13 and the execution of the mission by the drone 15. The information processing unit 21 includes a setting unit 31, a simulation control unit 32, a learning unit 33, and a mission control unit 34.

[0020] The setting unit 31 executes various setting processes necessary for executing missions and mission simulations. For example, the setting unit 31 acquires mission-related data related to the missions from the administrator terminal 12. The mission-related data includes mission data related to the missions and condition data related to the execution conditions of the missions. The setting unit 31 stores the mission data in the mission DB 22 and the condition data in the condition DB 23.

[0021] The simulation control unit 32 controls the execution of a mission simulation in each user terminal 13. For example, the simulation control unit 32 creates a simulation for virtually executing a mission in the user terminal 13 based on the mission data stored in the mission DB 22 and the condition data stored in the condition DB 23. The simulation control unit 32 controls the execution of the simulation in the user terminal 13. The simulation control unit 32 receives an execution log indicating the execution content of the simulation from the user terminal 13 and stores the log in the execution result DB 24. The simulation control unit 32 obtains score data indicating the score for each user's simulation execution result from the learning unit 33 and transmits the score data to the user terminal 13.

[0022] The learning unit 33 learns how to execute a mission. For example, the learning unit 33 evaluates the execution results of each user's simulation based on the mission data stored in the mission DB 22, the condition data stored in the condition DB 23, and the execution logs of each user stored in the execution result DB 24, and learns how to execute the mission. The learning unit 33 generates execution method data indicating how to execute the mission, and stores the data in the mission DB 22. The learning unit 33 generates evaluation data indicating an evaluation of the execution results of each user's simulation, and stores the evaluation data in the execution result DB 24. The learning unit 33 generates score data indicating the score for the execution results of each user's simulation, and supplies the score data to the simulation control unit 32.

[0023] The mission control unit 34 controls the execution of missions by the drone 2. For example, the mission control unit 34 creates a mission execution plan based on the mission data and execution method data stored in the mission DB 22 and the condition data stored in the condition DB 23. The mission control unit 34 generates a mission file for causing the drone 2 to execute the mission in accordance with the execution plan based on the created execution plan and the aircraft data included in the condition data. The mission control unit 34 transmits the mission file to the control terminal 14.

[0024] The administrator terminal 12 is an information processing terminal used by an administrator such as the owner of the object, the owner of the drone 2, or the business operator that executes the mission. The administrator terminal 12 is configured by an information processing terminal such as a smartphone, a tablet terminal, or a PC (personal computer).

[0025] The user terminal 13 is an information processing terminal used by a user who executes a mission simulation. The administrator terminal 12 is configured by an information processing terminal such as a smartphone, a tablet terminal, a PC, or a game terminal.

[0026] The control terminal 14 is configured, for example, by a proportional system. The control terminal 14 is capable of directly communicating wirelessly with the drone 2 and is used to remotely control the drone 2.

[0027] The drone 2 can communicate directly with the control terminal 14 by a predetermined communication method. For example, a 2.4 GHz spread spectrum method is used as the communication method between the drone 2 and the control terminal 14.

[0028] The drone 2 is equipped with an image sensor such as a camera, and is capable of photographing the surroundings of the drone 2.

[0029] The drone 2 can fly by remote control using the control terminal 14. The drone 2 can also fly autonomously without being remotely controlled by the control terminal 14. For example, the drone 2 can execute a mission while flying autonomously in accordance with a mission file provided by the control terminal 14.

[0030] <Processing of drone control system 1> Next, processing of the drone control system 1 will be described with reference to the flowchart of FIG. 2 .

[0031] In step S1, the drone control system 1 executes a setting process.

[0032] Here, the setting process will be described in detail with reference to the sequence diagram of FIG.

[0033] In step S101, the manager terminal 12 inputs mission-related data.

[0034] For example, the administrator uses a web application provided by the cloud system 11 on the user terminal 13 to input mission-related data regarding the mission to be performed by the drone 2.

[0035] The mission-related data includes, for example, mission data and condition data.

[0036] The mission data includes, for example, object data relating to the object for which the mission is to be executed, and mission content data indicating the content of the mission.

[0037] For example, the object data may include data of a three-dimensional model of the object, and may also include characteristics of the object (e.g., size, shape, appearance, material, location, area, movement, etc.).

[0038] The mission content data includes, for example, at least one of the following: type of mission, specific content, completion conditions, and limiting conditions.

[0039] The types of missions are classified into, for example, photographing, inspecting, and surveying the target object.

[0040] The specific contents of the mission may include, for example, target points indicating the position and range of an object to be photographed, inspected, surveyed, etc. Note that the mission may also include three-dimensional model data indicating each target point of the object. The specific contents of the mission may include, for example, procedures for photographing, inspecting, surveying, etc.

[0041] The mission completion conditions are conditions for determining whether the mission is complete. For example, the number of target points that require photography, inspection, surveying, etc., and the percentage of target points that require photography, inspection, surveying, etc., are set as the mission completion conditions.

[0042] The condition data includes, for example, data related to the execution conditions of the mission, such as aircraft data indicating conditions related to the drone 2 used for the mission, environmental data indicating conditions related to the environment in which the mission is executed, and limiting condition data related to limiting conditions that limit the execution of the mission.

[0043] The aircraft data includes, for example, data related to the model name, performance, specifications, etc. of the drone 2. The performance of the drone 2 includes, for example, one or more of maximum speed, maximum ascent speed, maximum descent speed, maximum angular velocity, maximum tilt angle, maximum wind resistance, maximum operational altitude limit, operating ambient temperature, maximum flight time, hovering accuracy, etc. The specifications of the drone 2 include, for example, one or more of size, weight, the type and number of sensors such as cameras installed, and performance (for example, resolution, shooting range, continuous shooting speed, etc.).

[0044] The environmental data includes, for example, one or more of data indicating the location of the object (e.g., map data), data on obstacles around the object, data on the weather around the object, etc. The data on obstacles around the object includes, for example, three-dimensional model data indicating the positions, shapes, etc. of obstacles.

[0045] The restriction condition data includes, for example, one or more of the following: the time period during which the mission can be carried out, the weather conditions under which the mission can be carried out, the speed limit of the drone 2, the area in which the drone 2 can fly, and the recommended distance that the drone 2 should stay away from the target object (hereinafter referred to as the recommended separation distance).

[0046] For example, some or all of the parameters of the mission-related data may be preset, and the administrator may change them as necessary.

[0047] The administrator terminal 12 transmits the mission-related data to the cloud system 11 .

[0048] In response to this, in step S102, the setting unit 31 of the cloud system 11 sets the mission-related data. Specifically, the setting unit 31 receives the mission-related data from the administrator terminal 12. The setting unit 31 stores the mission data included in the mission-related data in the mission DB 22, and stores the condition data included in the mission-related data in the condition DB 23.

[0049] The setting process then ends.

[0050] Returning to Figure 2, in step S2, the drone control system 1 executes a simulation process.

[0051] Here, the simulation process will be described in detail with reference to the flowchart of FIG.

[0052] In step S121, the simulation control unit 32 creates a simulation. Specifically, the simulation control unit 32 reads mission data from the mission DB 22 and reads condition data from the condition DB 23. Based on the mission data and the condition data, the simulation control unit 32 creates a simulation for simulating the execution of a given mission on the user terminal 13. For example, the simulation control unit 32 generates a game execution engine, which is software for executing a game that simulates the given mission (hereinafter referred to as a simulated mission game) on the user terminal 13.

[0053] The simulated mission game is a game in which a simulated mobile object, which is a simulation of a drone 2, is used to execute a simulated mission against a simulated target object, which is a simulation of an object.

[0054] The type of simulated mission game is not particularly limited as long as it simulates a mission, and may be, for example, a shooting game.

[0055] For example, the simulated object does not necessarily have to be the same type of object as the real object as long as it has a similar shape. For example, as shown in Fig. 5, if the real object is a steel tower 101, an object 102 that is a deformed version of the steel tower 101 may be used as the simulated object. Furthermore, for example, a dragon 103 that is a character version of the object 102 may be used as the simulated object.

[0056] In this figure, only the outline of the dragon 103 is shown, but in an actual simulated mission game, a detailed picture of the dragon is drawn.

[0057] The simulated moving object may be a different type of moving object from a drone, as long as it is capable of moving in the same manner as the drone 2 in the simulated mission game.

[0058] For example, the mission data input from the manager terminal 12 may include two-dimensional or three-dimensional models of the simulated objects and simulated moving bodies.

[0059] The simulated mission does not necessarily have to be the same as the real mission, as long as the execution method is similar to the real mission. For example, if the real mission is to photograph an object, the simulated mission may be a mission in which, instead of photographing the simulated object, points on the simulated object that correspond to target points on the real object are attacked or painted over.

[0060] Furthermore, the execution conditions of the simulated mission are set based on the condition data. For example, the execution conditions of the real mission indicated in the condition data are converted into execution conditions that correspond to the environment of the simulated mission game.

[0061] For example, the condition data input from the administrator terminal 12 may include data regarding the execution conditions of a simulated mission in a simulated mission game.

[0062] The simulated mission game may be a game played by one person, or may be a game played by multiple users (for example, a competitive game, etc.). The simulated mission may also be a game in which a player competes against a simulated player (computer).

[0063] In step S122, the simulation control unit 32 controls the execution of the simulation. For example, the simulation control unit 32 controls the execution of the simulated mission game in each user terminal 13 by providing each user terminal 13 with a game execution engine.

[0064] The game execution engine may be executed on the user terminal 13 or on the simulation control unit 231 (on the cloud system 11). Also, for example, the game execution engine may be executed in the user terminal 13 and the simulation control unit 231 in cooperation with each other.

[0065] In response to this, in step S123, each user terminal 13 executes a simulation. For example, each user terminal 13 uses a game execution engine provided by the simulation control unit 32 to control the execution of the simulated mission game.

[0066] In response to this, each user plays the simulated mission game using each user terminal 13. As a result, a mission simulation (simulated mission) is executed on each user terminal 13. For example, each user competes with other users in terms of the time it takes to complete the simulated mission, the degree of completion, etc.

[0067] Each user may be allowed to freely set various settings such as the environment of the simulated mission game, etc. For example, the user may be allowed to freely set the performance and equipment of the simulated moving object, etc.

[0068] 6 and 7 show an example of a screen of a simulated mission game. Specifically, Fig. 6 and Fig. 7 show an example of a screen of a simulated mission game in which a drone 121, which is a simulated moving body, is moved to capture images of a steel tower 122, which is a simulated object, in full detail.

[0069] The hatched area A1 in Fig. 6 indicates the imaging range of the drone 121. The elliptical area A2 in Fig. 7 indicates the area that has already been imaged. For example, the game is cleared when the imaged area A2 covers the entire pylon 122.

[0070] It is anticipated that, for example, by utilizing the customer base of an online game service or the like, an unspecified number of users will be able to play the simulated mission game.

[0071] In step S124, the simulation control unit 32 collects the execution log of the simulation for each user.

[0072] Specifically, each user terminal 13 transmits an execution log of the simulated mission game to the cloud system 11 .

[0073] The execution log includes, for example, the movement route of the simulated moving body and the operation details for the simulated moving body. For example, if the simulated mission is to photograph a simulated object, the operation details for the simulated moving body include the position of the simulated moving body when photographing the simulated object, the photographing direction (camera orientation), camera settings (for example, exposure, zoom, etc.), etc. For example, if the simulated mission is to attack a simulated object (for example, shooting), the operation details for the simulated moving body include the position of the simulated moving body when attacking the simulated object, the attack method, and the attack direction (for example, shooting direction, etc.).

[0074] In response to this, the simulation control unit 32 of the cloud system 11 receives the execution logs from each user terminal 13. The simulation control unit 32 stores the execution logs received from each user terminal 13 in the execution result DB 24.

[0075] Each user terminal 13 may transmit the execution log to the cloud system 11 in real time while the simulated mission game is being executed, or may transmit the execution log to the cloud system 11 all at once after execution.

[0076] Then, the simulation process ends.

[0077] Returning to Figure 2, in step S3, the drone control system 1 executes a learning process.

[0078] Here, the learning process will be described in detail with reference to the sequence diagram of FIG.

[0079] In step S141, the learning unit 33 evaluates the execution results of the simulation for each user. Specifically, the learning unit 33 reads mission data from the mission DB 22, reads condition data from the condition DB 23, and reads the execution log of each user from the execution result DB 24.

[0080] For example, the learning unit 33 evaluates the results of each user's simulation execution based on the mission data, condition data, and each user's execution log. For example, the learning unit 33 calculates each user's simulated mission achievement rate, completion rate, achievement time, etc. The learning unit 33 calculates a score indicating an evaluation of each user's simulation execution result. The score is expressed, for example, by points, level, ranking among all users, etc.

[0081] The learning unit 33 generates evaluation data indicating an evaluation and score for each user's simulation execution result, and stores the evaluation data in the execution result DB 24. The learning unit 33 generates score data indicating a score for each user's simulation execution result, and transmits the score data to the user terminal 13 of each user.

[0082] In response to this, in step S142, the user terminal 13 of each user presents the score. Specifically, each user terminal 13 receives score data from the learning unit 33. Each user terminal 13 presents the score of each user in the simulated mission game based on the score data. In this way, an evaluation of the results of the simulation is fed back to the user.

[0083] In step S143, the learning unit 33 learns how to execute the mission based on the results of the simulation executed by each user.

[0084] For example, the learning unit 33 extracts the execution logs of users with high ratings. For example, the learning unit 33 may extract the execution logs of the user with the highest rating, or may extract the execution logs of users whose ratings are equal to or higher than a threshold, or may extract the execution logs of a predetermined number of users with the highest ratings.

[0085] The learning unit 33 determines a method for executing the mission based on the extracted execution log.

[0086] If there is only one extracted execution log, for example, the learning unit 33 converts the execution log into an actual mission execution method based on the mission data and condition data.

[0087] When two or more execution logs are extracted, for example, the learning unit 33 combines the multiple execution logs. In this case, for example, optimal portions may be extracted from the multiple execution logs and combined, or an average log may be generated from the multiple execution logs. For example, the learning unit 33 converts the combined execution logs into an actual mission execution method based on the mission data and condition data.

[0088] As a result, for example, when the flight route of drone 2 is set based on the movement routes of multiple users' simulated moving bodies, the flight route may be set based on a route that connects the optimal parts of the movement routes of the multiple users, or the flight route may be set based on the average route of the movement routes of the multiple users.

[0089] The learning unit 33 may determine a mission execution method through machine learning. For example, the learning unit 33 determines a mission execution method by performing machine learning using the execution logs and evaluation data of each user, the mission data, and the condition data. In this case, the execution logs and evaluation data of all users may be used, or the execution logs and evaluation data of some users (e.g., users with high ratings) may be used.

[0090] The mission execution method includes, for example, a method for photographing an object when photographing or inspecting the object. The method for photographing an object includes, for example, the movement route of the drone 2 and the photographing positions at which the object is photographed from the drone 2. The method for photographing an object may also include one or more of the orientation and settings (e.g., exposure, zoom, backlight compensation, etc.) of the camera of the drone 2 at each photographing position. Furthermore, for example, if the drone is equipped with multiple cameras, the method for photographing an object may also include the camera to be used.

[0091] The learning unit 33 generates execution method data indicating the execution method thus obtained and stores the data in the mission DB 22 .

[0092] The learning process then ends.

[0093] Returning to Figure 2, in step S4, the drone control system 1 executes a mission execution process.

[0094] Here, details of the mission execution process will be described with reference to the sequence diagram of FIG.

[0095] In step S161, the mission control unit 34 creates a mission execution plan. Specifically, the mission control unit 34 reads execution method data from the mission DB 22 and reads condition data from the condition DB 23. The mission control unit 34 creates a mission execution plan based on the execution method data and the condition data.

[0096] The mission execution plan includes, for example, a method for executing the mission and a schedule.

[0097] In step S162, the mission control unit 34 generates a mission file. Specifically, the mission control unit 34 generates a mission file for causing the drone 2 to execute the mission in accordance with the execution plan based on the created execution plan and the aircraft data included in the condition data. The mission file is, for example, a file in KML (Keyhole Markup Language) format. The mission control unit 34 transmits the mission file to the control terminal 14.

[0098] In step S163, the control terminal 14 causes the drone 2 to execute the mission. Specifically, the control terminal 14 receives the mission file from the mission control unit 34. The control terminal 14 imports the mission file and transfers the imported mission file to the drone 2.

[0099] In response, the drone 2 receives the mission file from the control terminal 14. The drone 2 executes the mission based on the mission file. For example, the drone 2 flies along a set flight route based on the mission file and photographs or inspects an object.

[0100] In this way, it is possible to easily collect a large number of simulation execution result samples (execution logs) for a given mission. Furthermore, by learning how to execute the mission based on the collected samples, it is possible to easily determine an appropriate execution method before executing the mission. This allows the mission to be executed appropriately.

[0101] For example, if the type of object and the type of mission are the same, even if the object or the surrounding environment changes, it is possible to learn using samples collected for the same type of object and mission. For example, it is possible to use samples collected when inspecting a certain steel tower to learn how to inspect another steel tower.

[0102] This makes it possible to easily and quickly establish an appropriate execution method for each type of object and mission, which will, for example, improve the reliability of various drone-based operations and reduce costs, thereby contributing to the development of the drone industry and an expansion of the customer base.

[0103] <<2. Modifications>> Modifications of the above-described embodiments of the present technology will now be described.

[0104] <First Modified Example of Drone Control System> FIG. 10 shows an example configuration of a drone control system 201 which is a first modified example of the drone control system.

[0105] Note that parts corresponding to those in the drone control system 1 in Figure 1 are given the same symbols, and their explanations will be omitted as appropriate.

[0106] 1, the drone control system 201 is the same as the drone control system 1 in FIG. 1 in that it includes an administrator terminal 12, user terminals 13-1 to 13-n, and a control terminal 14, but differs in that it includes a cloud system 211 instead of the cloud system 11. When comparing the cloud system 211 with the cloud system 11, it is the same as the cloud system 11 in that it includes a mission DB 22 and a condition DB 23, and differs in that it includes an information processing unit 221 instead of the information processing unit 21 and the execution result DB 24 has been deleted. When comparing the information processing unit 221 with the information processing unit 21, it is the same as the cloud system 11 in that it includes a setting unit 31. When comparing the information processing unit 221 with the information processing unit 21, it is different in that it includes a simulation control unit 231 and a mission control unit 232 instead of the simulation control unit 32 and the mission control unit 34, a model generation unit 233 has been added, and the learning unit 33 has been deleted.

[0107] The simulation control unit 231, like the simulation control unit 32 of the cloud system 11, controls the execution of the simulation in each user terminal 13. The simulation control unit 231 receives an execution log from each user terminal 13 and provides it to the mission control unit 232.

[0108] The mission control unit 232 creates a mission file based on the execution log, the mission data stored in the mission DB 22 , and the condition data stored in the condition DB 23 , and transmits the file to the control terminal 14 .

[0109] The control terminal 14 imports the mission file and transfers the imported mission file to the drone 2.

[0110] Drone 2 executes the mission based on the mission file.

[0111] The model generation unit 233 receives an image of an object captured by the drone 2. The model generation unit 233 generates a three-dimensional model of the object based on the received image. The model generation unit 233 stores the three-dimensional model of the object in the mission DB 22.

[0112] The simulation control unit 231 reflects the three-dimensional model of the object generated by the model generation unit 233 in the simulation.

[0113] As a result, the results of the simulation executed on the user terminal 13 are reflected in real time in the operation of the drone 2. In addition, a three-dimensional model of the object generated based on an image of the object photographed by the drone 2 is reflected in real time in the simulation. As a result, by executing a simulation on the user terminal 13, the user can remotely control the operation of the drone 2 in real time and cause it to carry out a mission.

[0114] For example, the number of user terminals 13 that can simultaneously execute a simulation may be limited to one, or multiple user terminals 13 may be allowed to simultaneously execute a simulation.

[0115] In the latter case, for example, the mission control unit 232 selects an execution log to be used for executing the mission from among the execution logs from multiple user terminals 13. For example, the mission control unit 232 may most appropriately select the execution log of the user currently executing the simulation, or may randomly select an execution log.

[0116] <Second Modification of Drone Control System> FIG. 11 shows an example of the configuration of a drone control system 301 which is a second modification of the drone control system.

[0117] Note that parts corresponding to the drone control system 1 in Figure 1 and the drone control system 201 in Figure 10 are given the same symbols, and their explanations will be omitted as appropriate.

[0118] 1, the drone control system 301 is the same as the drone control system 1 in that it includes an administrator terminal 12, user terminals 13-1 to 13-n, and a control terminal 14, but differs in that it includes a cloud system 311 instead of the cloud system 11. When comparing the cloud system 311 with the cloud system 11, it is the same as the cloud system 11 in that it includes a mission DB 22, a condition DB 23, and an execution result DB 24, but differs in that it includes an information processing unit 321 instead of the information processing unit 21. When comparing the information processing unit 321 with the information processing unit 21, it is the same as the cloud system 11 in that it includes a setting unit 31, a learning unit 33, and a mission control unit 34. When comparing the information processing unit 221 with the information processing unit 21, it is the same as the cloud system 11 in that it includes a simulation control unit 331 instead of the simulation control unit 32, and a model generation unit 233 has been added.

[0119] The simulation control unit 331 executes substantially the same processing as the simulation control unit 32 of the cloud system 11. However, the simulation control unit 331 creates a simulation by further using a three-dimensional model of the object generated by the model generation unit 233. This further enhances the sense of realism of the simulation.

[0120] <Modifications Regarding Simulation> The simulation performed by the user does not necessarily have to be a game. For example, the user may perform a simulation that faithfully reproduces a mission.

[0121] For example, the learning unit 33 may learn how to execute a mission based on the results of one user executing a simulation multiple times.

[0122] <Modifications of Processing Allocation> The above-described processing allocation can be changed as appropriate.

[0123] For example, the user terminal 13 and the control terminal 14 may execute part of the processing of the above-described cloud system 11. For example, the control terminal 14 may generate a mission file.

[0124] <Other Modifications> The present technology can be applied to autonomous flying objects that can fly and execute missions autonomously, other than drones. For example, the present technology can be applied to flying cars that execute missions autonomously.

[0125] This technology can be applied to autonomous moving objects that can move and execute missions autonomously, in addition to autonomous air vehicles. For example, this technology can be applied to self-driving vehicles that execute missions autonomously.

[0126] <<3. Others>> <Example of Computer Configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, that can execute various functions by installing various programs.

[0127] FIG. 12 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0128] In the computer 1000 , a CPU (Central Processing Unit) 1001 , a ROM (Read Only Memory) 1002 , and a RAM (Random Access Memory) 1003 are interconnected by a bus 1004 .

[0129] An input / output interface 1005 is further connected to the bus 1004. An input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010 are connected to the input / output interface 1005.

[0130] The input unit 1006 includes input switches, buttons, a microphone, an image sensor, etc. The output unit 1007 includes a display, a speaker, etc. The storage unit 1008 includes a hard disk, a non-volatile memory, etc. The communication unit 1009 includes a network interface, etc. The drive 1010 drives removable media 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0131] In the computer 1000 configured as described above, the CPU 1001 performs the above-described series of processes by, for example, loading a program recorded in the memory unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing it.

[0132] The program executed by the computer 1000 (CPU 1001) can be provided by being recorded on a removable medium 1011 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0133] In the computer 1000, the program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting the removable medium 1011 into the drive 1010. The program can also be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. Alternatively, the program can be installed in the ROM 1002 or the storage unit 1008 in advance.

[0134] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0135] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0136] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0137] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0138] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0139] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0140] <Examples of Combinations of Configurations> The present technology can also have the following configurations.

[0141] (1) An information processing system comprising: a mission control unit that creates an execution plan for a mission based on the results of a user running a simulation of the mission, including photographing an object from a moving body, on an information processing terminal. (2) The information processing system described in (1), wherein the execution plan includes an execution method for the mission. (3) The information processing system described in (2), further comprising: a learning unit that learns the execution method based on the execution results of the simulation, wherein the mission control unit creates the execution plan based on the learning results of the execution method. (4) The information processing system described in (3), wherein the learning unit learns the execution method based on the execution results of the simulations of multiple users. (5) The information processing system described in any of (2) to (4), wherein the execution method includes a method for photographing the object. (6) The information processing system described in (5), wherein the method for photographing the object includes a movement route of the moving body and a photographing position for photographing the object from the moving body. (7) The information processing system according to (6), wherein the method for photographing the object further includes at least one of a direction of a camera used to photograph the object and a setting value of the camera. (8) The information processing system according to any of (1) to (7), further comprising a simulation control unit that controls execution of the simulation in the information processing terminal. (9) The information processing system according to (8), wherein the simulation control unit creates the simulation based on the object, the content of the mission, and execution conditions of the mission. (10) The information processing system according to (9), wherein the execution conditions include at least one of a condition related to the moving object, a condition related to an environment in which the mission is executed, and a condition that restricts execution of the mission. (11) The information processing system according to any of (8) to (10), wherein the simulation control unit acquires an execution log of the simulation from the information processing terminal, and further comprises a learning unit that evaluates execution results of the simulation based on the execution log.(12) The information processing system according to (11), wherein the simulation control unit feeds back an evaluation of the execution result of the simulation to the information processing terminal. (13) The information processing system according to any of (1) to (12), further comprising: the mission control unit controlling execution of the mission by the moving object based on the execution plan. (14) The information processing system according to any of (1) to (13), wherein the simulation is realized by a game. (15) The information processing system according to (14), wherein, in the game, a simulated moving object that simulates the mission is executed against a simulated object that simulates the object, using a simulated moving object that simulates the moving object. (16) The information processing system according to (15), wherein the simulated object is generated based on a three-dimensional model of the object. (17) The information processing system according to any of (1) to (16), wherein the moving object is an air vehicle. (18) The information processing system according to (17), wherein the air vehicle is a drone. (19) An information processing method, in which an information processing system creates an execution plan for a mission based on a result of a simulation of the mission, which includes photographing an object from a moving body, performed by a user on an information processing terminal. (20) An information processing device, comprising: a mission control unit that creates an execution plan for the mission based on a result of a simulation of the mission, which includes photographing an object from a moving body, performed by a user on an information processing terminal.

[0142] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0143] REFERENCE SIGNS LIST 1 Drone control system, 2 Drone, 11 Cloud system, 12 Administrator terminal, 13-1 to 13-n User terminal, 14 Control terminal, 21 Information processing unit, 22 Mission DB, 23 Condition DB, 24 Execution result DB, 31 Setting unit, 32 Simulation control unit, 33 Learning unit, 34 Mission control unit, 201 Drone control system, 211 Cloud system, 221 Information processing unit, 231 Simulation control unit, 232 Mission control unit, 233 Model generation unit, 301 Drone control system, 311 Cloud system, 321 Information processing unit, 331 Simulation control unit

Claims

1. An information processing system comprising a mission control unit that creates an execution plan for a mission including photographing an object from a moving body based on the results of a simulation of the mission performed by a user on an information processing terminal.

2. The information processing system according to claim 1, wherein the execution plan includes a method for executing the mission.

3. The information processing system according to claim 2, further comprising a learning unit that learns the execution method based on the execution results of the simulation, wherein the mission control unit creates the execution plan based on the learning results of the execution method.

4. The information processing system according to claim 3, wherein the learning unit learns the execution method based on execution results of the simulation by a plurality of users.

5. The information processing system according to claim 2, wherein the execution method includes a method for photographing the object.

6. An information processing system according to claim 5, wherein the method for photographing the object includes a movement route of the mobile body and a photographing position for photographing the object from the mobile body.

7. The information processing system according to claim 6, wherein the method for photographing the object further includes at least one of the orientation of a camera used to photograph the object and settings of the camera.

8. The information processing system according to claim 1, further comprising a simulation control unit that controls the execution of the simulation in the information processing terminal.

9. The information processing system according to claim 8, wherein the simulation control unit creates the simulation based on the object, the content of the mission, and the execution conditions of the mission.

10. The information processing system according to claim 9, wherein the execution conditions include at least one of a condition related to the moving body, a condition related to the environment in which the mission is executed, and a condition that restricts the execution of the mission.

11. The information processing system according to claim 8, wherein the simulation control unit acquires an execution log of the simulation from the information processing terminal, and further comprises a learning unit that evaluates the execution results of the simulation based on the execution log.

12. The information processing system according to claim 11, wherein the simulation control unit feeds back an evaluation of the results of the simulation to the information processing terminal.

13. The information processing system according to claim 1, further comprising: the mission control unit controlling the execution of the mission by the mobile object based on the execution plan.

14. The information processing system according to claim 1, wherein the simulation is realized by a game.

15. The information processing system according to claim 14, wherein in the game, a simulated mission is executed by simulating the mission against a simulated object that simulates the object, using a simulated moving object that simulates the moving object.

16. The information processing system according to claim 15, wherein the simulated object is generated based on a three-dimensional model of the object.

17. The information processing system according to claim 1, wherein the moving body is an air vehicle.

18. The information processing system according to claim 17, wherein the flying object is a drone.

19. An information processing method in which an information processing system creates an execution plan for a mission including photographing an object from a moving body based on the results of a simulation of the mission performed by a user on an information processing terminal.

20. An information processing device comprising a mission control unit that creates an execution plan for a mission including photographing an object from a moving body based on the results of a simulation of the mission performed by a user on an information processing terminal.

Images