Model generation device, route proposal device, model generation method, and program

Abstract

In this model generation system, a proposed path generation means generates and outputs a proposed path that indicates a movement path of a mobile object. An evaluation acquisition means acquires a user evaluation of the proposed path. A training data generation means uses the acquired user evaluation to generate training data. A model generation means uses the training data to generate a path evaluation model that indicates the relationship between the movement path and the user evaluation.

Description

[Technical Field]

[0001] The present disclosure relates to a method for evaluating a path of a moving object. [Background technology]

[0002] In recent years, the use of drones for various purposes has been considered. Drone operation methods include a method called IFR (Instrument Flight Rules), in which the drone is flown under instructions from an air traffic controller, and a method called VFR (Visual Flight Rules), in which the pilot flies the drone while visually recognizing other aircraft. Patent Document 1 describes a device that sets the flight path of an unmanned aircraft based on a departure point and a destination point. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-39875 Summary of the Invention [Problem to be solved by the invention]

[0004] While VFR pilots are encouraged to register flight plans, it is not possible to register flight plans while piloting a drone. Furthermore, in VFR, the drone is essentially flown at the pilot's discretion, but it is desirable for an external system to assist the pilot in suggesting flight routes and other information to enhance flight safety. In this case, it is desirable for the external system to be able to suggest flight routes and other information suitable for the pilot based on the pilot's piloting characteristics.

[0005] One object of the present disclosure is to enable evaluation of a travel path according to the characteristics of the operator of a moving object. [Means for solving the problem]

[0006] In one aspect of the present disclosure, a model generating device includes: a proposed route generating means for generating and outputting a proposed route indicating a travel route of the moving object; evaluation acquisition means for acquiring a user's evaluation of the proposed route; a training data generating means for determining whether the evaluation is positive or negative and setting the result as an evaluation value, and generating training data using the evaluation value as correct answer data; and a model generation means for generating a route evaluation model that indicates a relationship between a travel route and a user's evaluation, using the training data. 、 The training data generation means assigns the highest evaluation value when the user gives a positive evaluation to the proposed route once, and assigns a lower evaluation value when the user repeatedly gives a negative evaluation to the proposed route the more times the user repeats the negative evaluation. do.

[0007] In another aspect of the present disclosure, a model generation method includes: The computer Generate and output a proposed route indicating a route of travel of the mobile object; obtaining a user's evaluation of the proposed route; determining the degree to which the evaluation is positive or negative and setting the result as an evaluation value; When the user gives a positive evaluation to the proposed route once, the evaluation value is set to the maximum, and when the user gives a negative evaluation repeatedly, the more times the user gives a negative evaluation, the lower the evaluation value is assigned. generating training data using the evaluation value as correct answer data; The training data is used to generate a route evaluation model that indicates the relationship between travel routes and user evaluations.

[0008] In yet another aspect of the disclosure, a program includes: Generate and output a proposed route indicating a route of travel of the mobile object; obtaining a user's evaluation of the proposed route; determining the degree to which the evaluation is positive or negative and setting the result as an evaluation value; When the user gives a positive evaluation to the proposed route once, the evaluation value is set to the maximum, and when the user gives a negative evaluation repeatedly, the more times the user gives a negative evaluation, the lower the evaluation value is assigned. generating training data using the evaluation value as correct answer data; Using the training data, a computer is caused to execute a process of generating a route evaluation model that indicates the relationship between travel routes and user evaluations. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to evaluate a travel path according to the characteristics of the operator of a moving object. [Brief explanation of the drawings]

[0010] [Figure 1] 1 shows the configuration of a drone flight system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of the control device. [Figure 3] FIG. 2 is a block diagram showing a hardware configuration of the route suggestion device. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of a control device and a route proposal device. [Figure 5] 10 is an example of a control screen. [Figure 6] 10 is a flowchart of a training process for a route evaluation model performed by the route suggestion device. [Figure 7] 10 is a flowchart of a route suggestion process using a trained route evaluation model. [Figure 8] FIG. 10 is a block diagram showing the functional configuration of a model generation device according to a second embodiment. [Figure 9] 10 is a flowchart of processing by a model generating device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings. First Embodiment [Overall configuration] 1 shows the configuration of a drone flight system according to the first embodiment. The drone flight system is a system in which a pilot flies a drone 1 to a destination, and includes a control device 100 and a route proposal device 200.

[0012] The control device 100 is operated by a pilot and flies the drone 1 by wirelessly transmitting control signals to the drone 1. The drone 1 is equipped with a camera 2. The drone 2 captures images of at least the direction in which the drone 1 is traveling with the camera 2 and wirelessly transmits the captured images to the control device 100. The pilot controls the drone 1 while viewing a control screen displayed based on the captured images of the direction in which the drone 1 is traveling.

[0013] The route proposal device 200 proposes a flight route for the drone 1. Specifically, the route proposal device 200 creates a route that the drone 1 should fly (hereinafter also referred to as a "proposed route") based on the flight plan of the drone 1, and transmits it to the control device 100. The pilot operating the control device 100 pilots the drone 1 while taking into consideration the proposed route received from the route proposal device 200, and also evaluates the proposed route. The evaluation entered by the pilot for the proposed route is transmitted to the route proposal device 200. The route proposal device 200 acquires the pilot's evaluation of the proposed route and learns the relationship between the proposed route and the pilot's evaluation of it, thereby becoming able to propose a route that is suitable for that pilot.

[0014] [Hardware configuration] (Control device) 2 is a block diagram showing the hardware configuration of the control device 100. As shown in the figure, the control device 100 includes a communication unit 111, a processor 112, a memory 113, a recording medium 114, a database (DB) 115, a display unit 116, and an input unit 117.

[0015] The communication unit 111 transmits and receives data to and from external devices. Specifically, the control device 100 transmits control signals for flying the drone 1 to the drone 1 through the communication unit 111, and receives images captured by the camera 2 from the drone 1. The control device 100 also receives a proposed route from the route proposal device 200 through the communication unit 111, and transmits the pilot's evaluation of the proposed route to the route proposal device 200.

[0016] The processor 112 is a computer such as a CPU (Central Processing Unit), and executes a pre-prepared program to control the entire controller 100. The processor 112 may be a GPU (Graphics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, or an FPGA (Field-Programmable Gate Array).

[0017] The memory 113 is configured by a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 113 is also used as a working memory while the processor 112 is executing various processes.

[0018] The recording medium 114 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from the control device 100. The recording medium 114 records various programs to be executed by the processor 112. When the control device 100 executes various processes, the programs recorded on the recording medium 114 are loaded into the memory 113 and executed by the processor 112.

[0019] DB 115 stores data used by and generated by the control device 100. Specifically, DB 115 stores images captured by camera 2 transmitted from drone 1, proposed routes transmitted from route proposal device 200, and the like. DB 115 also stores evaluations entered by the pilot regarding the proposed routes. Furthermore, DB 115 stores various types of information required for the pilot to remotely control drone 1 using the control device 100.

[0020] The display unit 116 is, for example, a liquid crystal display device, and displays to the pilot captured images transmitted from the drone 1. The display unit 116 also displays information about the proposed route transmitted from the route proposal device 200. The input unit 117 is, for example, an input device such as a joystick or various buttons, or an audio input device including a microphone, and is used by the pilot to give instructions and inputs required while operating the drone 1.

[0021] (Route suggestion device) 3 is a block diagram showing the hardware configuration of the route proposal device 200. As shown in the figure, the route proposal device 200 includes a communication unit 211, a processor 212, a memory 213, a recording medium 214, a DB 215, a display unit 216, and an input unit 217.

[0022] The communication unit 211 transmits and receives data to and from external devices. Specifically, the proposed route transmitted to the control device 100 and the pilot evaluation transmitted from the control device 100 are transmitted and received via the communication unit 211.

[0023] The processor 212 is a computer such as a CPU, and executes a prepared program to control the entire route proposal device 200. The processor 212 may be a GPU, a TPU, a quantum processor, or an FPGA. As will be described later, the processor 212 executes a training process for a route evaluation model and a route proposal process using the route evaluation model.

[0024] The memory 213 is configured by a ROM, a RAM, etc. The memory 213 is also used as a working memory while the processor 212 is executing various processes.

[0025] The recording medium 214 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the route proposal device 200. The recording medium 214 records various programs to be executed by the processor 212. When the route proposal device 200 executes various processes, the programs recorded on the recording medium 214 are loaded into the memory 213 and executed by the processor 212.

[0026] DB215 stores data used by and generated by route proposal device 200. Specifically, DB215 stores a plurality of proposed routes generated by route proposal device 200, pilot evaluations of the proposed routes transmitted from control device 100, and the like. DB215 also stores training data for generating a route evaluation model, the generated route evaluation model, and the like.

[0027] The display unit 216 is, for example, a liquid crystal display device, and displays the generated proposed route, etc., as necessary. The input unit 117 is, for example, an input device such as a keyboard or a mouse, and is used by an operator to give necessary instructions and inputs in the training process for the route evaluation model and the route proposal process.

[0028] [Function Configuration] Next, we will explain the functional configuration of the control device 100 and the route proposal device 200. Fig. 4 is a block diagram showing the functional configuration of the control device 100 and the route proposal device 200. As shown in the figure, the control device 100 includes a display control unit 121 and an evaluation transmission unit 122 in addition to the display unit 116 and input unit 117 described above.

[0029] On the other hand, the route proposal device 200 includes a route proposal unit 221, a route transmission unit 222, a training data generation unit 223, and a model training unit 224. The route proposal unit 221 and the route transmission unit 222 are examples of proposed route generation means, the training data generation unit 223 is an example of evaluation acquisition means and training data generation means, and the model training unit 224 is an example of model generation means.

[0030] First, the control device 100 will be described. The display control unit 121 controls the display on the display unit 116. Specifically, the display control unit 121 generates a control screen using captured images transmitted from the drone 1 and displays it on the display unit 116. FIG. 5 is an example of the control screen. In the example of FIG. 5, the control screen is an image that simulates the view from the cockpit. The captured images received from the drone 1 are displayed in AR (Augmented Reality) on the control screen. Since the control screen is generated based on the captured images from the drone 1, it is updated as the drone 1 moves.

[0031] Furthermore, information about the proposed route transmitted from the route proposal device 200 is superimposed on the control screen. In the example of FIG. 5, information (arrow) 31 indicating the movement direction of the drone 1 along the proposed route is displayed. Furthermore, a message 32 requesting the pilot's evaluation of the proposed route is transmitted from the route proposal device 200 as necessary and displayed on the control screen. The pilot views the information 31 about the movement direction along the proposed route and the message 32 to understand the route proposed by the route proposal device 200 and inputs an evaluation of the proposed route. In the example of FIG. 5, the pilot responds by voice input, "More to the right," to the movement direction information 31. Note that for convenience of explanation, FIG. 5 illustrates the pilot's response "More to the right," but in reality, the pilot's response does not need to be displayed on the control screen. Furthermore, the pilot may use the joystick or the like described above instead of voice input to input an instruction to correct the movement direction as an evaluation of the proposed route.

[0032] The input unit 117 acquires the pilot's evaluation of the guided route, which is input using voice input or a joystick, and outputs the evaluation to the evaluation transmission unit 122. The evaluation transmission unit 122 transmits the pilot's evaluation to the route proposal device 200. Specifically, when the pilot inputs the evaluation using voice input, the evaluation transmission unit 122 may transmit the input voice directly to the route proposal device 200, or may analyze the content of the voice and transmit it to the route proposal device 200 as information indicating that "correction to the right is necessary." Furthermore, when the pilot inputs the evaluation using a joystick, the evaluation transmission unit 122 transmits the joystick operation direction, the operation amount, or the operation duration, etc. to the route proposal device 200. In this way, the pilot's evaluation of the proposed route is sent to the route proposal device 200.

[0033] Next, the route proposal device 200 will be described. The route proposal unit 221 generates a proposed route based on a flight plan specified by a pilot or the like. The flight plan includes at least the drone's departure point and destination, and may further include aircraft information about the drone. The route proposal unit 221 generates, for example, a flight route that connects the departure point and destination on a map in a straight line, or a route that avoids high-rise buildings and mountainous areas by taking into account topographical information. Note that the route proposal unit 221 may generate the proposed route by taking into account environmental information, such as weather information for the planned flight area at the time of the drone's flight and topographical information for the planned flight area, in addition to the drone's aircraft information. The route proposal unit 221 may also generate the proposed route using a route generation model based on artificial intelligence (AI) that has been prepared in advance. The route proposal unit 221 outputs the generated proposed route to the route transmission unit 222.

[0034] The route transmission unit 222 transmits the proposed route generated by the route proposal unit 221 to the control device 100. In the control device 100, as described above, the display control unit 121 receives the proposed route and displays information 31 on the direction of travel along the proposed route and the like on the control screen.

[0035] The training data generator 223 receives pilot evaluations transmitted from the pilot control device 100 and generates training data for training the route evaluation model. The route evaluation model evaluates proposed routes taking into account the characteristics of each pilot, and is a model that learns the relationship between the proposed route presented to the pilot and the pilot's evaluation of the proposed route. In other words, when a proposed route is input, the route evaluation model generated for a certain pilot A predicts and outputs pilot A's evaluation of the proposed route.

[0036] The training data is data that uses a proposed route as input data and the pilot's evaluation of the proposed route as ground truth data. Here, the pilot's evaluation is a value obtained by quantifying the evaluation input by the pilot to the control device 100 using some method. For example, the training data generation unit 223 may assign a maximum evaluation value when the pilot gives a single positive evaluation of the proposed route, and assign a lower evaluation value the more negative evaluations the pilot gives. Specifically, the training data generation unit 223 may set the evaluation value based on the number of negative evaluations (instructions for correction) made by the pilot via voice input or the number of times instructions for correction are input using a joystick. Alternatively, when the pilot inputs an evaluation via voice, the training data generation unit 223 may analyze the content of the evaluation and determine the degree to which the evaluation of the proposed route is positive, and use this as the evaluation value. The training data generation unit 223 outputs the generated training data to the model training unit 224.

[0037] The model training unit 224 trains the route evaluation model using the training data to generate a route evaluation model. The route evaluation model thus obtained outputs an evaluation of the input proposed route that is close to the perception of each pilot.

[0038] [Training process] 6 is a flowchart of a process for training a route evaluation model by the route proposal device 200. This process is realized by the processor 212 shown in FIG. 3 executing a prepared program and operating as a component of the route proposal device 200 shown in FIG.

[0039] First, the route proposal unit 221 receives a flight plan for the drone (step S10). The flight plan includes at least the drone's departure point and destination. The route proposal unit 221 generates a proposed route for the drone to the destination based on the flight plan and transmits it to the control device 100 (step S11). The proposed route is presented to the pilot on the control device 100, and the pilot inputs an evaluation of the proposed route. The pilot's evaluation is transmitted from the control device 100 to the route proposal device 200.

[0040] Upon receiving the pilot's evaluation from the control device 100 (step S12), the training data generation unit 223 digitizes the pilot's evaluation and generates training data using the obtained evaluation value as ground truth data (step S13). The training data is output to the model training unit 224. Next, the model training unit 224 determines whether a sufficient amount of training data has been generated (step S14). If a sufficient amount of training data has not been generated (step S14: No), the process returns to step S11, and steps S11 to S14 are repeated until a sufficient amount of training data has been generated. On the other hand, if a sufficient amount of training data has been generated (step S14: Yes), the model training unit 224 trains the route evaluation model using the training data and generates a route evaluation model (step S15). Then, the training process ends. Note that the training process is not limited to when generating a route evaluation model for the first time, and may be performed periodically as retraining of an existing route evaluation model.

[0041] [Route proposal processing] Next, an example of a method for proposing a route to a pilot using a route evaluation model generated by training will be described. Fig. 7 is a flowchart of a route proposal process using a trained route evaluation model. This process is realized by the processor 212 shown in Fig. 3 executing a prepared program and operating mainly as the route proposal unit 221 shown in Fig. 4.

[0042] First, the route proposal unit 221 receives a flight plan for the drone (step S21). Next, the route proposal unit 221 generates multiple proposed routes to the destination based on the flight plan (step S22). Next, the route proposal unit 221 evaluates the generated multiple proposed routes using a trained route evaluation model (step S23). Specifically, the route proposal unit 221 inputs the generated multiple proposed routes into the trained route evaluation model and obtains an evaluation value for each. Then, the route proposal unit 221 transmits one or more proposed routes with high evaluation values ​​output by the route evaluation model from among the multiple proposed routes to the pilot device 100 via the route transmission unit 222. This enables the route proposal device 200 to propose routes for the flight plan that are more suited to the characteristics of each pilot.

[0043] [Variations] In the above embodiment, the present disclosure is applied to proposing a movement route for a drone, but the application of the present disclosure is not limited thereto. The present disclosure can be similarly applied to various moving objects that move along a predetermined route, such as automobiles, automated guided vehicles used in factories, warehouses, etc.

[0044] Second Embodiment 8 is a block diagram showing the functional configuration of a model generation device 70 according to the second embodiment. The model generation device 70 includes a proposed route generation means 71, an evaluation acquisition means 72, a training data generation means 73, and a model generation means 74.

[0045] 9 is a flowchart of processing by the model generation device 70 of the second embodiment. The proposed route generation means 71 generates and outputs a proposed route indicating a travel route of a mobile object (step S71). The evaluation acquisition means 72 acquires a user's evaluation of the proposed route (step S72). The training data generation means 73 generates training data using the acquired user evaluation (step S73). The model generation means 74 uses the training data to generate a route evaluation model indicating the relationship between the travel route and the user evaluation (step S74).

[0046] According to the model generation device 70 of the second embodiment, a route evaluation model capable of evaluating a proposed route taking into account the characteristics of the pilot is generated.

[0047] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0048] (Appendix 1) a proposed route generating means for generating and outputting a proposed route indicating a travel route of the moving object; evaluation acquisition means for acquiring a user's evaluation of the proposed route; training data generation means for generating training data using the acquired user ratings; a model generation means for generating a route evaluation model indicating a relationship between a travel route and a user's evaluation using the training data; A model generation device comprising:

[0049] (Appendix 2) The model generation device according to claim 1, wherein the proposed path generation means outputs information indicating the direction of movement of the moving body relative to the current position of the moving body to a control device used by the user to control the moving body.

[0050] (Appendix 3) 3. The model generation device according to claim 2, wherein the evaluation acquisition means acquires the user's evaluation of the moving direction of the moving object from the control device.

[0051] (Appendix 4) 2. The model generation device according to claim 1, wherein the training data generation means generates the training data by quantifying the acquired user evaluations.

[0052] (Appendix 5) a flight plan acquisition means for acquiring a flight plan including at least a departure point and a destination point of the moving object from the user; a proposed route generation means for generating a plurality of proposed routes based on the flight plan; evaluation result output means for evaluating the plurality of routes using the route evaluation model generated by the model generation device according to Supplementary Note 1 and outputting evaluation results; an output means for selecting and outputting a proposed route based on the evaluation result; A route suggestion device comprising:

[0053] (Appendix 6) Generate and output a proposed route indicating a route of travel of the mobile object; obtaining a user's evaluation of the proposed route; Generate training data using the obtained user ratings; A model generation method that uses the training data to generate a route evaluation model that indicates the relationship between travel routes and user evaluations.

[0054] (Appendix 7) Generate and output a proposed route indicating a route of travel of the mobile object; obtaining a user's evaluation of the proposed route; Generate training data using the obtained user ratings; A recording medium storing a program for causing a computer to execute a process of generating a route evaluation model showing the relationship between travel routes and user evaluations using the training data.

[0055] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. [Explanation of symbols]

[0056] 1. Drone 2 Cameras 112, 212 processors 121 Display control unit 122 Evaluation Transmission Unit 221 Route Suggestion Unit 222 Route Transmission Unit 223 Training Data Generation Unit 224 Model Training Department 100 Controls 200 Route suggestion device

Claims

1. a proposed route generating means for generating and outputting a proposed route indicating a travel route of the moving object; evaluation acquisition means for acquiring a user's evaluation of the proposed route; a training data generating means for determining whether the evaluation is positive or negative and setting the result as an evaluation value, and generating training data using the evaluation value as correct answer data; a model generation means for generating a route evaluation model indicating a relationship between a travel route and a user's evaluation using the training data; The training data generation means is a model generation device that assigns the highest evaluation value to the proposed route if the user gives a positive evaluation the first time, and assigns a lower evaluation value the more times the user repeatedly gives negative evaluations.

2. The model generation device according to claim 1 , wherein the proposed path generation means outputs information indicating the direction of movement of the moving body relative to the current position of the moving body to a control device used by the user to control the moving body.

3. The model generating device according to claim 2 , wherein the evaluation acquisition means acquires the user's evaluation of the moving direction of the moving object from the control device.

4. 2. The model generating device according to claim 1, wherein the training data generating means generates the training data by quantifying the acquired user evaluations.

5. a flight plan acquisition means for acquiring a flight plan including at least a departure point and a destination point of the moving object from the user; a proposed route generation means for generating a plurality of proposed routes based on the flight plan; an evaluation result output means for evaluating a plurality of routes using the route evaluation model generated by the model generation device according to claim 1 and outputting evaluation results; an output means for selecting and outputting a proposed route based on the evaluation result; A route suggestion device comprising:

6. The computer Generate and output a proposed route indicating a route of travel of the mobile object; obtaining a user's evaluation of the proposed route; determining the degree to which the evaluation is positive or negative and setting the evaluation value as an evaluation value; if the user gives a positive evaluation for the proposed route once, setting the evaluation value as the maximum; if the user gives a negative evaluation repeatedly, setting the evaluation value as the number of repetitions is increased, and generating training data using the evaluation value as correct answer data; A model generation method that uses the training data to generate a route evaluation model that indicates the relationship between travel routes and user evaluations.

7. Generate and output a proposed route indicating a route of travel of the mobile object; obtaining a user's evaluation of the proposed route; determining the degree to which the evaluation is positive or negative and setting the evaluation value as an evaluation value; if the user gives a positive evaluation for the proposed route once, setting the evaluation value as the maximum; if the user gives a negative evaluation repeatedly, setting the evaluation value as the number of repetitions is increased, and generating training data using the evaluation value as correct answer data; A program that causes a computer to execute a process of generating a route evaluation model that indicates the relationship between travel routes and user evaluations using the training data.

Images