Information processing device, information processing method, and program

Abstract

To prevent invasion of privacy of a subject that may appear in a captured image when an aircraft takes an image during flight. [Solution] The information processing device 1 has an acquisition unit 131 that acquires captured images taken by an imaging unit equipped in the aircraft while the aircraft is in flight; a judgment unit 133 that determines whether a specified object is captured in the captured image while the aircraft is in flight; a decision unit 134 that determines an identification criterion that serves as a standard for whether a specific part of the object can be identified, and that indicates that the specific part can be identified from the captured image provided that the identification criterion is satisfied; and an aircraft control unit 132 that controls the aircraft so that the identification criterion is not satisfied, provided that the judgment unit has determined that an object is captured in the captured image.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program for processing information relating to the flight of an aircraft. [Background technology]

[0002] Patent Document 1 discloses a device that recognizes the characteristics of objects in an image captured by an aircraft, and performs a process to blur part or all of the object when an object to be protected is recognized. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-207636 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 describes a technology that acquires an image captured by an aircraft and then performs processing to blur a predetermined subject in the captured image. In the technology described in Patent Document 1, the aircraft continues to capture images of the subject, so there is a problem that if the captured image is leaked to the outside due to an accident, theft, or the like before processing of the captured image is performed, the privacy of the subject may be violated.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to prevent the violation of the privacy of subjects who may appear in captured images when an aircraft takes images while in flight. [Means for solving the problem]

[0006] An information processing device of a first aspect of the present invention comprises an acquisition unit that acquires an image captured by an imaging unit equipped in the aircraft while the aircraft is flying; a determination unit that determines whether a specified object is captured in the captured image while the aircraft is flying; a determination unit that determines an identification criterion that serves as a basis for whether a specific part of the object can be identified, and that indicates that the specific part can be identified from the captured image if the identification criterion is satisfied; and an aircraft control unit that controls the aircraft so that the identification criterion is not satisfied, if the determination unit determines that the object is captured in the captured image.

[0007] The determination unit may determine as the identification criterion that the distance between the object and the aircraft is equal to or less than a reference distance that indicates that the object can be identified from an image captured by the imaging unit.

[0008] The aircraft control unit may change a planned flight path, which is the path along which the aircraft will fly, so that the distance between the target object and the aircraft becomes longer than the reference distance.

[0009] The acquisition unit may acquire performance information indicating imaging performance of the imaging unit, and the determination unit may determine the reference distance in accordance with the imaging performance.

[0010] The aircraft control unit may change a planned flight path, which is the path along which the aircraft will fly, so that the identification criteria are not satisfied.

[0011] The aircraft control unit may change the planned flight route so that a difference between the planned flight route before the change and the planned flight route after the change is equal to or less than a predetermined reference value.

[0012] The aircraft control unit may change the planned flight path so that the distance or period of flight of the aircraft while the identification criteria are satisfied is below a predetermined standard value, provided that the planned flight path cannot be changed so that the identification criteria are not satisfied over the entire planned flight path.

[0013] The aircraft control unit may perform one of the following actions during a section of the planned flight path where the aircraft flies in a state where the identification criteria are satisfied: causing the imaging unit to stop capturing images, changing the direction in which the imaging unit captures images, or reducing the resolution at which the imaging unit captures images, provided that the planned flight path cannot be changed so that the identification criteria are not satisfied over the entire planned flight path.

[0014] The aircraft control unit may perform imaging control, which is one of causing the imaging unit to stop imaging, changing the direction in which the imaging unit captures images, and reducing the resolution at which the imaging unit captures images, in a section of the planned flight path along which the aircraft will fly and in which the identification criteria are satisfied.

[0015] The aircraft is equipped with a first imaging unit that captures a first image used to assist the flight of the aircraft, and a second imaging unit that captures a second image not used to assist the flight of the aircraft, and the aircraft control unit may perform the imaging control on the second imaging unit without performing the imaging control on the first imaging unit in a section of the planned flight route where the aircraft flies while the identification criteria are satisfied.

[0016] The determination unit may determine whether or not a target object is captured in the captured image by inputting the captured image into a determination model, which is a machine learning model that outputs information indicating whether or not a known target object is captured in the captured image when a known captured image is input.

[0017] The determination unit may determine as the identification criterion that the specific part can be identified by inputting the captured image into an identification model, which is a machine learning model that outputs a result of identifying a known specific part when a known captured image is input.

[0018] The determination unit may determine whether the object is moving based on the captured image, and the aircraft control unit may control the aircraft so that the direction in which the imaging unit captures the image is at a predetermined angle relative to the direction of the specific part of the object, on the condition that the determination unit determines that the object is not moving.

[0019] An information processing method of a second aspect of the present invention includes the steps of: acquiring an image captured by an imaging unit equipped in the aircraft while the aircraft is flying; determining whether a specified object is captured in the captured image while the aircraft is flying; determining an identification criterion that serves as a basis for whether a specific part of the object can be identified, and that indicates that the specific part can be identified from the captured image if the identification criterion is satisfied; and controlling the aircraft so that the identification criterion is not satisfied, if it is determined in the determining step that the object is captured in the captured image.

[0020] An information processing program of a third aspect of the present invention causes a processor to execute the following steps: acquiring an image captured by an imaging unit equipped in the aircraft while the aircraft is flying; determining whether a specified object is captured in the captured image while the aircraft is flying; determining an identification criterion that serves as a basis for whether a specific part of the object can be identified, and that indicates that the specific part can be identified from the captured image if the identification criterion is satisfied; and controlling the aircraft so that the identification criterion is not satisfied, if it is determined in the determining step that the object is captured in the captured image. [Effects of the Invention]

[0021] According to the present invention, it is possible to suppress invasion of the privacy of a subject that may appear in a captured image when an aircraft takes an image during flight. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 is a schematic diagram of a flight management system according to an embodiment. [Figure 2] FIG. 1 is a block diagram of an information processing apparatus according to an embodiment. [Figure 3] 3 is a schematic diagram for explaining the process by which the acquisition unit acquires flight plan information. FIG. [Figure 4] 4A and 4B are schematic diagrams for explaining a process in which an acquisition unit acquires a captured image. [Figure 5] 10A and 10B are schematic diagrams for explaining a process in which a determination unit determines whether or not a target object is captured in a captured image. [Figure 6] FIG. 10 is a schematic diagram for explaining a distance used as an identification criterion. [Figure 7] FIG. 10 is a schematic diagram for explaining a classification model used as a classification criterion. [Figure 8] 10 is a schematic diagram for explaining the process of changing the planned flight path and the process of controlling imaging by the aircraft control unit. FIG. [Figure 9] FIG. 1 is a flowchart illustrating an information processing method executed by an information processing apparatus according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0023] [Flight Management System S Overview] 1 is a schematic diagram of a flight management system S according to this embodiment. The flight management system S includes an information processing device 1, an aircraft 2, and an information terminal 3. The flight management system S may also include other terminals, devices, etc.

[0024] The information processing device 1 is a computer that processes information related to the flight of the aircraft 2. The information processing device 1 receives flight plan information from the information terminal 3, including a planned flight route, which is the route along which the aircraft 2 is planned to fly, and a planned flight period, which is the period during which the aircraft 2 is planned to fly. The information processing device 1 makes a predetermined determination on captured images captured by the aircraft 2 during flight, and performs a predetermined control on the aircraft 2 according to the determination result.

[0025] The aircraft 2 is a flying device that flies in the air along a planned flight path and captures images during flight. The aircraft 2 is, for example, an unmanned flying device such as a drone. The aircraft 2 is equipped with an imaging unit such as a camera for capturing images of a predetermined imaging range. The aircraft 2 transmits and receives information to and from the information processing device 1 via, for example, a wireless communication network.

[0026] The information terminal 3 is a computer used by a user. The information terminal 3 is, for example, a smartphone, a tablet terminal, a personal computer, etc. The user is, for example, a person who pilots, manages, or owns the aircraft 2. The information terminal 3 has a display unit such as an LCD display for displaying information, and an operation unit such as a touch panel for accepting operations by the user. The information terminal 3 accepts specifications of a planned flight route and planned flight period from the user via the operation unit. The information terminal 3 also displays information related to the flight of the aircraft 2 on the display. The information terminal 3 transmits and receives information to and from the information processing device 1, for example, via a wireless communication network.

[0027] An outline of the processing executed by the information processing device 1 according to this embodiment will be described below. The information processing device 1 acquires captured images captured by an imaging unit included in the flying object 2 while the flying object 2 is flying.

[0028] The information processing device 1 determines whether or not a predetermined object is captured in a captured image while the aircraft 2 is flying. The object is, for example, a person, a vehicle, or a building. The information processing device 1 determines whether or not the object is captured in the captured image by, for example, inputting the captured image into a predetermined machine learning model.

[0029] The information processing device 1 determines an identification criterion that serves as a criterion for determining whether a specific part of an object can be identified, and indicates that the specific part can be identified from the captured image if the identification criterion is satisfied. The specific part may be, for example, a human face, a license plate or seat of a vehicle, a window of a building, etc. The identification criterion is determined, for example, by the distance between the aircraft 2 and the object.

[0030] The information processing device 1 controls the flying object 2 so that the identification criterion is not satisfied, on the condition that it has determined that the target object is captured in the captured image. For example, the information processing device 1 changes the planned flight path of the flying object 2 so that the distance between the target object and the planned flight path is longer than a predetermined reference distance.

[0031] In this way, the information processing device 1 determines whether or not an object is captured in an image captured by the aircraft 2 while in flight, and if it determines that an object is captured in the captured image, controls the aircraft 2 so that a specific part of the object becomes unidentifiable. This allows the information processing device 1 to prevent violation of the privacy of a subject who may appear in a captured image when the aircraft 2 captures an image while in flight.

[0032] [Configuration of information processing device 1] FIG. 2 is a block diagram of an information processing device 1 according to this embodiment. In FIG. 2, arrows indicate main data flows, and data flows other than those shown in FIG. 2 may also exist. In FIG. 2, each block indicates a functional configuration rather than a hardware (device) configuration. Therefore, the blocks shown in FIG. 2 may be implemented in a single device, or may be implemented separately in multiple devices. Data may be exchanged between blocks via any means, such as a data bus, a network, or a portable storage medium.

[0033] The information processing device 1 includes a storage unit 11, a communication unit 12, and a control unit 13. The information processing device 1 may be configured by connecting two or more physically separate devices via wired or wireless connections. The information processing device 1 may also be configured by a cloud, which is a collection of computer resources.

[0034] The storage unit 11 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, an SSD (Solid State Drive), etc. The storage unit 11 stores in advance programs to be executed by the control unit 13. The storage unit 11 may be provided outside the information processing device 1, in which case data may be exchanged between the storage unit 11 and the control unit 13 via a network.

[0035] The communication unit 12 has a communication controller for transmitting and receiving data between the aircraft 2 and the information terminal 3 via the network. The communication unit 12 notifies the control unit 13 of data received from the aircraft 2 or the information terminal 3 via the network. The communication unit 12 also transmits data output from the control unit 13 to the aircraft 2 or the information terminal 3 via the network.

[0036] The control unit 13 has an acquisition unit 131, an aircraft control unit 132, a determination unit 133, and a decision unit 134. The control unit 13 is a processor such as a CPU (Central Processing Unit), and functions as the acquisition unit 131, the aircraft control unit 132, the determination unit 133, and the decision unit 134 by executing a program stored in the memory unit 11.

[0037] The following describes a configuration for executing processing according to this embodiment by the information processing device 1. Before the aircraft 2 starts flying, the acquisition unit 131 acquires flight plan information including a planned flight route, which is the route along which the aircraft 2 is planned to fly, and a planned flight period, which is the period during which the aircraft 2 is planned to fly.

[0038] 3 is a schematic diagram for explaining the process of the acquisition unit 131 acquiring flight plan information. The user specifies a planned flight route and a planned flight period, for example, using the operation unit of the information terminal 3. The planned flight route is represented, for example, by a combination of a route on a two-dimensional plane along which the aircraft 2 is planned to fly and an altitude along which the aircraft 2 is planned to fly. The planned flight route may also be represented by a route in three-dimensional space.

[0039] The planned flight period is expressed, for example, as a combination of the date and time when the aircraft 2 is scheduled to start flight and the date and time when the aircraft 2 is scheduled to end flight. The planned flight period may also include the date and time when the aircraft 2 is scheduled to arrive at one or more points along the planned flight route.

[0040] The information terminal 3 transmits flight plan information indicating the planned flight route and planned flight period specified by the user to the information processing device 1. In the information processing device 1, the acquisition unit 131 acquires the flight plan information transmitted by the information terminal 3 and stores the acquired flight plan information in the storage unit 11.

[0041] The aircraft control unit 132 transmits flight control information to the aircraft 2 for causing the aircraft 2 to fly along the planned flight path indicated by the flight plan information during the planned flight period indicated by the flight plan information. The aircraft 2 begins flying in accordance with the flight control information transmitted by the information processing device 1.

[0042] The acquisition unit 131 receives data transmitted by the flying object 2 and thereby acquires captured images captured by an imaging unit included in the flying object 2 during the flight of the flying object 2. Fig. 4 is a schematic diagram for explaining the process by which the acquisition unit 131 acquires captured images.

[0043] After starting flight in accordance with the flight control information, the flying object 2 generates captured images by capturing images using the imaging unit periodically or irregularly. During flight, the flying object 2 transmits data including the captured images to the information processing device 1. In the information processing device 1, the acquisition unit 131 sequentially acquires the captured images included in the data transmitted by the flying object 2.

[0044] The acquisition unit 131 may also acquire performance information indicating the imaging performance of the imaging unit included in the aircraft 2. The performance information is, for example, the number of pixels of the imaging element or the characteristics of the lens. The performance information is, for example, input in advance by a user and stored in advance in the storage unit 11 or another storage device. The acquisition unit 131 acquires the performance information from the storage unit 11 or another storage device.

[0045] The determination unit 133 determines whether a predetermined object is captured in the captured image acquired by the acquisition unit 131 while the flying object 2 is flying. The object may be, for example, a person, a vehicle, or a building. The object may also be an open-air bath, a swimming pool, or other object that should not be captured in the captured image.

[0046] 5 is a schematic diagram illustrating a process in which the determination unit 133 determines whether or not an object is captured in a captured image. The determination unit 133 acquires, for example, a determination model, which is a machine learning model used to determine whether or not an object is captured in a captured image. The determination model is configured to receive a known captured image and output information indicating whether or not a known object is captured in the captured image, as well as output information indicating the type of object (human, vehicle, building, etc.) captured in the captured image.

[0047] The determination model according to this embodiment is generated in advance by machine learning using known deep learning processing such as DNN (Deep Neural Network) using known captured images containing a specific type of object as training data, and is stored in the storage unit 11. The determination unit 133 acquires the determination model from the storage unit 11.

[0048] The determination unit 133 determines whether or not an object is captured in the captured image according to the information output by the determination model by inputting the captured image acquired by the acquisition unit 131 to the determination model. Furthermore, the determination unit 133 identifies the type of object (human, vehicle, building, etc.) captured in the captured image indicated by the information output by the determination model.

[0049] The determination unit 133 may determine whether or not a target object is captured in a captured image by other known processing, without using a machine learning model.

[0050] Furthermore, the determination unit 133 may determine whether or not the object is moving based on the captured images acquired by the acquisition unit 131. For example, the determination unit 133 identifies the position at which the object is captured in multiple captured images captured by the flying body 2 at multiple points in time, and determines that the object is moving if the positions at which the object is captured in the multiple captured images are different, and determines that the object is not moving if the positions at which the object is captured in the multiple captured images are the same.

[0051] The determination unit 134 determines an identification criterion that is used as a criterion for determining whether a specific part of an object can be identified, and indicates that the specific part can be identified from the captured image if the identification criterion is satisfied. The specific part differs depending on the type of object (human, vehicle, building, etc.). The specific part is, for example, a human face, a vehicle license plate or seat, a building window, an open-air bath or the entire pool, etc.

[0052] The determination unit 134 may determine the identification criteria before the aircraft 2 starts flying. Alternatively, the determination unit 134 may determine the identification criteria on the condition that the determination unit 133 determines that an object is captured in the captured image.

[0053] The determination unit 134 uses, as an identification criterion, either that the distance between the target object and the aircraft 2 satisfies a predetermined condition, or that an identification model, which is a machine learning model that identifies a specific part in a captured image, produces a predetermined output. Whether to use the distance or the identification model as the identification criterion is set in advance in the flight management system S or specified by the user.

[0054] 6 is a schematic diagram illustrating the distance used as the identification standard. The determination unit 134 determines a reference distance L1, which is a distance used as a standard for whether a specific portion of an object can be identified. The determination unit 134 determines, as the reference distance L1, the distance between the object and the aircraft 2, which indicates that a specific portion of the object can be identified from an image captured by the imaging unit of the aircraft 2.

[0055] The determination unit 134 determines that the identification criterion is that the distance L2 between the object and the aircraft is equal to or less than the determined reference distance L1. That is, when the distance L2 between the object and the aircraft is equal to or less than the reference distance L1, the identification criterion is met, and when the distance L2 between the object and the aircraft is longer than the reference distance L1, the identification criterion is not met.

[0056] The determination unit 134 may determine the reference distance L1, for example, according to the imaging performance of the imaging unit of the aircraft 2. In this case, the determination unit 134 acquires, for example, a table stored in advance in the storage unit 11, which associates known imaging performance with known distances indicating that an object can be identified from a captured image. The determination unit 134 determines, as the reference distance L1, the known distance associated with the imaging performance indicated by the performance information acquired by the acquisition unit 131 in the acquired table. This allows the flight management system S to flexibly set the reference distance L1 used as the identification standard according to the imaging performance of the imaging unit, for example, by increasing the reference distance L1 when the aircraft 2 is equipped with a high-performance imaging unit capable of capturing images at long distances.

[0057] The determination unit 134 may determine different reference distances L1 depending on the type of object (human, vehicle, building, etc.). In this case, the determination unit 134 may, for example, determine a first reference distance to be applied to a human object and a second reference distance to be applied to a vehicle object, the second reference distance being different from the first reference distance. The first reference distance to be applied to a human object is, for example, longer than the second reference distance to be applied to a vehicle. This allows the flight management system S to flexibly set the reference distance L1 used as an identification standard depending on the type of object, such as by increasing the reference distance when the object is a human object whose privacy is more important.

[0058] The determination unit 134 is not limited to the specific method shown here, and may determine the reference distance L1 using other methods.

[0059] 7 is a schematic diagram illustrating a classification model used as a classification standard. The determination unit 134 determines a classification model to be used as a classification standard from among a plurality of classification models pre-stored in the storage unit 11. The classification model is a machine learning model that receives a known captured image as input and outputs a result of identifying a known specific part. The classification model is different from the determination model used by the determination unit 133.

[0060] The classification model according to this embodiment is generated in advance by, for example, using a known deep learning process such as DNN to perform machine learning on a known captured image showing a specific part of a specific type of object as training data, and is stored in association with the type of the object in the storage unit 11. The determination unit 133 acquires a classification model associated with the type of object identified by the determination unit 133 from among a plurality of classification models corresponding to a plurality of object types stored in advance in the storage unit 11.

[0061] The determination unit 134 determines, on the condition that the determination unit 133 has determined that the target object is captured in the captured image, a section of the planned flight path acquired by the acquisition unit 131 along which the aircraft 2 will fly while the identification criteria are satisfied, as the target section. The determination unit 134 estimates the position of the target object, for example, based on the position at which the aircraft 2 captured the captured image determined by the determination unit 133 to contain the target object. For example, the determination unit 134 estimates, as the position of the target object, a position that is a predetermined distance along the optical axis of the lens of the imaging unit included in the aircraft 2 from the position of the aircraft 2 measured by a GNSS (Global Navigation Satellite System) or the like.

[0062] When the reference distance L1 is used as the identification criterion, the determination unit 134 determines, for example, a section of the planned flight route acquired by the acquisition unit 131 that is included in an area within the reference distance L1 from the position of the target object as the target section.

[0063] When an identification model is used as the identification criterion, the determination unit 134 inputs the captured image determined by the determination unit 133 to contain an object into the identification model, thereby acquiring an identification result output by the identification model. On the condition that the identification result indicates that a specific portion can be identified in the captured image, the determination unit 134 determines, for example, a section included in an area within a predetermined distance (100 m, 500 m, etc.) from the position of the object in the planned flight route acquired by the acquisition unit 131 as the target section.

[0064] The determination unit 134 may determine the target section by other methods, without being limited to the specific methods shown here.

[0065] The aircraft control unit 132 controls the aircraft 2 so that the identification criteria determined by the determination unit 134 (the above-mentioned reference distance L1 or identification model) are not satisfied, provided that the determination unit 133 determines that an object is captured in the captured image.

[0066] The aircraft control unit 132 either changes the planned flight path so that the identification criteria are not satisfied, or controls imaging so that the identification criteria are not satisfied. Whether to change the planned flight path or control imaging is set in advance in the flight management system S or specified by the user. The aircraft control unit 132 controls the aircraft 2 by transmitting information for changing the planned flight path or information for controlling imaging to the aircraft 2.

[0067] 8(a) is a schematic diagram for explaining the process of changing the planned flight path by the aircraft control unit 132. When the reference distance L1 is used as the identification criterion, the aircraft control unit 132 changes the planned flight path so that the distance L2 between the target object and the aircraft 2 is longer than the reference distance L1. For example, the aircraft control unit 132 changes the target section of the planned flight path identified by the determination unit 134 so that it is away from the target object, thereby changing the distance L2 so that it is longer than the reference distance L1.

[0068] When an identification model is used as the identification standard, the aircraft control unit 132 changes the planned flight path so that the distance L2 between the target object and the aircraft 2 is a distance at which a specific part cannot be identified from the captured image using the identification model. For example, the aircraft control unit 132 changes the target section of the planned flight path identified by the determination unit 134 so that it is away from the target object, thereby changing the distance L2 so that it is longer than a predetermined distance (100 m, 500 m, etc.).

[0069] This allows the flight management system S to protect the privacy of the target by changing the planned flight path, provided that the aircraft 2 approaches the target sufficiently to be able to identify specific parts of the target from the captured images during flight.

[0070] The aircraft control unit 132 may change the planned flight route so that the difference between the planned flight route before the change and the planned flight route after the change is equal to or less than a predetermined reference value. In this case, the aircraft control unit 132 calculates the difference in flight distance between the planned flight route before the change and the planned flight route after the change, and applies the changed planned flight route to the aircraft 2 if the calculated difference is equal to or less than the reference value, and applies the planned flight route before the change to the aircraft 2 if the calculated difference is greater than the reference value. In this way, the flight management system S can prevent the planned flight route from being changed significantly, thereby minimizing the impact on the flight plan of the aircraft 2.

[0071] The aircraft control unit 132 may change the planned flight route so that the distance or period of flight by the aircraft while the identification criteria are satisfied is equal to or less than a predetermined reference value, provided that the planned flight route cannot be changed so that the identification criteria are not satisfied over the entire planned flight route (for example, when the difference is greater than the reference value). In this case, the aircraft control unit 132 changes only a portion of the target section of the planned flight route identified by the determination unit 134, and does not change the remaining portion. This allows the flight management system S to protect the privacy of the target object while minimizing the impact on the flight plan of the aircraft 2.

[0072] The aircraft control unit 132 may perform image capture control, which will be described later using Fig. 8(b), for the target section of the planned flight route on the condition that the planned flight route cannot be changed so that the identification criteria are not satisfied over the entire planned flight route (for example, when the above-mentioned difference is greater than the reference value). This allows the flight management system S to perform image capture control to protect the privacy of the target object when the planned flight route cannot be changed.

[0073] 8(b) is a schematic diagram for explaining the process of image capture control performed by the aircraft control unit 132. The aircraft control unit 132 performs image capture control, which is one of causing the image capture unit to stop capturing images, changing the image capture direction of the image capture unit, and reducing the image capture resolution of the image capture unit, in the target section of the planned flight route identified by the determination unit 134. The content of the image capture control is set in advance in the flight management system S or specified by the user.

[0074] The aircraft control unit 132 generates flight control information that associates the target section with the content of imaging control, for example, and transmits it to the aircraft 2. While flying along the planned flight path, the aircraft 2 performs one of the following in accordance with the flight control information: stopping imaging by the imaging unit, changing the orientation in which the imaging unit captures images, or reducing the resolution at which the imaging unit captures images. Changing the orientation in which the imaging unit captures images is performed by changing the orientation of the aircraft 2 or by changing the orientation of a movable imaging unit.

[0075] This allows the flight management system S to protect the privacy of the target by controlling the imaging unit not to capture images of the target, provided that the aircraft 2 approaches the target close enough during flight that specific parts of the target can be identified from the captured image.

[0076] The aircraft 2 may include a first imaging unit that captures a first captured image (such as an image for collision avoidance) used to support the flight of the aircraft 2, and a second imaging unit that captures a second captured image (such as an image for investigation) that is not used to support the flight of the aircraft 2. In this case, the aircraft control unit 132 may perform the above-mentioned imaging control on the second imaging unit without performing the above-mentioned imaging control on the first imaging unit in the target section of the planned flight route identified by the determination unit 134.

[0077] As a result, the flight management system S does not perform imaging control on the imaging unit that assists the flight of the aircraft 2, thereby ensuring the safety of the flight of the aircraft 2 while protecting the privacy of the subject.

[0078] The air vehicle control unit 132 may control the air vehicle 2 so that the direction in which the imaging unit of the air vehicle 2 captures an image is at a predetermined angle relative to the direction of a specific part of the object, on the condition that the determination unit 133 determines that the object is not moving (such as a parked vehicle). In this case, the determination unit 134 estimates the direction of the specific part of the object (such as a vehicle seat) based on the captured image. The air vehicle control unit 132 generates flight control information for setting the direction in which the imaging unit of the air vehicle 2 captures an image at a predetermined angle (such as an angle at which the vehicle seat is captured from behind) relative to the estimated direction of the specific part, and transmits the information to the air vehicle 2.

[0079] While flying along the planned flight path, the flying vehicle 2 changes the image capturing direction of the imaging unit of the flying vehicle 2 in accordance with the flight control information. The image capturing direction of the imaging unit is changed by changing the orientation of the flying vehicle 2 or by changing the orientation of the movable imaging unit.

[0080] This allows the flight management system S to protect the privacy of the target object while continuing to capture images using the aircraft 2, for example, in cases where a specific part of the target object cannot be captured by capturing an image from a specified angle.

[0081] [Information processing method flow] 9 is a diagram showing a flowchart of an information processing method executed by the information processing device 1 according to this embodiment. The acquisition unit 131 acquires flight plan information indicating a planned flight route and planned flight period specified by the user from the information terminal 3, and stores the acquired flight plan information in the storage unit 11 (S11). The aircraft control unit 132 transmits flight control information to the aircraft 2 for causing the aircraft 2 to fly along the planned flight route indicated by the flight plan information during the planned flight period indicated by the flight plan information. The aircraft 2 begins flying in accordance with the flight control information transmitted by the information processing device 1.

[0082] After starting flight in accordance with the flight control information, the aircraft 2 generates captured images by capturing images using the imaging unit periodically or irregularly. During flight, the aircraft 2 transmits data including the captured images to the information processing device 1. In the information processing device 1, the acquisition unit 131 acquires the captured images included in the data transmitted by the aircraft 2 (S12).

[0083] The determination unit 133 determines whether or not a predetermined object is captured in the captured image acquired by the acquisition unit 131 while the flying object 2 is flying (S13).

[0084] The determination unit 134 determines an identification criterion that is used as a criterion for whether a specific portion of an object can be identified, and indicates that the specific portion can be identified from the captured image on the condition that the identification criterion is satisfied (S14). The determination unit 134 uses as the identification criterion either that the distance between the object and the aircraft 2 satisfies a predetermined condition, or that an identification model, which is a machine learning model that identifies the specific portion in the captured image, produces a predetermined output.

[0085] If the determination unit 133 determines that the captured image does not include an object (NO in S15), the information processing device 1 proceeds to step S18.

[0086] If the judgment unit 133 determines that an object is captured in the captured image (YES in S15), the determination unit 134 determines the section of the planned flight path acquired by the acquisition unit 131 along which the aircraft 2 will fly while satisfying the identification criteria as the target section (S16).

[0087] The aircraft control unit 132 controls the aircraft 2 so that the identification criteria determined by the determination unit 134 are not satisfied (S17). The aircraft control unit 132 either changes the planned flight path so that the identification criteria are not satisfied, or controls imaging so that the identification criteria are not satisfied.

[0088] If a predetermined termination condition (such as when the aircraft 2 has finished flying) is satisfied (YES in S18), the information processing device 1 terminates the processing. If the predetermined termination condition is not satisfied (NO in S18), the information processing device 1 returns to step S12 and continues the processing.

[0089] [Effects of the embodiment] According to the flight management system S of this embodiment, the information processing device 1 determines whether or not an object is captured in an image captured by the aircraft 2 while in flight, and if it determines that an object is captured in the captured image, controls the aircraft 2 so that a specific part of the object becomes unidentifiable. This allows the information processing device 1 to prevent the privacy of a subject who may appear in the captured image when the aircraft 2 captures images while in flight from being violated.

[0090] <Modification> In the above-described embodiment, the information processing device 1 mainly performs the processing, but at least part of the above-described processing may be performed by the aircraft 2. In this case, the processor of the aircraft 2 executes a program pre-stored in a storage unit provided in the aircraft 2, thereby functioning as at least part of the acquisition unit 131, aircraft control unit 132, determination unit 133, and decision unit 134.

[0091] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and resilience."

[0092] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments. [Explanation of symbols]

[0093] S Flight Management System 1. Information processing equipment 11 Storage section 12 Communications Department 13 Control Unit 131 Acquisition Department 132 Aircraft control unit 133 Judgment section 134 Decision Section 2. Aircraft 3. Information terminals

Claims

1. an acquisition unit that acquires images captured by an imaging unit included in the aircraft during flight of the aircraft; a determination unit that determines whether a predetermined object is captured in the captured image while the flying object is flying; a determination unit that determines an identification criterion that is used as a criterion for whether a specific portion of the object can be identified, the identification criterion indicating that the specific portion can be identified from the captured image on condition that the identification criterion is satisfied; an aircraft control unit that controls the aircraft so that the identification criterion is not satisfied, on the condition that the determination unit determines that the object is captured in the captured image; An information processing device having the above.

2. The determination unit determines, as the identification criterion, that the distance between the object and the aircraft is equal to or shorter than a reference distance indicating that the object can be identified from an image captured by the imaging unit. The information processing device according to claim 1 .

3. the aircraft control unit changes a planned flight path, which is a path along which the aircraft will fly, so that the distance between the object and the aircraft becomes longer than the reference distance. The information processing device according to claim 2 .

4. the acquisition unit acquires performance information indicating imaging performance of the imaging unit; the determination unit determines the reference distance in accordance with the imaging performance. The information processing device according to claim 2 .

5. the aircraft control unit changes a planned flight path, which is a path along which the aircraft will fly, so that the identification criterion is not satisfied. The information processing device according to claim 1 .

6. The aircraft control unit changes the planned flight route so that a difference between the planned flight route before the change and the planned flight route after the change is equal to or less than a predetermined reference value. The information processing device according to claim 5 .

7. the aircraft control unit changes the planned flight route so that the distance or period of flight of the aircraft in a state where the identification criteria are satisfied is equal to or less than a predetermined reference value, on the condition that the planned flight route cannot be changed so that the identification criteria are not satisfied over the entire planned flight route; The information processing device according to claim 5 .

8. The aircraft control unit performs one of the following actions in a section of the planned flight route in which the aircraft flies while the identification criteria are satisfied: causing the imaging unit to stop capturing images; changing the direction in which the imaging unit captures images; and reducing the resolution at which the imaging unit captures images, on the condition that the planned flight route cannot be changed so that the identification criteria are not satisfied over the entire planned flight route. The information processing device according to claim 5 .

9. The aircraft control unit performs imaging control, which is one of stopping imaging by the imaging unit, changing the imaging direction of the imaging unit, and reducing the resolution of imaging by the imaging unit, in a section of the planned flight path along which the aircraft flies while the identification criteria are satisfied, without changing the planned flight path, which is the path along which the aircraft flies. The information processing device according to claim 1 .

10. The aircraft includes a first imaging unit that captures a first captured image used to support the flight of the aircraft, and a second imaging unit that captures a second captured image that is not used to support the flight of the aircraft, The aircraft control unit performs the imaging control on the second imaging unit without performing the imaging control on the first imaging unit in a section of the planned flight route in which the aircraft flies while the identification criterion is satisfied. The information processing device according to claim 9 .

11. the determination unit determines whether or not a target object is captured in a captured image by inputting the captured image to a determination model, which is a machine learning model that outputs information indicating whether or not a known target object is captured in the captured image when the captured image is input; The information processing device according to claim 1 .

12. the determination unit determines, as the identification criterion, that the specific part can be identified by inputting the captured image into an identification model, which is a machine learning model that outputs a result of identifying a known specific part when a known captured image is input, The information processing device according to claim 11.

13. the determination unit determines whether the object is moving based on the captured image; the aircraft control unit controls the aircraft so that the direction in which the imaging unit captures an image is at a predetermined angle with respect to the direction of the specific part of the object, on the condition that the determination unit determines that the object is not moving. The information processing device according to claim 1 .

14. The processor executes acquiring an image captured by an imaging unit included in the aircraft during flight of the aircraft; determining whether a predetermined object is captured in the captured image during flight of the aircraft; determining an identification criterion as a criterion for determining whether a specific portion of the object can be identified, the identification criterion indicating that the specific portion can be identified from the captured image on condition that the identification criterion is satisfied; a step of controlling the aircraft so that the identification criterion is not satisfied, on the condition that it is determined in the determining step that the object is captured in the captured image; An information processing method comprising:

15. The processor acquiring an image captured by an imaging unit included in the aircraft during flight of the aircraft; determining whether a predetermined object is captured in the captured image during flight of the aircraft; determining an identification criterion as a criterion for determining whether a specific portion of the object can be identified, the identification criterion indicating that the specific portion can be identified from the captured image on condition that the identification criterion is satisfied; a step of controlling the aircraft so that the identification criterion is not satisfied, on the condition that it is determined in the determining step that the object is captured in the captured image; A program that executes.

Images