Unmanned aerial vehicle, method for controlling unmanned aerial vehicle, program, control system, and control device

Abstract

[Problem] To provide an unmanned aerial vehicle, a method for controlling an unmanned aerial vehicle, a program, a control system, and a control device with which it is possible to detect an unexpected object during flight and perform appropriate emergency control. [Solution] An unmanned aerial vehicle according to one embodiment of the present invention comprises: an imaging device that is provided to the unmanned aerial vehicle and generates a captured image; an object detection unit that detects an object from the captured image; a determination unit that determines whether the object detected by the object detection unit satisfies a predetermined condition; and a flight control unit that, when the determination unit determines that the condition is satisfied, causes the own vehicle body to execute emergency control.

Description

Unmanned aerial vehicle, unmanned aerial vehicle control method, program, control system, and control device

[0001] The present invention relates to an unmanned aerial vehicle, a control method for an unmanned aerial vehicle, a program, a control system, and a control device.

[0002] In recent years, autonomously controllable mobile objects, such as air vehicles such as drones and unmanned aerial vehicles (UAVs) and running objects such as unmanned ground vehicles (UGVs), have begun to be used in industry. Patent Document 1 discloses delivery of deliveries by air vehicles.

[0003] Japanese Patent Application Laid-Open No. 2021-160887

[0004] However, conventional unmanned aerial vehicles (UAVs) require a system that can detect unexpected targets during flight and take appropriate emergency control measures. In particular, realizing real-time target detection and rapid flight control based on the detection results has been a technical challenge.

[0005] Therefore, the present invention has been made in consideration of this background, and aims to provide an unmanned aerial vehicle, a control method for an unmanned aerial vehicle, a program, a control system, and a control device that are capable of detecting unexpected objects during flight and performing appropriate emergency control.

[0006] In order to achieve the above-mentioned object, one embodiment of the unmanned aerial vehicle of the present invention comprises an imaging device provided on the unmanned aerial vehicle and generating an image, an object detection unit that detects an object from the image, a judgment unit that determines whether the object detected by the object detection unit satisfies predetermined conditions, and a flight control unit that executes emergency control on the aircraft if the judgment unit determines that the conditions are satisfied.

[0007] According to the present invention, it is possible to provide an unmanned aerial vehicle, a control method for an unmanned aerial vehicle, a program, a control system, and a control device that are capable of detecting unexpected objects during flight and performing appropriate emergency control.

[0008] 1 is a diagram showing an overview of one embodiment of the present invention. FIG. 2 is a diagram showing the configuration of a control device according to one embodiment of the present invention. FIG. 3 is a block diagram showing the functions of the control unit of FIG. 2. FIG. 4 is a block diagram showing the structure of the storage of FIG. 2. FIG. 5 is a diagram showing an example of processing according to one embodiment of the present invention. FIG. 6 is a schematic diagram showing a modified example of this embodiment.

[0009] The details of the embodiments of the present invention will be listed below. An unmanned aerial vehicle or the like according to the embodiments of the present invention has the following configuration. [Item 1] An unmanned aerial vehicle comprising: an imaging device provided to the unmanned aerial vehicle and generating an image; an object detection unit that detects an object from the image; a determination unit that determines whether the object detected by the object detection unit satisfies a predetermined condition; and a flight control unit that causes the air vehicle to execute emergency control if the determination unit determines that the condition is satisfied. [Item 2] The unmanned aerial vehicle described in Item 1, wherein the determination unit determines that the predetermined condition is satisfied if the object is detected in the image. [Item 3] The unmanned aerial vehicle described in Item 1, wherein the determination unit calculates the distance between the object and the unmanned aerial vehicle and determines whether the calculated distance satisfies a predetermined distance condition. [Item 4] The unmanned aerial vehicle described in Item 3, wherein the determination unit determines that the distance condition is satisfied if the distance is closer than a predetermined distance. [Item 5] The unmanned aerial vehicle described in Item 1, wherein the determination unit determines that the predetermined condition is met when the object enters a predetermined specific range in the captured image. [Item 6] The unmanned aerial vehicle described in Item 1, wherein the determination unit determines that the predetermined condition is met when the detection duration of the object exceeds a predetermined time threshold. [Item 7] The unmanned aerial vehicle described in Item 1, wherein the determination unit determines that the predetermined condition is met when a predicted collision time calculated based on the movement speed and movement trajectory of the object is within a predetermined time threshold. [Item 8] The unmanned aerial vehicle described in Item 1, wherein the determination unit determines that the predetermined condition is met when the number or density of objects detected in the captured image exceeds a predetermined threshold. [Item 9] The unmanned aerial vehicle described in Item 1, wherein the determination unit determines that the predetermined condition is met when the behavior pattern of the object corresponds to a predetermined abnormal behavior pattern.[Item 10] The unmanned aerial vehicle described in Item 1, wherein the object detection unit determines whether the detected object is equipped with a specific identifier, and excludes objects equipped with the identifier from the objects to be determined by the determination unit. [Item 11] The unmanned aerial vehicle described in Item 1, wherein, when it determines that the predetermined condition is met, the determination unit determines one of multiple levels of attention level based on the positional relationship with the object, and the flight control unit selects and executes an appropriate control action from among continuing flight, taking evasive action, or making an emergency stop depending on the attention level. [Item 12] The unmanned aerial vehicle described in Item 1, wherein the object detection unit detects the object using at least one of object detection by deep learning using a trained model, template matching, contour analysis by edge detection, threshold processing based on color information, and motion detection by background subtraction. [Item 13] The unmanned aerial vehicle described in Item 1, wherein when the determination unit determines that the predetermined condition is met, detection information of the target object is sent to an operator terminal, and the control operation by the flight control unit is determined based on a control instruction from the operator terminal. [Item 14] A control method for an unmanned aerial vehicle, comprising: generating an image using an imaging device; detecting an target object from the image; determining whether the detected target object meets a predetermined condition; and executing emergency control on the unmanned aerial vehicle if it is determined that the condition is met. [Item 15] A program for causing a computer to execute the following processes: detecting an target object from the image; determining whether the detected target object meets the predetermined condition; and executing emergency control on the unmanned aerial vehicle if it is determined that the condition is met.[Item 16] A control system including an unmanned aerial vehicle and a control device that controls the unmanned aerial vehicle, wherein the unmanned aerial vehicle is equipped with an imaging device that generates captured images, and the control device is equipped with: an object detection unit that detects an object from the captured images, a determination unit that determines whether the detected object satisfies a predetermined condition, and a flight control unit that executes emergency control on the unmanned aerial vehicle if the determination unit determines that the condition is satisfied. [Item 17] A control device for an unmanned aerial vehicle, comprising: an object detection unit that detects an object from an captured image generated by the imaging device of the unmanned aerial vehicle, a determination unit that determines whether the detected object satisfies a predetermined condition, and a flight control unit that executes emergency control on the unmanned aerial vehicle if the determination unit determines that the condition is satisfied.

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the accompanying drawings, identical or similar elements are designated by identical or similar reference symbols and names, and duplicate descriptions of identical or similar elements may be omitted in the description of each embodiment. Furthermore, features shown in this embodiment may also be applied to other embodiments as long as they are not mutually inconsistent, and multiple embodiments may be combined to form a configuration.

[0011] FIG. 1 is a diagram illustrating an overview of one embodiment of the present invention. As illustrated in FIG. 1, this embodiment relates to a technology for controlling an unmanned aerial vehicle (UAV, drone). In particular, during drone flight, an object is detected from an image captured by an imaging device, and safe flight control is performed by determining whether the detected object satisfies predetermined conditions. As an example, during a landing phase, the drone shown in FIG. 1 carries a payload such as luggage, and the drone descends to land in order to drop the payload at a landing site such as a drone port. If an object such as a person enters or is present near the landing site, there is a possibility that the drone may come into contact with the object, or that the drone may be unable to land at the landing site.

[0012] The unmanned aerial vehicle, unmanned aerial vehicle control method, program, control system, and control device according to the present embodiment capture images of the drone's surroundings using an imaging device such as a camera, analyze the captured images, and detect whether a target object, such as a person, is present in the captured images. If the drone detects a target object, it determines whether the positional relationship between the drone and the target object satisfies a predetermined condition. If the predetermined condition is met, the drone executes emergency control based on an emergency control signal. As a more specific example, if the distance between the detected target object and the drone meets a predetermined condition (e.g., if it is below a safe distance), the drone executes emergency control (e.g., evasive maneuver, hovering, emergency stop, etc.). This can prevent problems such as contact between the target object and the drone or an emergency landing.

[0013] Note that this technology is not limited to the landing phase, but can be applied as safety control based on the positional relationship between the aircraft and the target object throughout the entire flight, from takeoff to landing. That is, the technology of this embodiment can be applied according to each phase of flight, with target object detection and distance determination during landing as its basic configuration. During cruising flight, it can be applied as detection and determination processing in a dynamic environment where both the aircraft and the target object are moving, and during the landing phase, it can be applied as precise detection and determination processing at the landing point. In this way, the basic detection and determination logic is common, and various flight scenarios can be accommodated by adjusting parameters and processing according to the flight situation.

[0014] The control of the above-mentioned flying object can be performed by a processor, memory, etc. incorporated in hardware (control device) such as a flight controller mounted on the flying object. The flying object may also be equipped with various sensors (not particularly limited) such as an acceleration sensor, a barometric sensor, a magnetic sensor, an image capture device (camera), a communication module, a GPS module, etc. The flight controller may acquire signals obtained from these various devices and process them appropriately, or a companion computer may be mounted separately to perform determination processing and send instructions to the flight controller.

[0015] 2 is a block diagram showing an example of the configuration of a control device according to this embodiment. The control device includes at least a control unit 10, a memory 11, a storage 12, a transmission / reception unit 13, an input / output unit 14, etc., which are electrically connected to one another via a bus 15.

[0016] The control unit 10 is a computing device that controls the operation of the entire control device, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication processing, etc. For example, the control unit 10 is a CPU (Central Processing Unit) and / or GPU (Graphics Processing Unit), and executes programs for this system stored in the storage 12 and deployed in the memory 11 to perform various information processing.

[0017] The memory 11 includes a main memory configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory) and an auxiliary memory configured with a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive). The memory 11 is used as a work area for the processor 10, and also stores a BIOS (Basic Input / Output System) that is executed when the control device is started, various setting information, etc.

[0018] The storage 12 stores various programs such as application programs, etc. A database that stores data used for each process may be constructed in the storage 12.

[0019] The transmitter / receiver 13 connects the management server 1 to the network and the blockchain network. The transmitter / receiver 13 may include a short-range communication interface for Bluetooth (registered trademark) and BLE (Bluetooth Low Energy).

[0020] The input / output unit 14 is an information input device such as a keyboard and a mouse, and an output device such as a display.

[0021] The bus 15 is commonly connected to the above elements and transmits, for example, address signals, data signals and various control signals.

[0022] <Functions of the Control Unit> Figure 3 is a block diagram showing the functions of the control unit 10 of Figure 2. In this embodiment, the control unit 10 has an imaging control unit 110, an object detection unit 120, a determination unit 130, and a flight control unit 140 as functional units. The determination unit 130 includes a condition comparison unit 131, an attention level evaluation unit 132, and an operation determination unit 133. These functional units are realized by a program that runs on the control unit 10. This control unit realizes safety control during flight in various applications, from commercial flights such as payload delivery to monitoring and inspection.

[0023] <Storage Structure> Figure 4 is a block diagram showing the structure of the storage 12 in Figure 2. The storage 12 includes various databases, such as a setting information storage unit 121, a learning model storage unit 122, a safety standard storage unit 123, a captured image storage unit 124, a flight path information storage unit 125, and a log information storage unit 126. The setting information storage unit 121 stores imaging control settings (imaging interval, imaging start conditions, etc.), flight control parameters, payload information (weight, dimensions, delivery destination, etc.), and operator terminal linkage settings (registered IP address, wireless communication ID, communication protocol settings, timeout period, data format settings, control mode selection, etc.). The learning model storage unit 122 stores trained models for object detection (deep learning models such as CNN and YOLO), as well as reference images for template matching, edge detection parameters, color space threshold settings, conventional machine learning models (SVM classifier, Cascade classifier, etc.), etc. The safety standard storage unit 123 stores a minimum safe distance table for each object type, risk assessment criteria, settings for areas requiring attention within an image (coordinate information for areas around the landing point, areas ahead of the flight path, etc.), settings for allowing / prohibiting object detection, settings for judgment conditions by flight phase, an identifier information database (permission markers, worker identification codes, etc.), and operator terminal linkage settings (registered IP address, wireless communication ID, communication protocol settings, timeout period, data format settings, etc.). The flight path information storage unit 125 stores waypoint information, flight path data, delivery point information, etc. transmitted from a GCS (Ground Control Station). Note that the information stored in each storage unit is an example for explanatory purposes and is not limited to these.

[0024] The imaging control unit 110 is a function executed by the control unit 10 and controls the imaging device based on the drone's flight state and flight phase. Specifically, it references flight path information stored in the flight path information storage unit 125, sends an imaging start command to the imaging device during flight phases in which target detection is required, and executes processing to receive captured image data. For example, during the landing phase, imaging begins upon approaching a landing waypoint or receiving a landing instruction from the pilot, and during cruising flight, imaging is performed for continuous surrounding monitoring. The imaging interval and imaging range are dynamically adjusted based on the settings in the setting information storage unit 121 depending on the flight speed, altitude, flight phase, presence or absence of a payload, etc. For example, when the flight speed is high (e.g., 15 m / s or higher), the imaging interval is shortened (e.g., 1 / 60 second interval), and when the flight speed is low (e.g., less than 5 m / s), the imaging interval is lengthened (e.g., 1 / 15 second interval). Regarding altitude, the imaging range is set narrow and precise during low-altitude flight (e.g., less than 50 m), and set wide during high-altitude flight (e.g., 100 m or higher). Furthermore, when a payload is loaded, the imaging angle is adjusted taking into account changes in the aircraft's center of gravity, and during the landing phase, the imaging setting is switched to a more precise setting than during cruising flight. These settings can be determined by pre-setting from an external system (GCS, pilot terminal, etc.), automatic setting based on the flight plan or aircraft status, or dynamic adjustment during flight.

[0025] The object detection unit 120 is a function executed by the control unit 10. It performs image analysis processing on captured image data acquired by the imaging control unit 110 to detect objects (people, animals, vehicles, etc.) in the captured image and determine the object type. The detection processing can apply various image analysis algorithms, either singly or in combination, such as deep learning-based object detection processing using trained models (CNN, YOLO, etc.) stored in the learning model storage unit 122, template matching, contour analysis using edge detection, threshold processing based on color information, motion detection using background subtraction, and conventional machine learning methods using HOG features and Haar-like features. When an object is detected, position information such as bounding box coordinates and center of gravity coordinates of the detected area, size information such as width, height, and area, and shape features such as aspect ratio, contour features, and symmetry are extracted. The object type is determined by classification output from a trained model or rule-based determination based on extracted features (size, shape, color, movement pattern, etc.), to identify the object type (people, animals, vehicles, buildings, etc.). During the landing phase, it precisely detects objects near the landing point, and during cruising flight, it continuously detects objects around the flight path. Information about the detected objects (type, position coordinates, size, shape features, etc.) is sent to the determination unit 130 and used in safety determination processing according to the type.

[0026] The determination unit 130 is a function executed by the control unit 10. Based on the object information detected by the object detection unit 120, the determination unit 130 determines whether the positional relationship between the drone and the object satisfies predetermined conditions and determines whether the drone should continue flying. The determination unit 130 is composed of subordinate functional units including a condition comparison unit 131, an attention level evaluation unit 132, and an action determination unit 133. The determination of the positional relationship can be performed using any of the following patterns, either individually or in combination: (1) distance-based determination, (2) presence-based determination within an image, (3) specific range-based determination within an image, (4) time duration-based determination, (5) speed / trajectory-based determination, (6) density-based determination, and (7) behavioral pattern-based determination. The processing of the condition comparison unit 131, attention level evaluation unit 132, and action determination unit 133 for each pattern will be described in detail below. Note that detailed condition settings, such as numerical values, are merely examples and are not limited to these.

[0027] (1) The processing of the condition comparison unit 131 in a distance-based judgment pattern will be described. The condition comparison unit 131 first calculates the distance between the target and the drone. In the case of a stereo camera, the distance is calculated by triangulation using the parallax between the left and right images. In the case of a monocular camera, the estimated distance is calculated by comparing the bounding box size of the detected target with the standard size for each target type stored in the learning model storage unit 122 (e.g., person: 170 cm, passenger car: 4.5 m, etc.). In the case of a depth camera, directly acquired depth information is used. The calculated distance is compared with a minimum safe distance table stored in the safety standard storage unit 123 (e.g., person: 5 m, animal: 3 m, vehicle: 2 m, building: 1 m, etc.). If the distance is equal to or less than the standard value, a caution state is determined. If the distance is greater than the standard value, a safe state is determined. The result is transmitted to the caution level evaluation unit 132. Furthermore, during cruising flight, in addition to the current distance, a comparison process is also performed on the predicted closest approach distance calculated from the movement vector of the target and the drone.

[0028] (1) The processing of the attention level evaluation unit 132 in a distance-based judgment pattern will be described. Based on the distance comparison results from the condition comparison unit 131, the attention level evaluation unit 132 comprehensively analyzes the distance value, object type, relative speed, approach direction, and object movement pattern (e.g., stationary, walking speed, running speed, etc.) to determine the attention level. Specific examples include evaluation in five levels: Level 1 (e.g., safe: 150% or more of the reference distance), Level 2 (e.g., slight caution required: 120-150% of the reference distance), Level 3 (e.g., caution required: 100-120% of the reference distance), Level 4 (e.g., high caution required: 50-100% of the reference distance), and Level 5 (e.g., urgent caution required: less than 50% of the reference distance). Furthermore, correction processing is performed, such as increasing the attention level by one level when the approach speed is high (e.g., relative speed of 5 m / s or more) or when the object is approaching directly, and decreasing it by one level when the object is moving away. When multiple objects are detected simultaneously, the highest attention level is adopted.

[0029] (1) The processing of the operation determination unit 133 in a distance-based determination pattern will be described. The operation determination unit 133 determines a control operation by combining the attention level from the attention level evaluation unit 132, the current flight phase (e.g., takeoff, cruising, landing), and the aircraft status (e.g., speed, altitude, presence or absence of payload). For example, in the landing phase, the unit determines whether to continue landing at level 1 or 2, whether to temporarily hover (wait and see for 10 seconds) at level 3, whether to avoid ascent (reevaluate after ascent of 5 m) at level 4, and whether to make an emergency stop and abort landing at level 5. For example, during cruising flight, the unit determines whether to continue flying at level 1 or 2, whether to decelerate flight (reduce speed by 50%) at level 3, whether to horizontally avoid flight (move 3 m left or right) at level 4, and whether to perform an emergency hover at level 5. The determined control operation is transmitted to the flight control unit 140 as a control command. After the control operation is executed, the unit reevaluates at a predetermined time interval (e.g., every 3 seconds), and if a safe state is restored, the unit returns to normal flight.

[0030] (2) The processing of the condition comparison unit 131 in the judgment pattern based on presence in an image will be described. In this pattern, distance calculation is not performed, and judgment is made solely based on whether or not an object is detected in the captured image. The condition comparison unit 131 receives the detection results (object type, reliability, and position coordinates) from the object detection unit 120 and compares them with the detection permission settings stored in the safety standard storage unit 123. For example, judgment is made based on settings such as prohibiting human and animal detection in high-security areas, prohibiting human and animal detection in general areas, and permitting human and animal detection in general areas. If the detected object is a prohibited or a warning object, it is judged as requiring caution; if it is a permitted object or not detected, it is judged as safe. The information is then transmitted to the caution level evaluation unit 132 along with the object type and detection reliability. The continuous detection time of the same object is also recorded and provided as evaluation material based on residence time.

[0031] (2) The processing of the attention level evaluation unit 132 in the judgment pattern based on presence in the image will be described. The attention level evaluation unit 132 comprehensively evaluates the object type, detection reliability, detection duration, and position in the image to determine the attention level. The basic level is Level 4 (high level of caution required) if a person is detected, Level 3 (high level of caution required) if an animal is detected, and Level 2 (low level of caution required) if a vehicle is detected. If the detection reliability is high (90% or higher), the basic level is maintained, and if the reliability is low (less than 70%), the level is lowered by one level. In addition, if the same object is detected for a long time (30 seconds or more), it is treated as a stationary object and lowered by one level. If it is detected multiple times in a short period of time (three or more times within five seconds), it is treated as a moving object and raised by one level. Position correction is also performed, where detections in the center of the image are likely to be on the flight path and are therefore raised by one level, and detections in the periphery of the image are lowered by one level.

[0032] (2) The processing of the operation determination unit 133 in the image presence determination pattern will be described. Since distance information is not available in this pattern, the operation determination unit 133 determines a standard control operation based on the object type and attention level. The basic operation is to immediately decelerate (30% of the current speed) and pause (for 5 seconds) when a person is detected (level 3-4), to gradually decelerate (50% of the current speed) when an animal is detected (level 2-3), and to continue flying while continuously monitoring when a vehicle is detected (level 1-2). For each flight phase, if a person or animal is detected during the landing phase, the aircraft will abort landing and avoid ascent; during cruising flight, the aircraft will continue to decelerate and fine-tune its route; and during the takeoff phase, the aircraft will pause and recheck its surroundings. Object detection continues even during control operations, and if no object is detected (no detection for three consecutive frames), the aircraft will return to normal flight. If multiple types of objects are detected simultaneously, the most restrictive control operation will be selected. Although an example of responding to each type of target object has been described, it may be simplified to perform caution control when it is determined that a specific type of target object is present, depending on whether it is present or not.

[0033] (3) The processing of the condition comparison unit 131 in a judgment pattern based on a specific range within an image will be described. The condition comparison unit 131 compares the position coordinates of the object within the image (bounding box center coordinates) received from the object detection unit 120 with the coordinates of a caution area previously set in the safety standard storage unit 123. The caution area is set according to the application, and is defined as a circular area (radius 50 to 100 pixels) centered on the landing point during the landing phase, a fan-shaped area (central angle 30 to 60 degrees, distance 100 to 200 pixels) forward in the flight direction during cruising flight, or a rectangular area around the takeoff point during the takeoff phase. If the position coordinates of the object are within the caution area, a caution state is determined, and if outside the area, a safe state is determined. Furthermore, if the object is detected near the area boundary (within 10 pixels of the boundary), a quasi-caution state is determined as a possible boundary intrusion, and information on the object type, intrusion area type, and distance from the boundary is sent to the caution level evaluation unit 132.

[0034] (3) The processing of the attention level evaluation unit 132 in a judgment pattern based on a specific range within an image will be described. The attention level evaluation unit 132 comprehensively evaluates the intrusion area type, object type, and location within the area to determine the attention level. Area importance is classified into landing site area (highest importance), forward flight path area (important), and perimeter monitoring area (general), with higher importance assigned a higher attention level. A person intruding into the landing site area is assigned level 5 (urgent attention required), an animal intruding level 4 (high level attention required), and a vehicle intruding level 3 (attention required). The evaluation is lowered by one level for the forward flight path area and two levels for the perimeter monitoring area. Intrusion into the center of the area is also evaluated one level higher than intrusion into the perimeter, and simultaneous intrusion of multiple objects is corrected by the maximum level plus one level. Correction is also performed based on the duration of intrusion: a short-term intrusion (less than 5 seconds) is considered a passing intrusion and is lowered by one level, while a long-term intrusion (30 seconds or more) is considered a lingering intrusion and is raised by one level.

[0035] (3) The processing of the operation determination unit 133 in the judgment pattern based on a specific range within the image will be described. The operation determination unit 133 determines a control operation that prioritizes area protection based on the type of intrusion area and the caution level. When an object intrudes into the landing point area, it determines whether to abort the landing and avoid ascent (ascend 10 m) at level 3 or higher, or whether to perform an emergency ascent (ascend 20 m) and change the landing point at level 5. When an object intrudes into the area ahead of the flight path, it selects decelerated flight at level 2-3, temporary suspension at level 4, or detouring flight (moving 10 m left or right) at level 5. When an object intrudes into the surrounding monitoring area, it selects continuous monitoring at levels 1-2 and cautionary flight (70% speed) at levels 3-4. The control operation continues until the intrusion is resolved, and if the object leaves the area (confirmed after five consecutive frames), it gradually returns to normal flight. When an object intrudes into multiple areas simultaneously, it adopts the most restrictive control operation, and while the control operation is being executed, it doubles the monitoring frequency of the relevant area (from the usual 1 / 15 second interval to 1 / 30 second interval).

[0036] (4) The processing of the condition comparison unit 131 in the time continuation judgment pattern will be described. The condition comparison unit 131 manages the detection information of the same object continuously transmitted from the object detection unit 120 in chronological order and measures the dwell time of each object. Object identity is determined based on proximity of position coordinates (movement within 5 pixels from the previous frame), size similarity (area change within 20%), and matching of shape features (80% or more). The measured dwell time is compared with the dwell time threshold stored in the safety standard storage unit 123 (people: 30 seconds, animals: 45 seconds, vehicles: 60 seconds, buildings: 120 seconds, etc.), and if the threshold is exceeded, it is determined to be in a state requiring caution. Furthermore, if an object repeatedly moves slightly within a certain range (within a 3-pixel radius), it is also determined to be dwelling and managed separately from completely stationary objects. If detection is interrupted (no detection for three consecutive frames), the dwell time is reset, and if it is detected again, it is treated as a new object.

[0037] (4) The processing of the attention level evaluation unit 132 in the time duration judgment pattern will be described. The attention level evaluation unit 132 determines the attention level by comprehensively evaluating the length of stay time, object type, and importance of the stay location. The basic attention level is set as Level 2 (minor caution required) when the stay time is 100-120% of the threshold, Level 3 (caution required) when it is 120-150%, Level 4 (high caution required) when it is 150-200%, and Level 5 (urgent caution required) when it is over 200%. Corrections based on object type are made by raising the level by one level for prolonged human stays because they are likely intentional behavior, lowering the level by one level for animal stays as they are resting behavior, and maintaining the basic level for vehicle stays as they are considered to be breakdowns or congestion. Corrections based on stay location are made by raising the level by two levels for stays at landing sites, raising the level by one level for stays along flight paths, and not correcting in surrounding areas. Furthermore, when multiple objects are staying at the same time, a correction of the highest level + one level is applied to account for the crowding effect.

[0038] (4) The processing of the operation decision unit 133 in the time duration judgment pattern will be described. The operation decision unit 133 determines a control action to promote the resolution of the congestion based on the congestion status and attention level. At levels 2 and 3, an alarm (1 kHz, 3 seconds) is emitted to alert the congestion occupant, and a visual warning is given by flashing an LED. At level 4, a temporary ascent (5 m) to ensure a safe distance, a detour flight on an alternative route, or a temporary change of landing site (movement within a 50 m range) is selected. At level 5, flight is suspended until the congestion is resolved, the aircraft changes to an emergency landing site, and the situation is reported to ground control. The aircraft continues to monitor the congestion time even while the control action is being executed, and if the target begins to move (continuous movement distance of 10 pixels or more), the aircraft gradually returns to normal flight. If the congestion exceeds the scheduled flight time (30 minutes or more), the aircraft automatically transitions to an alternative mission (return, standby, etc.).

[0039] (5) The processing of the condition comparison unit 131 in the speed / trajectory judgment pattern will be described. The condition comparison unit 131 calculates the movement speed and movement direction from the change in the object's position between consecutive frames and determines the possibility of intersection with the drone's flight trajectory. The object's movement speed is calculated by dividing the change in position coordinates by the time interval (e.g., 10 pixel movement / 0.1 second = 100 pixels / second), and conversion to actual distance is estimated from altitude information and the angle of view. The movement direction is calculated by linear approximation using the least squares method from the position coordinates of three consecutive frames. The drone's flight trajectory is predicted from the planned route stored in the flight path information storage unit 125 and the current flight status, and an intersection with the object's trajectory is calculated. If the predicted intersection time is within the setting value of the safety standard storage unit 123 (e.g., 10 seconds for people, 15 seconds for vehicles, 8 seconds for animals), a caution state is determined. In addition, if the object is moving straight toward the drone (within an angle difference of 30 degrees), it is treated as a special caution state.

[0040] (5) The processing of the caution level evaluation unit 132 in the speed / trajectory judgment pattern will be described. The caution level evaluation unit 132 comprehensively evaluates the intersection prediction time, object speed, approach angle, and trajectory accuracy to determine the caution level. The basic levels based on the intersection prediction time are Level 2 (minor caution required) for 10 to 15 seconds, Level 3 (caution required) for 5 to 10 seconds, Level 4 (high caution required) for 3 to 5 seconds, and Level 5 (urgent caution required) for less than 3 seconds. However, if analysis of the object's movement direction and speed predicts that the object will rapidly move away from the caution range (e.g., move out of the safe distance within 5 seconds at the current movement vector), the caution level is lowered by two levels or a determination is made not to execute emergency control. This makes it possible to avoid unnecessary emergency control for objects that approach temporarily but immediately move away.

[0041] (5) The processing of the operation determination unit 133 in the speed / trajectory judgment pattern will be described. The operation determination unit 133 determines preventive avoidance control based on the collision prediction time and attention level. At levels 2 and 3, flight continues while monitoring changes in the target object's trajectory, and avoidance preparations (aircraft attitude adjustment, avoidance direction calculation) are performed if the possibility of trajectory crossing increases. At level 4, immediate preventive avoidance actions are selected, such as moving perpendicular to the target object's trajectory (3 m horizontally, 2 m vertically) and adjusting flight speed (avoiding crossing timing by accelerating or decelerating). At level 5, emergency avoidance maneuvers are performed at maximum capacity (sudden ascent of 5 m, sudden stop, and departure at maximum speed). After the avoidance action is performed, the target object's trajectory is recalculated, and if the crossing risk is resolved (predicted crossing time is 30 seconds or more), operation to return to the original flight path is initiated. When simultaneous crossings with multiple targets are predicted, control actions are determined based on the earliest crossing prediction time.

[0042] (6) The processing of the condition comparison unit 131 in the density-based judgment pattern will be described. The condition comparison unit 131 calculates the number and distribution density of simultaneously detected objects and compares them with the density threshold stored in the safety standard storage unit 123. In the object count, the number of detections by type (people, animals, vehicles) is tallied and the density is calculated for the entire image and local areas (each area divided into 3x3 regions of the image). Density calculation is performed by dividing the number of objects by the area (number of pixels). If the density exceeds the standard density (people: 5 or more people in the entire image or 3 or more people in a local area; vehicles: 4 or more vehicles in the entire image or 2 or more vehicles in a local area, etc.), a caution state is determined. Furthermore, if the minimum distance between objects falls below a threshold (people: within 20 pixels; vehicles: within 50 pixels), a crowd or traffic jam state is determined. Objects with low detection reliability (less than 70%) are excluded as unreliable detections, and a reliable density evaluation is performed.

[0043] (6) The processing of the attention level evaluation unit 132 in a density-based judgment pattern is described below. The attention level evaluation unit 132 comprehensively evaluates the object density, type composition, and distribution pattern to determine the attention level. The basic attention level is set as Level 2 (minor caution required) for 100-150% of the reference density, Level 3 (high caution required) for 150-200%, Level 4 (high caution required) for 200-300%, and Level 5 (urgent caution required) for over 300%. Corrections based on object type are performed by raising the level by one level for crowds of people, as this represents a risk of unpredictable collective behavior; maintaining the basic level for collections of vehicles, as this represents movement restrictions due to congestion; and no correction is performed for groups of animals, as this represents herd behavior and the possibility of uniform movement direction. Corrections based on distribution pattern are performed by raising the level by one level when objects are concentrated (local density is more than twice the overall density), and no correction is performed when objects are evenly distributed. Furthermore, dynamic correction is applied, raising the level by one level when the density is increasing over time (increasing by 20% or more in the past minute).

[0044] (6) The processing of the operation decision unit 133 in the density-based judgment pattern will be described. The operation decision unit 133 determines control operations aimed at avoiding crowds and congestion based on the density situation and attention level. At levels 2 and 3, the unit selects detouring around high-density areas, increasing flight altitude (leaving high-density areas), or slowing flight speed (strengthening monitoring of crowd behavior). At level 4, the unit completely leaves high-density areas, changes to an alternative route, and, if necessary, changes the landing site (moving to a low-density area). At level 5, the unit decides to suspend flight and wait in a safe area, report crowd information to ground control, wait for the density to clear, or completely change the route. Density monitoring continues even while the control operation is being performed, and if the density falls below the threshold (below the threshold for 30 consecutive seconds), the unit gradually returns to normal flight. If there are multiple high-density areas, the unit automatically calculates a route that passes through the least dense area and determines the optimal detouring route.

[0045] (7) The processing of the condition comparison unit 131 in the judgment pattern based on behavioral patterns will be described. The condition comparison unit 131 analyzes the behavioral features of the target object in time series and compares them with abnormal behavioral patterns stored in the learning model storage unit 122. For human behavioral patterns, hand movements (waving, pointing, etc.), posture changes (standing up, crouching, etc.), and movement patterns (sudden stops, direction changes, starting to run, etc.) are extracted by skeletal detection. For animal behavioral patterns, head direction (gazing at the drone, shaking head, etc.), changes in body position (alert posture, preparing to attack, etc.), and flocking behavior (dispersing, gathering, etc.) are detected by shape analysis. For vehicle behavioral patterns, sudden braking (detection of tail lamp illumination), turn signal (detection of blinking blinker), and door opening / closing (detection of changes in vehicle shape) are extracted by image change analysis. The extracted behavioral patterns are compared with abnormal behavior definitions (hand waving: warning, sudden stops: emergency, etc.) in the safety standard storage unit 123, and if applicable, a caution-requiring state is determined.

[0046] (7) The processing of the attention level evaluation unit 132 for judgment patterns based on behavioral patterns will be described. The attention level evaluation unit 132 comprehensively evaluates the type, duration, and intensity of the detected behavioral pattern to determine the attention level. In the evaluation of human behavior, hand waving behavior is considered a warning and is rated as level 3 (caution required), pointing behavior is considered a directional indication and is rated as level 2 (minor caution required), rapid movement (running) is considered an emergency avoidance and is rated as level 4 (high level caution required), and falling behavior is considered an accident possibility and is rated as level 5 (urgent caution required). In the evaluation of animal behavior, an alert posture is rated as level 2, threatening behavior is rated as level 3, a posture preparing to attack is rated as level 4, and rapid movement of a group is rated as level 4. In the evaluation of vehicle behavior, turn signal activation is rated as level 2, sudden braking is rated as level 3, door opening is rated as level 3, and emergency flashing is rated as level 5. In the correction based on behavior duration, short-term behavior (less than 3 seconds) is considered temporary and is lowered by one level, while long-term behavior (30 seconds or more) is considered a persistent abnormality and is raised by one level. If the action intensity (suddenness of the movement, amplitude) is high, a correction will be applied that increases it by one level.

[0047] (7) The processing of the operation determination unit 133 in the judgment pattern based on the behavioral pattern will be described. The operation determination unit 133 estimates the intention of the detected behavioral pattern and determines the appropriate response control. When detecting human attention-seeking behavior (waving, pointing), the unit temporarily stops and checks the surroundings at levels 2 to 3, ascends to maintain a safe distance at level 4, and prepares for an emergency landing at level 5. When detecting alert or aggressive behavior from an animal, the unit selects silent mode (reduced propeller rotation speed) and detouring flight at levels 2 to 3, evacuates to a higher altitude at level 4, and completely leaves the area at level 5. When detecting emergency vehicle behavior (sudden braking, emergency flashing), the unit monitors traffic conditions and flies cautiously at level 3, avoids over roads at level 4, and performs a temporary landing to avoid misidentification with an emergency vehicle at level 5. The unit continues to monitor the behavioral pattern even while the control action is being executed, and if normal behavior returns (normal behavior for 10 consecutive seconds), the unit gradually returns to normal flight. When abnormal behavior from multiple objects is detected simultaneously, the unit determines the control action based on the behavior with the highest level of urgency.

[0048] For each of the above-mentioned assessment patterns (1) to (7), the assessment criteria can be dynamically adjusted according to environmental conditions. For time-of-day adjustments, the safety distance is set to 1.5 times and the residence time threshold is set to 0.7 times during night flights (two hours after sunset to two hours before sunrise) to account for reduced visibility. For weather adjustments, the caution level is increased by one level during rainy weather to account for the difficulty of predicting the behavior of targets, and the safety distance is set to double during strong winds (10 m / s or more) to account for reduced aircraft controllability. For location adjustments, the caution level when detecting a person is increased by one level around schools and hospitals (within a 200 m radius), and the vehicle and worker density assessment criteria are relaxed (1.5 times the standard density) around construction sites. For aircraft condition adjustments, the timing of evasive action is accelerated (prediction time is multiplied by 1.2) to account for reduced maneuverability when a payload is loaded, and the overall caution level is increased by one level when the battery level is low (less than 30%), providing conservative control. These environmental adaptive adjustments are applied in real time before or during flight based on the environmental parameter table stored in the setting information storage unit 121.

[0049] The flight control unit 140 is a function executed by the control unit 10, and executes normal control according to the flight phase and safety control based on control commands from the judgment unit 130. During normal flight, it executes autonomous flight control for the flight controller to sequentially pass through the planned flight path based on waypoint information transmitted from a GCS (Ground Control Station) stored in the flight path information storage unit 125. During safety control, it temporarily suspends normal flight control and executes emergency control as an interrupt process based on control commands received from the judgment unit 130.

[0050] When executing a control operation, the system generates executable control commands that take into account the aircraft's performance limits (maximum acceleration, maximum ascent speed, minimum turning radius, etc.) and transmits them in a format that conforms to the flight controller's control specifications. When control commands are received simultaneously from multiple judgment patterns, the system prioritizes the most restrictive (safety-oriented) control operation. Furthermore, if environmental adaptation adjustments are configured, the system applies control parameter adjustments (increasing safety distance, accelerating avoidance actions, restricting maneuverability, etc.) according to the time of day, weather, location, and aircraft status. After executing a control operation, the system continuously monitors changes in the target's situation, and gradually returns to normal flight control once the cautionary condition is resolved (return to safety conditions for each judgment pattern). During the return process, a three-second transition period is provided to gradually return to normal flight parameters to avoid sudden changes in control. The execution history of the control operations is recorded in the log information storage unit 126 and used as data for post-flight safety analysis and control performance improvement. If an emergency situation makes it difficult to return to normal flight, the system will automatically initiate an emergency landing sequence at the nearest safe landing site and report the situation to ground control.

[0051] <Target Exclusion Function by Identifier> As a modification of this embodiment, the target detection unit 120 may have a function to exclude targets wearing a specific identifier from emergency control targets. After detecting a target, the target detection unit 120 executes additional processing to determine whether the target has a preset identifier. Examples of identifiers that can be used include markers of specific colors (high-saturation color patches such as red, yellow, and green), machine-readable codes such as QR codes (registered trademark) and barcodes, helmets, bibs, work clothes in specific colors and patterns, and safety equipment with reflective materials.

[0052] In the color information analysis process, the object detection unit 120 extracts an area corresponding to a specific color gamut (threshold setting in HSV color space) from the head and torso area of ​​the detected person, and identifies identifier candidates based on the area size, shape, and positional relationship. In the code reading process, a high-contrast rectangular area is extracted from the detection area, and identification information is obtained using a code decoder / barcode reader function. In the pattern matching process, the similarity with identifier templates (safety helmet shapes, bib patterns, etc.) stored in the learning model storage unit 122 is calculated, and if the similarity is above a threshold, it is recognized as an identifier.

[0053] The identifier information detected by the object detection unit 120 is compared with the permitted identifier database stored in the safety standard storage unit 123, and if the identifier is a registered identifier, the object is classified as a "permitted object." When transmitting information about a permitted object to the determination unit 130, the object detection unit 120 assigns an exclusion flag to the information. The determination unit 130 excludes objects that have been assigned an exclusion flag from various determination processes such as distance determination and range determination, and does not subject the object to emergency control. However, even if the object is a permitted object, if a safety concern is detected due to abnormal behavior detection (such as collapsing or staying for a long time), attention is drawn to the object by displaying a warning, making an audio notification, or the like.

[0054] This function enables on-site workers, maintenance personnel, inspectors, and other personnel required for work to avoid emergency control by wearing appropriate identifiers, thereby achieving both work efficiency and safety.

[0055] <Control Decision-Making Function in Cooperation with Operator Terminal> As a variation of this embodiment, in addition to automatic control, control incorporating judgment by the operator terminal is also possible. When the judgment unit 130 determines that a caution is required, the flight control unit 140 can select a semi-automatic mode in cooperation with the operator terminal, in addition to an automatic mode in which emergency control is immediately executed. In the semi-automatic mode, the flight control unit 140 issues a notification data transmission command to the transceiver unit 13, and the transceiver unit 13 transmits target detection information (target type, position coordinates, distance, predicted trajectory, etc.), calculated caution level, and recommended control action (avoidance direction, avoidance distance, etc.) as a data packet in JSON format or XML format to the IP address or wireless communication ID of the operator terminal previously registered in the setting information storage unit 121.

[0056] The operator terminal analyzes and displays the received data, and transmits the control instructions entered by the operator (execute emergency control, continue monitoring, switch to manual control, abort landing, etc.) as response data. The transceiver unit 13 receives the response data from the operator terminal and transfers it to the control unit 10. The flight control unit 140 analyzes the received instruction data and, depending on the instruction content, executes or does not execute emergency control, or executes control operations using control parameters that differ from the standard (changing the avoidance distance, extending the waiting time, etc.).

[0057] Similarly, when the control unit 10 detects that the safety state has been restored, it sends cancellation notification data to the operator terminal via the transceiver unit 13, and the flight control unit 140 executes processing to return to normal flight based on the cancellation instruction data from the operator terminal. If a communication timeout occurs (e.g., no response within 30 seconds), the most restrictive control action (emergency stop, automatic landing at a safe point, etc.) is automatically executed to ensure safety.

[0058] This function allows human judgment based on on-site conditions to be incorporated into the control system, enabling flexible flight control in complex situations that are difficult to handle with automatic control.

[0059] 5 is a diagram showing the overall flow of the target detection and determination process according to this embodiment. The image capture control unit 110 of the control unit 10 starts the image capture process when it detects a situation during flight that requires target detection (such as the start of a landing phase or continuous monitoring during cruising flight) (step S201).

[0060] Next, the object detection unit 120 analyzes the captured image to determine whether or not an object is present (step S202). If an object is not detected (step S202: NO), the process returns to step S201, and the image capturing process continues.

[0061] Next, if an object is detected (step S202: YES), the object detection unit 120 determines the type of object and extracts features, and transmits the detection information to the determination unit 130 (step S203).

[0062] Next, the determination unit 130 determines whether the positional relationship between the drone and the target object satisfies predetermined conditions (step S204) based on the determination patterns (distance determination, presence in image determination, intrusion into specific range determination, time continuation determination, speed / trajectory determination, density determination, behavior pattern determination, etc.) set in the safety standard storage unit 123. If the predetermined conditions are not satisfied (step S204: NO), that is, if the drone is determined to be in a safe state, the process returns to step S201, and continuous monitoring is performed.

[0063] Next, if the predetermined condition is met (step S204: YES), that is, if it is determined that a caution state is required, the caution level evaluation unit 132 of the judgment unit 130 determines the caution level, and the operation determination unit 133 determines an appropriate control operation (step S205).

[0064] Next, the flight control unit 140 interrupts normal flight control and executes safety control (avoidance operation, hovering, emergency stop, etc.) based on a control command from the operation determination unit 133 (step S206). After executing safety control, it monitors whether the caution-requiring state has been resolved, and if the state returns to a safe state, it gradually returns to normal flight.

[0065] This flow can be applied to each phase of flight (takeoff, cruise, landing), and is executed flexibly according to the judgment pattern and environmental conditions. A specific example of landing will be described in detail below with reference to Figure 6.

[0066] FIG. 6 is a diagram showing a specific example of processing during landing according to this embodiment. The flow of processing will be described based on this flowchart. Note that all of the processing shown in FIG. 6 will be described as being performed solely by the control unit 10 as an edge device. However, as will be described later, image processing, generation of emergency control signals, and the like may be performed by other devices (e.g., a companion computer, a server, etc.). First, while the drone is performing a landing operation, the imaging control unit 110 of the control unit 10 captures (photographs) an image of the vicinity of the landing point using an imaging device mounted on the drone. The type of imaging device is not particularly limited, as long as it is a device capable of generating captured images of the vicinity of the landing point. The imaging device may be, for example, an infrared camera or a stereo camera. Such imaging processing may be performed continuously or intermittently.

[0067] Next, the object detection unit 120 of the control unit 10 analyzes the captured image and detects an object. The object may be a person, an animal, a plant, a structure, soil, sand, snow, etc. Here, a person will be used as an example. If the object detection unit 120 does not detect an object, the flight control unit 140 continues the landing operation. This detection process can be performed repeatedly.

[0068] When the object detection unit 120 detects an object, the determination unit 130 of the control unit 10 determines whether the object satisfies a predetermined condition. The predetermined condition may be, for example, whether the object is present within a predetermined range. Specifically, the predetermined range may be whether the distance between the drone and the object is within a predetermined distance. The distance between the drone and the object may be estimated based on, for example, the analysis results of an image captured by the drone. The distance between the drone and the object may be estimated by combining the analysis results of the image captured with information such as the drone's altitude.

[0069] Furthermore, the predetermined condition may be, for example, whether or not the target object is within the drone's drop dispersion range. The drop dispersion range defines the area within which the drone may crash. This drop dispersion range may be calculated as appropriate based on the drone's altitude, speed, flight path, and environment along the flight path (wind direction, wind speed, etc.). An example of the predetermined condition is whether or not the position of the target object detected by image processing is within the drop dispersion range.

[0070] The above-mentioned predetermined conditions are merely examples, and it is possible to specify predetermined conditions based on the positional relationship between the drone and the target object, and perform the next emergency control process based on whether the conditions are met. The determination unit 130 performs the determination process by referring to various determination conditions (determination based on distance, determination based on presence in an image, determination based on intrusion into a specific range, etc.) stored in the safety standard storage unit 123.

[0071] When a predetermined condition is met, the flight control unit 140 of the control unit 10 performs operational control based on an emergency control signal. This emergency control signal is generated by the control unit 10 and received by the flight controller, which can then perform operational control based on the signal. Examples of emergency control here include stopping or suspending the drone's landing operation, stopping the drone in mid-air, returning to the takeoff point, switching to manual control (using a radio control, etc.), and transmitting information, audio, etc. indicating an emergency.

[0072] With this system, if a person or other person enters a landing point of a drone while the drone is attempting to deliver a package, the system can detect the person and avoid contact between the drone and the person or situations where the drone is unable to land. Because this process is performed by the control unit 10, no special devices or the like are required at the landing point. Therefore, even when drones deliver packages to individual homes, for example, they can take evasive action at the landing point without installing landing equipment such as a drone port.

[0073] 7 is a schematic diagram showing a modified example of this embodiment. For example, image processing of an image captured by a drone may be performed by the drone itself, while emergency control decisions based on the image analysis results and signal transmission may be performed by a control device or the like connected to the drone via a network or the like.

[0074] In this specification, the steps describing the program recorded on the recording medium include not only processes that are performed in chronological order, but also processes that are not necessarily performed in chronological order but are performed in parallel or individually. Also, in this specification, the term "system" means an overall device composed of multiple devices or multiple means, etc.

[0075] Furthermore, one embodiment of the present invention may include the following aspect: A method for controlling an unmanned aerial vehicle, including a processor analyzing captured images of an area including a landing point and its vicinity, obtained by an imaging device mounted on an unmanned aerial vehicle that is performing a landing operation at the landing point, detecting an object based on the analysis results of the captured images, estimating a positional relationship between the unmanned aerial vehicle and the object based on at least the captured images, determining whether the positional relationship satisfies a predetermined condition, and performing emergency control of the landing operation of the unmanned aerial vehicle when the positional relationship satisfies the predetermined condition. A control device, control system, and program capable of implementing the method for controlling an unmanned aerial vehicle.

[0076] REFERENCE SIGNS LIST 1 control device 10 control unit 11 memory 12 storage 13 transmission / reception unit 14 input / output unit 15 bus 110 imaging control unit 120 object detection unit 130 judgment unit 131 condition comparison unit 132 caution level evaluation unit 133 operation determination unit 140 flight control unit 121 setting information storage unit 122 learning model storage unit 123 safety standard storage unit 124 captured image storage unit 125 flight path information storage unit 126 log information storage unit

Claims

1. An unmanned aerial vehicle comprising: an imaging device provided on the unmanned aerial vehicle for generating captured images; an object detection unit for detecting an object from the captured images; a judgment unit for judging whether the object detected by the object detection unit satisfies predetermined conditions; and a flight control unit for executing emergency control on the aircraft when the judgment unit judges that the conditions are satisfied.

2. An unmanned aerial vehicle as described in claim 1, characterized in that the judgment unit judges that the specified condition is met when the target object is detected in the captured image.

3. An unmanned aerial vehicle as described in claim 1, characterized in that the judgment unit calculates the distance between the target object and the unmanned aerial vehicle and determines whether the calculated distance satisfies a predetermined distance condition.

4. An unmanned aerial vehicle as described in claim 3, characterized in that the judgment unit judges that the distance condition is met when the distance is closer than a predetermined distance.

5. An unmanned aerial vehicle as described in claim 1, characterized in that the judgment unit judges that the specified condition is met when the target object enters a predetermined specific range within the captured image.

6. An unmanned aerial vehicle as described in claim 1, characterized in that the judgment unit judges that the specified condition is met when the continuous detection time of the target object exceeds a predetermined time threshold.

7. An unmanned aerial vehicle as described in claim 1, characterized in that the judgment unit judges that the specified condition is met when the predicted collision time calculated based on the movement speed and movement trajectory of the target object is within a predetermined time threshold.

8. An unmanned aerial vehicle as described in claim 1, characterized in that the judgment unit judges that the specified condition is met when the number or density of objects detected in the captured image exceeds a predetermined threshold.

9. An unmanned aerial vehicle as described in claim 1, characterized in that the judgment unit judges that the specified condition is met when the behavior pattern of the target object corresponds to a predetermined abnormal behavior pattern.

10. An unmanned aerial vehicle as described in claim 1, characterized in that the object detection unit determines whether the detected object is equipped with a specific identifier, and excludes objects equipped with the identifier from the objects to be determined by the determination unit.

11. An unmanned aerial vehicle as described in claim 1, wherein, when the judgment unit determines that the specified conditions are met, it determines one of multiple levels of caution level based on the positional relationship with the target object, and the flight control unit selects and executes an appropriate control action from among continuing flight, taking evasive action, or making an emergency stop depending on the caution level.

12. An unmanned aerial vehicle as described in claim 1, characterized in that the object detection unit detects the object using at least one of the following methods: object detection by deep learning using a trained model, template matching, contour analysis by edge detection, threshold processing based on color information, and motion detection by background subtraction.

13. An unmanned aerial vehicle as described in claim 1, characterized in that when the judgment unit determines that the specified conditions are met, it transmits detection information of the target object to an operator terminal, and determines the control action to be taken by the flight control unit based on control instructions from the operator terminal.

14. A method for controlling an unmanned aerial vehicle, comprising: generating an image using an imaging device; detecting an object from the image; determining whether the detected object satisfies a predetermined condition; and, if it is determined that the condition is satisfied, executing emergency control on the unmanned aerial vehicle.

15. A program for causing a computer to perform the following processes: detecting an object from a captured image; determining whether the detected object satisfies predetermined conditions; and, if it is determined that the object satisfies the conditions, executing emergency control on the aircraft.

16. A control system including an unmanned aerial vehicle and a control device that controls the unmanned aerial vehicle, wherein the unmanned aerial vehicle is equipped with an imaging device that generates an image, and the control device is equipped with: an object detection unit that detects an object from the image; a judgment unit that determines whether the detected object satisfies a predetermined condition; and a flight control unit that executes emergency control on the unmanned aerial vehicle when the judgment unit determines that the condition is satisfied.

17. A control device for an unmanned aerial vehicle, comprising: an object detection unit that detects an object from an image captured by an imaging device of the unmanned aerial vehicle; a judgment unit that determines whether the detected object satisfies predetermined conditions; and a flight control unit that executes emergency control on the unmanned aerial vehicle when the judgment unit determines that the condition is satisfied.

Images