Control device and control method

Abstract

To provide a control device and a control method of flight objects capable of obtaining an optimal flight path for each flight object.SOLUTION: A control device includes: a surrounding monitoring unit that monitors the surrounding conditions of a flight object using information detected by a sensor; a self-position estimating unit that obtains the self-position of the flight object; a flight object surrounding environment transmitting unit that transmits the surrounding conditions and the self-position of the flight object as flight object surrounding environment information; a route information receiving unit that receives route information; and a flight control unit that controls the operation of the flight object according to the route information. The surrounding conditions from the surrounding monitoring unit include information on the visibility in each direction from the flight object.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a control apparatus and a control method.

Background Art

[0002] When socially implementing flying objects such as aircraft as transportation and logistics infrastructure, it is necessary to safely and efficiently control a large number of aircraft.

[0003] Regarding this point, conventionally, when setting a movement route for a flying object such as an aircraft, information related to the surrounding conditions of the moving object is used. For example, the flight route of an aircraft is created and set based on current information acquired by various sensors of the aircraft. Therefore, during flight along this flight route, there is a possibility of encountering some obstacle that was not recognized at the time of information acquisition.

[0004] Regarding this point, for example in Patent Document 1, for the purpose of creating and setting a safe movement route that takes into account various factors that may threaten safety during movement, it is configured to create and set a safe movement route that takes into account factors that may threaten safety during movement (atmospheric conditions, birds, other flying objects, etc.), create a scenario predicting surrounding conditions for each of a plurality of predetermined future times based on the acquired surrounding information and a probability model, and search for a flight route based on the scenario.

Prior Art Literature

Patent Literature

[0005]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0006] According to Patent Document 1, while it is possible to create a safe flight path for anticipated factors during movement, the problem remains that it is not possible to plan long-distance routes and therefore the optimal flight path cannot be obtained. Furthermore, for aircraft operating in the suburbs of urban areas, it is desirable to achieve both safety and security during flight and operational efficiency, especially within a few square kilometers around the take-off and landing fields.

[0007] However, when determining a flight path, it is desirable to consider visibility as far as possible from the aircraft, but the necessary visibility estimation accuracy and sensing distance cannot always be ensured by sensing from the aircraft alone.

[0008] Therefore, the present invention aims to provide a control device and control method that can obtain the optimal flight path for each aircraft. [Means for solving the problem]

[0009] Based on the above, the present invention is a control device comprising: an area monitoring unit that monitors the surrounding conditions of an aircraft using information detected by sensors; a self-position estimation unit that determines the aircraft's own position; an aircraft surrounding environment transmission unit that transmits the surrounding conditions and the aircraft's own position as aircraft surrounding environment information; a route information receiving unit that receives route information; and a flight control unit that controls the operation of the aircraft according to the route information, wherein the surrounding conditions from the area monitoring unit include visibility information in each direction from the aircraft.

[0010] Furthermore, the present invention is characterized by "a control method that monitors the surrounding conditions of an aircraft using information detected by a sensor, determines the aircraft's own position, transmits the surrounding conditions and the aircraft's own position as aircraft surrounding environment information, receives route information, controls the aircraft's operation according to the route information, and includes visibility information in each direction from the aircraft as part of the surrounding conditions." [Effects of the Invention]

[0011] We can provide an air traffic control system and control method that can obtain the optimal flight path for each aircraft. [Brief explanation of the drawing]

[0012] [Figure 1] Figure showing an outline of facilities necessary for performing operation management of an aircraft. [Figure 2] Figure showing a configuration example of an air traffic control system configured mainly with an operation management device 10. [Figure 3] Figure showing a configuration example of an aircraft system configured mainly with an aircraft device 20. [Figure 4] Figure showing processing functions of a CPU for control by a control device (the operation management device 10 and the aircraft device 20). [Figure 5] Flowchart showing details of processing contents by the functions of a surrounding monitoring unit 207, a self-position estimation unit 203, and an aircraft surrounding environment transmission unit 204 among various functions of the aircraft device 20. [Figure 6] Figure showing detailed processing contents of processing step S207. [Figure 7] Figure showing a classification example which is a visibility determination result. [Figure 8] Figure showing a state of visibility determination on the aircraft side. [Figure 9] Figure explaining determination of a detection azimuth and an irradiation azimuth. [Figure 10] Figure showing detailed processing contents of processing step S208. [Figure 11] Figure showing an example of processing result information D1 sent from an aircraft system 200 to a control system 100. [Figure 12] Figure showing the concept of visibility map creation processing. [Figure 13] Figure showing the concept of visibility prediction map creation processing. [Figure 14] Figure showing the concept of processing by a flyable route calculation unit 105. [Figure 15] Flowchart showing detailed processing contents of an alternative processing proposal when there is not a sufficient number of aircraft. [Figure 16] Figure showing a functional configuration example of an obstacle detection unit 104 on the control system side. [Figure 17] Figure for explaining handling of uncontrolled objects. [Figure 18]1 is a flow diagram illustrating an example of a method for identifying uncontrolled targets in a control center 13. [Figure 19] Figure for explaining visibility from an uncontrolled target. [Figure 20] Figure showing a flow of processing for specifying an aircraft model. [Figure 21] Figure showing a flow of consistency determination processing with a database (DB).

Mode for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Example

[0014] Figure 1 is a diagram illustrating an overview of facilities necessary for operation management of aircraft. In Figure 1, a plurality of aircraft 4 use a port 11 as a takeoff and landing site, and fly along a planned operation route created by a control center 13.

[0015] When creating an operation route for the aircraft 4, the control center 13 creates an operation route from a start point to an end point while avoiding a no-fly area 9 above private houses 8, and at this time uses the following information obtained from sensors and the like at various locations. These are information above the port acquired by cameras 2 and radars 3 installed around the port 11, weather information from a weather company 7, wind condition information 6 acquired by a wind condition sensor 5 installed on the roof of a building 12, and airframe information acquired by the aircraft 4, etc.

[0016] In the operation management shown in Figure 1, flight operation is performed through mutual cooperation between ground facilities and the aircraft 4. In this case, the ground facilities are a control system 100 exemplified in Figure 2 that is mainly configured by an operation management device 10 in the control center 13, and the aircraft 4 is preferably equipped with an airframe system 200 that is mainly configured by an airframe device 20 exemplified in Figure 3, but the present invention is not limited to such a configuration. For example, as another configuration example, a part of functions provided on the control system side may be provided on the airframe system side.

[0017] As shown in Figures 2 and 3, the flight management system 10 and the aircraft system 20 are composed of computer devices. The main components of these devices include a CPU (processing unit) that performs calculations, a ROM (repository memory) that stores information and programs, a RAM (remote memory) that temporarily stores information during the calculation process, and a communication device T (communication device) that communicates with the outside world. The flight management system 10 also includes an input device I and a display device V. To distinguish between these, the components of the flight management system 10 are marked with the symbol A, and the components of the aircraft system 20 are marked with the symbol B.

[0018] In the control system 100 shown in Figure 2, information is input from the wind sensor 5A, camera 2A, and radar 3A, but these are all ground-mounted sensors. In contrast, the camera 2B, radar 3B, and GNSS in the aircraft system 200 shown in Figure 3 are sensors installed on the aircraft 4. The communication device TA in the control system 100 shown in Figure 2 receives camera images, radar information, and position information detected by GNSS from the communication device TB of the aircraft device 20, and the communication device TA provides flight path command information to the aircraft 4 via the communication device TB.

[0019] Figure 4 shows the processing functions of the CPUs (CPUA, CPUB) in the flight management device 10 and the aircraft device 20. Figure 4 represents the entire control system. The CPUB of the aircraft device 20 shown on the left of Figure 4 has the processing functions of a surrounding monitoring unit 207 which includes the functions of a visibility estimation unit 201 and an obstacle detection unit 202, a self-position estimation unit 203, an aircraft surrounding environment transmission unit 204, a route information reception unit 205, and a flight control unit 206. Note that the aircraft surrounding environment transmission unit 204 and the route information reception unit 205 represent the processing in the communication device TB.

[0020] The CPUA of the flight management device 10 shown on the right of Figure 4 has the following processing functions: an aircraft surrounding environment receiving unit 101, a visibility prediction unit 102, a wind condition prediction unit 103, an obstacle detection unit 104, a flight path calculation unit 105, and a path information transmission unit 106. Note that the aircraft surrounding environment receiving unit 101 and the path information transmission unit 106 represent the processing in the communication device TA.

[0021] Figure 5 is a flowchart showing the details of the processing performed by the surrounding monitoring unit 207, the self-position estimation unit 203, and the aircraft surrounding environment transmission unit 204, which are among the various functions of the aircraft device 20. In this flowchart, the aircraft 4's self-position is first obtained from GNSS in processing step S201, radar data is obtained from radar 3B in processing step S202, and camera image data is obtained from camera 2B in processing step S203.

[0022] The processing steps S204 to S210 in Figure 5 show the processing of the peripheral monitoring unit 207. Here, the processing between processing step S204 and processing step S210 is repeatedly executed while changing the number of radar irradiations. The number of radar irradiations is, for example, three. After a series of processing for the reflected wave from the first irradiation, processing for the second irradiation and reflected wave is performed, and then processing for the third irradiation and reflected wave is performed. The processing content for each time is as follows.

[0023] In processing step S205, the direction in which radar illumination will be performed is determined, and radar illumination is performed in that direction. In processing step S206, it is determined whether or not there are reflected waves from this direction. If there are reflected waves, the process moves to processing step S207 to perform visibility determination using radar and images. If there are no reflected waves, the process moves to processing step S208 to perform visibility determination using images. The detailed processing content of processing step S207 is explained separately in Figure 6, and the detailed processing content of processing step S208 is explained separately in Figure 10.

[0024] The visibility assessment results are classified into levels 1 to 4 according to the visibility conditions, as shown in Figure 7, for example. Level 1 is when the airspace is cloudy or foggy and visibility is poor, with a value of "-1". Level 2 is when there are obstacles in the airspace, with a value of "0". Level 3 is when visibility is good, with a value of "1". Level 4 is when you are in a no-visibility airspace, such as above a private residence, with a value of "-10".

[0025] In processing step S209, the required visibility level of the airspace is stored in a memory device such as ROM after each radar irradiation. Once the processing of a predetermined number of irradiations and reflected waves is complete, the process moves to processing step S211, where the self-position estimation unit 203 estimates the self-position (aircraft position) using position information from GNSS, and in processing step S211, the machine surrounding environment transmission unit 204 transmits processing result information D1 to the control system 100. An example of processing result information D1 sent from the aircraft system 200 to the control system 100 will be explained separately using Figure 11.

[0026] In response, the flight management device 10 in Figure 4 first receives processing result information D1 (visibility determination result and estimated self-position information) measured by the aircraft from the aircraft surrounding environment receiving unit 101. The wind condition prediction unit 5A predicts wind conditions, and the obstacle detection unit 104 detects obstacles. It also obtains information on the pre-set no-fly zone 9 above the private house 8. Furthermore, the visibility prediction unit 102 creates a visibility prediction map as exemplified in Figure 13 through the processing shown in Figure 12, which will be described later.

[0027] Subsequently, the flight path calculation unit 105 and the route information transmission unit 106 perform repeated processing for each aircraft, since the control system 100 typically manages multiple aircraft 104. Within the repeated processing, the flight path calculation unit 105 calculates the flight paths, and the route information transmission unit 106 transmits route data D2 to each aircraft 104.

[0028] In Figure 4, the route information receiving unit 205 of the aircraft system 200 receives the route data D2 that has been corrected and transmitted by the control system 100, and the flight control unit 206 performs the flight along the route that has been visually determined.

[0029] The processing on the aircraft system 200 side (Figure 5) and the processing on the control system 100 side are controlled in coordination, thereby enabling the acquisition of an optimal flight path for each aircraft in the present invention. The main functions of the present invention within this series of processes will be explained in more detail below.

[0030] First, in the processing step S207 of the aircraft system 200, under the condition that radar reflected waves from the direction are measured (processing step S206), visibility determination processing is performed using radar and images. The details of this process are shown in the flowchart of Figure 6.

[0031] In Figure 6, in processing step S207a, the distance to the object is measured using either or both information from the radar and / or camera images. In processing step S207b, the object is detected from the camera image information. In processing step S207c, it is determined whether or not an object was detected based on the results of the object detection process on the camera image. If an object was detected (Yes), the process proceeds to processing step S207d; if no object was detected (No), the process proceeds to processing step S207e.

[0032] In processing step S207d, it is checked whether the orientation of the object detected from the camera image is close to the direction of the reflected radar beam. If it is close, in processing step S207h, it is determined that the orientation visibility level is 2 (see Figure 7: Obstacle present). In other words, an obstacle is present if both the camera image judgment and the radar reflected wave judgment detect an object in the same direction.

[0033] Furthermore, even if the orientation of the object detected from the camera image is not close to the direction of the reflected radar beam, if the processing step S207f determines that the object's distance is closer than the threshold (No), the processing step S207h similarly determines that the orientation visibility level is 2 (see Figure 7: Obstacle present). If the processing step S207f determines that the object's distance is farther than the threshold (Yes), the processing step S207g determines that the orientation visibility level is 3 (see Figure 7: Good visibility).

[0034] Here, the processing of step S207d will be explained using Figures 8 and 9. First, Figure 8 shows the aircraft-side visibility range determination process, illustrating how the sensors (radar 3B, camera 2B) mounted on the aircraft 4 detect an object 1003, such as another aircraft. Here, 1001 is the illumination direction when radar 3B detects object 1003, and 1002 is the direction when the image from camera 2B is processed and object 1003 is detected. The proximity of the detected direction is determined by whether the angle θ between direction 1001 and direction 1002 exceeds a threshold.

[0035] Figure 9 illustrates the determination of detection direction and illumination direction. Here, the position of the aircraft 4 is shown as the origin of the three-dimensional coordinate system, and the position of the object 1003 detected by radar 3B is shown as (X0, Y0, Z0) in global coordinates. 1102 is a camera image captured by the camera, and the coordinates (u, v) represent the projection of object 1003 onto the two-dimensional camera coordinate system. 1101 is the pixel in image 1102 in which object 1003 is visible. In this case, the coordinate position of the object in the camera image can be expressed using the focal length f of camera 2B as u = f × X0 / Z0, v = f × Y0 / Z0.

[0036] In the processing steps excluding step S207e in Figure 6, the visibility level is determined using both the camera image and the reflected radar beam if object traces are present in both. In contrast, in processing step S207e for cases where no object is detected (No), the visibility level is determined when no aircraft or other objects are detected in the image. This involves determining the presence or absence of clouds or fog.

[0037] Figure 10 shows a detailed flow of processing step S207e in Figure 6. This process is identical to processing step S208 in Figure 5, and in either case, this process is used to determine the cloud and fog conditions. However, this explanation focuses on processing step S207e.

[0038] In the detailed flow of processing step S207e in Figure 6, first, in processing step S207e1, the pixel coordinates corresponding to the radar illumination direction are obtained. This means focusing on the coordinates (u,v) of point 1101 within the region of camera image 1102 in Figure 9. Next, in processing step S207e2, for example, the pixel values ​​of 25 neighboring points are obtained. This means focusing on the pixel values ​​of 25 neighboring points that include the point with coordinates (u,v), thereby extracting a small region defined by 25 pixels.

[0039] In the detailed flow of processing step S207e, it is checked whether the proportion of white pixels among the 25 pixels exceeds a threshold. If it does not exceed the threshold, it is determined that the condition of the direction of interest is dark, and therefore there is a high possibility of clouds or fog, and the visibility level for that direction is set to 1 (poor visibility due to clouds or fog) in Figure 7. Conversely, if it exceeds the threshold, it is determined that the condition of the direction of interest is bright, and therefore there is a low possibility of clouds or fog, and the visibility level for that direction is set to 3 (good visibility) in Figure 7. The color of a pixel can be determined by comparing the pixel value obtained from the camera image 1102 with an RGB color chart. Alternatively, in the case of a monochrome image, a pixel may be determined to be white if its brightness value is close to the maximum value. In this way, the surrounding monitoring unit 207 determines the surrounding conditions from the brightness and content of the camera image, as well as the camera's shooting direction and radar illumination direction, but the method for determining visibility may be other methods.

[0040] Figure 11 shows the detailed contents of processing result information D1 sent from the aircraft system 200 to the control system 100. In this example, processing result information D1 consists of aircraft ID (D1a), transmission time D1b, self-position D1c, flight speed D1d, visibility estimation result D1e, and obstacle detection result D1f. Each piece of information is further defined, and it consists of detailed measurement information such as self-position D1c being 3D coordinates, flight speed D1d being azimuth information expressed as vertical and horizontal angles and speed, visibility estimation result D1e being azimuth information including the number of azimuths and visibility level, and obstacle detection result D1f being azimuth information including the number of detections and visibility level.

[0041] The procedure for creating a visibility prediction map is described below. First, a process is performed to determine the current visibility conditions to create the visibility map shown in Figure 12. Next, a process is performed to determine the visibility conditions after a predetermined time to create the visibility prediction map exemplified in Figure 13.

[0042] Specifically, first, the self-position estimation result D1c is extracted from the received processing result information D1, and then the visibility position estimation result D1e is extracted from the received processing result information D1. This information is then used to determine the current visibility conditions.

[0043] The process of determining the current situation involves iterative calculations for all flying objects. Furthermore, within this loop, iterative calculations are performed for all directions.

[0044] Inside the loop, the voxel coordinates to which visibility is assigned are first calculated using, for example, Bresenham's algorithm. For example, for each voxel divided into a matrix of 14 horizontal and 16 vertical cells as shown in Figure 12, if the self-position of the first aircraft 4-1 is at coordinates of position 6 horizontal and 2 vertical (hereinafter simply (6,2)) as shown in Figure 12(a), then visibility position estimation result D1e was obtained for observation bearings 31, 32, and 33, and the voxel coordinates to which visibility is assigned are determined.

[0045] A similar process was repeated for the second aircraft in Figure 12(b). When the self-position of the second aircraft, 4-2, was set to the coordinates (10,7) at horizontal 10 and vertical 7 as shown in Figure 12(b), it had a visibility position estimation result D1e for observation bearings 34, 35, and 36, and the voxel coordinates to which visibility was assigned were determined.

[0046] Next, an internal loop performs iterative calculations targeting all assigned voxels. Here, visibility level information is added to the voxel coordinates to which visibility is assigned. Figure 12 shows each voxel position with information such as "0, 1, -1" assigned to it.

[0047] Through the three types of iterative processing described above, a visibility map, as shown in Figure 12(c), is formed on the matrix-like map. This map represents the current visibility state of the sky, compiled from information from multiple aircraft. The more aircraft involved, the wider and more accurately this visibility state can be grasped.

[0048] Next, we will explain the process of creating a visibility prediction map, as illustrated in Figure 13, by performing a process to determine the visibility conditions after a predetermined time.

[0049] The visibility prediction process involves iterative calculations targeting the prediction time. Within this loop, iterative calculations are also performed targeting the total number of voxels.

[0050] In the iterative process based on the predicted time and the number of voxels, the wind condition prediction results in voxel coordinates are first obtained from the wind condition prediction unit 103. In Figure 13, (a) is the current visibility map obtained in Figure 12, and (b) reflects the wind condition information for all voxels. This process forms Figure 13(b), which reflects the wind conditions at each voxel location.

[0051] Next, the voxel values ​​(levels in Figure 7) are moved based on wind direction and wind speed. Figure 13(c) is a simple superposition of (a) and (b), but by moving this in the direction corresponding to the wind direction and wind speed, the visibility after time t is obtained as the visibility prediction map in Figure 13(d). Note that the visibility prediction map may represent a fixed time period, or even a continuous time period.

[0052] When performing visibility prediction in this manner, a visibility prediction map is created by assuming a model in which the visibility level in each voxel moves to another voxel due to wind. In actual application, this is done by dividing the 3D space into voxels, but for simplicity, it is plotted in 2D here. Although a prediction map for only one time point is shown, predictions for multiple future time points can be performed using the data for each time point.

[0053] Figure 14 shows the concept of the processing of the flight path calculation unit 105. In Figure 14, (a) is the visibility map at time t, and (b) is the visibility map at time t+1. According to this, the visibility information described in each voxel at time t has moved to the position shown in (b) at time t+1. In contrast, (c) shows the planned flight path 1601 of the aircraft 4-1. According to this, the aircraft 4-1, currently at coordinates (6,2), will pass through voxel position 1602 at time t (6,6) and voxel position 1603 at time t+1. However, the visibility at all of these positions is "-1", indicating poor visibility due to clouds or fog.

[0054] The flight path calculation unit 105 determines the relationship between the flight path and visibility from the visibility prediction map, and if the relationship is as shown in Figure 14(c), it changes the route so as not to pass through the planned flight path that goes through points 1602 and 1603. This is the changed route 1604 shown in (d).

[0055] As described above, in this invention, visibility is predicted based on visibility information detected by the aircraft, and a new flight path is proposed that avoids airspace with poor visibility. [Examples]

[0056] Example 2 describes a modified alternative to the process described in Example 1. In the control-side processing described above, data is received from the aircraft 4 to create a visibility prediction map. However, depending on the conditions in the monitoring area, it is conceivable that a sufficient number of aircraft 4 may not be flying, and therefore sufficient data cannot be obtained.

[0057] Therefore, in Example 2, weather information can also be obtained from weather companies. In Example 1, the aircraft surrounding environment receiving unit 101 in Figure 4 was functioned to receive data D1 from each aircraft, but in Example 2, the processing of the aircraft surrounding environment receiving unit 101 (this processing is referred to as processing step S101) is further expanded as shown in Figure 15.

[0058] In the alternative processing plan shown in Figure 15, the position of each aircraft is first determined in processing step S101a, and the time of passage of each aircraft at the checkpoint is determined in processing step S101b.

[0059] Then, in processing steps S101c and S101d, the distance interval and time interval of the aircraft are determined, respectively. If it is determined that the distance interval is greater than the threshold, it is assumed that sufficient data cannot be obtained due to the low frequency of aircraft flights, and in processing step S101f, a visibility map is created using meteorological information obtained by the weather company. If it is determined that the distance interval is less than the threshold, it is assumed that sufficient data can be obtained due to the high frequency of aircraft flights, and in processing step S101e, a visibility map is created using information obtained by each aircraft. Thus, in processing step S101a, if the flight path of aircraft 4 is sparse, the visibility information that can be obtained will also be sparse, so visibility is predicted using data from a weather company. However, this should only be done if the weather company's data is denser than the data from aircraft 4. In addition, it is preferable to use weather data from a weather company in the following cases, such as when the distance interval between aircraft is greater than a certain amount (greater than the size of the weather company's data mesh) and when the flight time interval between aircraft is greater than a certain amount (greater than the weather company's data provision time interval).

[0060] Furthermore, compared to aircraft measurement data, the weather company's data includes at least the visibility estimation result D1e information, and is also associated with the coordinate information of the observed airspace, thus providing information on the airspace's location. Therefore, it can be used to create visibility prediction maps at the completion center. [Examples]

[0061] In Example 3, the processing at the obstacle detection unit 104 on the control system side shown in Figure 4 will be described. Figure 16 shows an example of the functional configuration of the obstacle detection unit 104, which includes a target aircraft database 104b and processing functions for a sensing unit 104a, an aircraft type estimation unit 104c, a position estimation unit 104d, a speed estimation unit 104e, an individual identification unit 104f, and a tracking unit 104g.

[0062] In the processing of the obstacle detection unit 104, the sensing unit 104a first obtains reflected wave and camera image information (sensor data) from the ground-mounted radar 3A and camera 2A. Then, from the sensor data, it extracts data of the reflected wave portion from the aircraft and image data of the aircraft as data of the area around the aircraft.

[0063] The target aircraft database 104b in Figure 16 stores information such as the shape and size of the aircraft, the planned flight path, and the planned flight time. The aircraft type estimation unit 104c uses data from the area around the aircraft to refer to the target aircraft database 104b and identify the aircraft type. The following explanation assumes that the aircraft type has been identified. The position estimation unit 104d and the speed estimation unit 104e then estimate the position and speed of the aircraft.

[0064] Next, the aircraft identification unit 104f processes the estimated and predicted positions of the aircraft by applying, for example, the Hungarian method to identify the aircraft type. This identifies the individual aircraft by using the temporal changes in the position of the identified aircraft type to match information on the flight path of an aircraft that has been known in advance. It is then determined whether the aircraft detected at the previous time point and the aircraft detected at the current time point are the same individual.

[0065] Based on these, the tracking unit 104g operates the drive units of the radar 3A and camera 2A of the sensing unit 104a to track the position of the aircraft. It is preferable to use a Kalman filter for tracking the aircraft. [Examples]

[0066] Example 4 describes how to handle uncontrolled objects. Above the landing and takeoff field, there are not only controlled aircraft 4 but also uncontrolled objects such as birds in nature. Therefore, it is necessary to distinguish whether an object in the sky is a controlled aircraft 4 or an uncontrolled object such as a bird and change the handling accordingly.

[0067] Figure 17 is a diagram illustrating the handling of uncontrolled objects. The control center 13 receives information from sensors (camera 2A, radar 3A) around the landing and takeoff area and from aircraft 4 flying overhead. In this case, sensors 2A, 3A, or aircraft whose monitoring area is 2003 (which has a monitoring area of ​​2002) will notify the control center 13 of the presence of object 2001.

[0068] Here, the response of the control center 13 is changed according to the notification result. From the perspective of ensuring safety from the uncontrolled object's side, this involves identifying the uncontrolled object. For example, an aircraft 4 that sends its own position to the control center 13 is treated as a controlled object, and if any other aircraft 4 is detected, they are treated as uncontrolled objects. In addition, the control center 13 estimates the visibility from the uncontrolled object, and if the visibility is long, the controlled aircraft 4 is prohibited from flying in the direction in which the uncontrolled object is moving. This is based on the assumption that the uncontrolled object will fly in the direction of longest visibility from its own position.

[0069] Figure 18 is a flowchart showing an example of an uncontrolled object identification method in the control center 13. This flowchart S104c is incorporated as part of the processing of the aircraft type estimation unit 104c in Figure 16 in the obstacle detection unit 104. In other words, processing step S104c is the process of identifying the aircraft type by matching with the database 104b, but the process when no matching aircraft type exists is shown as flowchart S104c in Figure 18. Note that Figure 16 explains the process assuming that the aircraft has been identified, but in reality, there may be cases where there is no match or the aircraft cannot be identified, as shown in Figure 18.

[0070] In the initial process shown in Figure 18, the first step S104c1 obtains the detection result of the obstacle detected by the obstacle detection unit in Figure 4. Based on this, the process is repeated between steps S104c2 and S104c8 until all detected obstacles have been processed. Within the loop, in step S104c3, information on the location of the detected obstacles is obtained. This information is obtained as transmitted information from the aircraft 4-1 in Figure 17, or as information from ground sensors 2A and 3A.

[0071] In processing step S104c4, information about other flying objects located near the current position is searched using the information transmitted from flying object 4-1. If object 2001 in Figure 17 is a flying object, then information about other flying objects located near the current position should exist. However, in the case of birds, this positional information does not exist. Therefore, in processing step S104c5, it is determined whether or not information about nearby objects has been found. If information has been found, it is identified as a control target in processing step S104c6; otherwise, it is identified as an uncontrollable object in processing step S104c7.

[0072] In addition, the determination in processing step S104c4 may be made based on the position detected by the ground sensor to determine whether or not there is information about a nearby position.

[0073] In the process shown in Figure 18, the uncontrolled object was recognized. However, from the perspective of protecting birds and other animals in the natural environment, or from the perspective of preventing the aircraft from crashing due to a collision, it is desirable for aircraft 4 to change its course to avoid birds and other animals in the natural environment.

[0074] Therefore, in Figure 19, the following correspondence is shown on the visibility map created assuming visibility from an uncontrolled object. The planned flight path of aircraft 4-1 is 2202, while the direction of travel of bird 2001 is estimated to be 2201. This indicates that the course of aircraft 4-1 is changed to 2203 while referring to the visibility information. However, the basis for estimating that the direction of travel of bird 2001 is 2201 is based on the assumption that birds fly in the direction with good visibility.

[0075] Figure 20 shows the flow of the process for identifying the aircraft type. In this process, even before entering the monitoring area of ​​the control center 13, the aircraft 4 can communicate with the control center 13. By using the data received at that time, it is possible to determine consistency with the flight schedule. Furthermore, as the aircraft 4 approaches the port, a camera is also used to improve the accuracy of aircraft type identification.

[0076] In the first processing step S104c10 of Figure 20, the radar detects the aircraft, and in processing step S104c11, the position of the aircraft is determined. Next, in processing step S104c12, consistency with the flight schedule is checked by referring to the target aircraft database 104b, etc. If the result is that the flight position is not listed in the target aircraft database 104b, the process moves to processing step S104c16, where it is determined to be an uncontrolled target.

[0077] If the aircraft detected by radar is consistent with the planned flight path from its current position, in processing step S104c13, it is determined whether the distance to the aircraft is within a threshold. If it is far away (No), in processing step S104c18, it is determined that the aircraft type is unknown. If it is nearby (Yes), the process moves to processing step S104c14, where the aircraft is detected and photographed by the camera.

[0078] In processing step S104c15, the captured image is compared with the information in the target aircraft database 104 to check whether there is a match. If there is no match (no corresponding information), in processing step S104c18, it is determined that the aircraft type is unknown. If there is a match, in processing step S104c17, it is determined that the aircraft type is the one that was matched. Specifically, the target aircraft database 104 stores images of target aircraft or the learning results of images, and for the determination in processing step S104c15, it is preferable to use an object detection algorithm that uses template matching with the camera image obtained in processing step S104c14 or the results of machine learning such as deep learning. For example, when using deep learning, in determining whether there is a match or not, the determination may be made that there is a match if the probability value output by the detection algorithm is greater than a threshold, and there is no match otherwise.

[0079] Figure 21 shows the flow of the consistency determination process with the target aircraft database 104b in processing step S104c12. In the first step S104c20, the flight plan for the monitoring range at that time is obtained. The flight plan consists of the aircraft ID of the aircraft scheduled to fly and the planned passing position in each time period. In the next steps S104c21~23, the detection positions of all aircraft detected in the monitoring range at that time are obtained, and the distance between the detection position of the aircraft and the planned passing position of all aircraft is calculated by brute force. In the next step S104c24, the match probability is updated Bayesianly to find the probability that the detected aircraft matches each scheduled aircraft. In the next step S104c25, the optimal value is selected from the match probabilities calculated for all combinations. The Hungarian method is suitable for this, and it is possible to output the optimal combination of aircraft ID and match probability for each detected aircraft without duplication. In the final step S104c26~30, the probability of matching is compared with a threshold. If the probability is greater than the threshold, it is determined that the aircraft matches the target aircraft database 104b; otherwise, it is determined that it does not match. [Explanation of Symbols]

[0080] 2, 2A, 2B: Camera 3, 3A, 3B: Radar 4: Flying object 5: Wind Condition Sensor 6: Wind conditions 7: Weather companies 8: Private house 9: No-Fly Zone 10:Operation control device 11: Port 12: Bill 13: Air Traffic Control Center 20: Aircraft equipment 100: Air traffic control system 101: Aircraft surrounding environment receiving unit 102: Visibility Prediction Unit 103: Wind Condition Forecasting Department 104: Obstacle detection unit 105: Flight path calculation unit 106: Route Information Transmission Unit 200: Aircraft System 201: Visibility Estimation Unit 202: Obstacle detection unit 207: Peripheral Surveillance Unit 203: Self-position estimation part 204: Aircraft surrounding environment transmission unit 205: Route information receiving unit 206: Flight Control Unit CPU: Arithmetic unit T:Communication device I: Input device V:Display device

Claims

1. A control system comprising an aircraft device and an operation management device, The aforementioned aircraft device is The system comprises: an area monitoring unit that monitors the surrounding conditions of an aircraft using information detected by a first sensor including a first radar and a first camera; a self-position estimation unit that determines the aircraft's own position; an aircraft surrounding environment transmission unit that transmits the surrounding conditions and the aircraft's own position as aircraft surrounding environment information; a route information receiving unit that receives route information; and a flight control unit that controls the operation of the aircraft according to the route information. The surrounding conditions from the surrounding monitoring unit include visibility information in each direction from the aircraft. The aforementioned operation management device is The system comprises: an aircraft surrounding environment receiving unit that receives the aforementioned aircraft surrounding environment information; a wind condition prediction unit that determines the wind conditions in the airspace in which the aircraft is flying; a visibility prediction unit that predicts the visibility information included in the aircraft surrounding environment information over time; a flight path calculation unit that calculates a possible flight path for the aircraft using the visibility and the wind conditions; a path information transmission unit that transmits the possible flight path to the aircraft as modified path information; a target aircraft database that holds data on aircraft to be monitored; a second sensor including a second radar and a second camera; and an obstacle detection unit. The visibility prediction unit superimposes multiple visibility pieces of information in three dimensions to obtain current visibility information in the airspace, and uses wind condition information to obtain future visibility information in the airspace; the flight path calculation unit modifies the path information using the flight path information and future visibility information of the aircraft; the obstacle detection unit estimates the aircraft type by referring to the data in the target aircraft database using data around the aircraft detected by the second sensor, identifies the aircraft from its flight position and speed, and tracks and monitors the aircraft. A control device characterized by the following features.

2. A control device according to claim 1, A control device characterized in that the surrounding conditions reported by the surrounding monitoring unit include information on the detection of obstacles.

3. A control device according to claim 1, The surrounding monitoring unit of the aircraft device is a control device characterized by determining the surrounding conditions from the brightness and content of the image captured by the first camera, the shooting direction of the first camera, and the illumination direction of the first radar.

4. A control device according to claim 1, The control device is characterized in that the visibility information includes information on whether the visibility is good or bad, and whether or not there are obstacles.

5. A control device according to claim 1, The control device is characterized in that the visibility prediction unit of the aforementioned flight management device performs visibility prediction using meteorological information measured by a weather company.

6. A control device according to claim 1, The control device is characterized in that the obstacle detection unit of the flight management device identifies the aircraft to be monitored and the aircraft not to be monitored, and when the aircraft is not to be monitored, modifies the route information for the aircraft to be monitored so as to avoid the aircraft not to be monitored.

7. A control method performed by a control system comprising an aircraft device and an operation management device, In the aforementioned aircraft device, the surrounding monitoring unit monitors the surrounding conditions of the aircraft using information detected by the first sensor, the self-position estimation unit determines the aircraft's own position, the aircraft surrounding environment transmission unit transmits the surrounding conditions and the aircraft's own position as aircraft surrounding environment information, the route information receiving unit receives route information, and the flight control unit controls the operation of the aircraft according to the route information, and the surrounding conditions include visibility information in each direction from the aircraft. In the aforementioned flight management device, the aircraft surrounding environment receiving unit receives the aircraft surrounding environment information, the wind condition prediction unit determines the wind conditions in the airspace in which the aircraft will fly, the visibility prediction unit predicts the visibility information included in the aircraft surrounding environment information over a period of time, the flight path calculation unit calculates the flight path for the aircraft using the visibility and the wind conditions, and the path information transmission unit transmits the flight path as modified path information to the aircraft, The visibility prediction unit superimposes multiple visibility pieces of information in three dimensions to obtain current visibility information in the airspace, and uses wind condition information to obtain future visibility information in the airspace. The flight path calculation unit modifies the flight path information using the flight path information of the aircraft and the future visibility information. The obstacle detection unit estimates the aircraft type by referencing data in the target aircraft database using data around the aircraft detected by the second sensor, identifies the aircraft from its flight position and speed, and tracks and monitors the aircraft. A control method characterized by the following features.

Images