Airplane monitoring system

Abstract

An aircraft monitoring system (10) monitors a body (120) of an aircraft (100), wherein the aircraft monitoring system includes a camera (12) mounted to the body and acquires at least a captured image captured by the camera of a moving part (40) associated with generation of lift or thrust for flight of the aircraft.

Description

Technical Field

[0001] This disclosure relates to an aircraft monitoring system. Background Technology

[0002] Inspection systems for inspecting aircraft are known in the past. For example, Patent Document 1 discloses an appearance inspection system for inspecting the exterior of an aircraft. In this appearance inspection system, a striped pattern is irradiated onto the aircraft fuselage to photograph the surface of the fuselage, and damage to the fuselage is detected based on the difference between the image and an image obtained in the same manner in the initial state before use.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2016-112947 Summary of the Invention

[0006] However, in methods like the visual inspection system in Patent Document 1, which involves photographing a pre-defined pattern projected onto the aircraft and detecting damage based on the obtained image, surface defects such as scratches or dents can be detected without visual inspection. On the other hand, there is a problem that it is impossible to confirm whether there are any malfunctions in the operation of parts that move in relation to lift or thrust generation, such as the propeller or rotor. Therefore, the confirmation of whether there are any malfunctions in the operation of parts that move in relation to lift or thrust generation, such as the propeller or rotor, is generally performed visually by the captain before flight. With the future widespread adoption of eVTOL aircraft and a dramatic increase in the number of flights, it is anticipated that it will be difficult to perform the same visual inspection of the entire aircraft as before, requiring automated and simplified visual inspection of the actuators. Furthermore, in the aforementioned methods, there is a problem that the location of the malfunction cannot be identified when a malfunction occurs during flight. In eVTOL aircraft, the number of actuators is greater than in conventional aircraft, making it even more difficult to determine the cause when a malfunction occurs during flight. Therefore, a technology that can identify malfunctions in the operation of actuators during flight is desired.

[0007] This disclosure can be implemented in the following ways.

[0008] [Method 1] An aircraft monitoring system that monitors the airframe of an aircraft, wherein the aircraft monitoring system includes: an imaging device mounted on the airframe and acquiring at least an image of a moving part associated with the generation of lift or thrust for the flight of the aircraft; and an image processing unit that uses the image to generate an overhead image of the aircraft from above the airframe.

[0009] [Method 2] An aircraft monitoring system that monitors the airframe of an aircraft, wherein the aircraft monitoring system includes: an imaging device mounted on the airframe and acquiring at least one image of an action unit associated with the generation of lift or thrust for the flight of the aircraft; an action control unit that controls the imaging device; and an action unit normality determination unit that causes the action unit to move by sending action command information for the action unit to an airframe control unit that causes the action unit to move, and determines the normality of the action of the action unit based on the action command information and the image, wherein the imaging device captures the action unit in chronological order and acquires a plurality of the imaged images when the action unit moves according to the action command information, and the action unit normality determination unit uses the plurality of imaged images acquired by the imaging device to determine the normality of the action.

[0010] [Method 3] An aircraft monitoring system that monitors the airframe of an aircraft, wherein the aircraft monitoring system includes an imaging device mounted on the airframe, and acquires at least images of moving parts associated with the generation of lift or thrust for the flight of the aircraft. The imaging device also captures images of lighting devices mounted on the airframe, and further activates the lighting devices by sending lighting command information instructing the lighting devices to be turned on and off to an airframe control unit that activates the moving parts. The aircraft monitoring system also includes a lighting device normality determination unit, which determines the normality of the lighting devices based on the lighting command information and the lighting device... The system uses a captured image to determine the normality of the operation of the lighting device. The lighting device normality determination unit performs at least one of the following two determinations: if the brightness of the lighting device in the captured image obtained after any time from the moment when the lighting instruction information indicating that the lighting device is sent to the lighting device is sent is greater than or equal to a predetermined first threshold, the lighting device is determined to be normal; and if the brightness of the lighting device in the captured image obtained after any time from the moment when the lighting instruction information indicating that the lighting device is sent to the lighting device is sent is less than or equal to a second threshold that is less than the first threshold, the lighting device is determined to be normal.

[0011] As one aspect of this disclosure, an aircraft monitoring system is provided. This aircraft monitoring system monitors the airframe of an aircraft, and includes an imaging device mounted on the airframe, which acquires images of at least the moving parts associated with the generation of lift or thrust for the flight of the aircraft.

[0012] According to this aircraft monitoring system, since it includes an imaging device mounted on the aircraft body, and acquires images of at least the actuators associated with the generation of lift or thrust for the aircraft's flight, it is possible to confirm the presence or absence of malfunctions in the actuators associated with the generation of lift or thrust for the aircraft's flight without visual inspection during pre-flight maintenance or when the aircraft is landing. Furthermore, during flight, it is possible to confirm the malfunction status of the actuators associated with the generation of lift or thrust for the aircraft's flight. Therefore, during aircraft landing and flight, it is possible to confirm the presence and status of malfunctions in the aircraft's actuators.

[0013] This invention can also be implemented in various ways. For example, it can be implemented as a control method for an aircraft monitoring system that can monitor the aircraft's airframe. Attached Figure Description

[0014] The foregoing and other objects, features, and advantages of this disclosure will become more apparent from the accompanying drawings and the detailed description below. The drawings are described below.

[0015] Figure 1 This is a schematic diagram showing an electric vertical takeoff and landing aircraft equipped with an aircraft monitoring system according to one embodiment of the present disclosure.

[0016] Figure 2 This is a block diagram representing the structure of an aircraft monitoring system.

[0017] Figure 3 It is a block diagram representing the structure of the control device.

[0018] Figure 4 This is a flowchart representing the steps involved in the action confirmation process. Detailed Implementation

[0019] A. First implementation method:

[0020] A-1. System Structure:

[0021] like Figure 1 As shown, an aircraft monitoring system 10, as one embodiment of this disclosure, is installed on an electric vertical take-off and landing (eVTOL) aircraft 100 (hereinafter also referred to as "eVTOL 100"). In this embodiment, the aircraft monitoring system 10 monitors the eVTOL 100. First, the eVTOL 100 will be described.

[0022] The eVTOL 100 is configured as an electrically driven manned aircraft capable of taking off and landing in the vertical direction. For example... Figure 1 As shown, the eVTOL 100 includes: a main body 120; a plurality of rotors 130; and a plurality of electric drive systems 110 (hereinafter also referred to as "EDS (Electric Drive System) 110") for driving each rotor 130 to rotate. The eVTOL 100 of this embodiment includes six rotors 130, six EDS 110, and six pitch drive units (not shown). Furthermore, the number of rotors 130, EDS 110, and pitch drive units is not limited to six; any number, as long as there is one or more, is acceptable.

[0023] The main body 120 corresponds to the part of the eVTOL 100 excluding the six rotors 130, EDS 110, and pitch drive unit. The main body 120 includes a main body 121, a strut section (not shown), six first support sections 123, six second support sections 124, a main wing 125, and a tail fin 128.

[0024] The main body 121 constitutes the trunk of the eVTOL 100. A passenger compartment (not shown) is formed inside the main body 121. A support column (not shown) has a generally columnar shape extending vertically and is fixed to the upper part of the fuselage main body 121. In this embodiment, the support column is positioned to overlap with the center of gravity of the (not shown) main body of the eVTOL 100 when viewed vertically. Six first support portions 123 are fixed to one end of each of the six support columns. Each of the six first support portions 123 has a generally rod-shaped shape and is arranged radially at equal angular intervals, extending along a plane perpendicular to the vertical direction. A rotor 130 and an EDS 110 are respectively disposed at the other end of each first support portion 123, i.e., the end furthest from the support column. Six second support portions 124 each have a generally rod-shaped shape and connect the other ends (the ends not connected to the support columns) of adjacent first support portions 123 to each other.

[0025] The main wing 125 includes a right wing 126 and a left wing 127. The right wing 126 extends to the right from the main fuselage section 121. The left wing 127 extends to the left from the main fuselage section 121. The tail fin 128 is formed at the rear end of the main fuselage section 121.

[0026] Six rotor blades 130 are disposed at the ends of each of the second support sections 124. During takeoff and landing, they generate thrust in the vertical direction and operate primarily as lifting rotors to obtain lift from the main body section 120. Then, via a pitch drive, they deflect in a direction capable of generating thrust in the propulsion direction and operate primarily as cruise rotors to obtain thrust from the main body section 120. Each rotor blade 130 is driven to rotate independently around its own axis of rotation. Each rotor blade 130 has three blades 133 arranged at equal angular intervals.

[0027] The six EDS 110s constitute a drive unit for driving each rotor 130 to rotate. The pitch drive unit is a drive unit for changing the direction of the thrust generated by the rotor 130.

[0028] Next, the aircraft monitoring system 10 will be described. For example... Figure 2 As shown, the aircraft monitoring system 10 includes a camera 12, an image processing unit 20, a control unit 30, and an overhead image display unit 50.

[0029] like Figure 1 and Figure 2 As shown, in this embodiment, the imaging device 12 includes four cameras 12a to 12d. The four cameras 12a to 12d are mounted on the main body 120. Specifically, the four cameras 12a to 12d are arranged and mounted above a support column (not shown) in a manner that allows them to shoot in different directions. Figure 1 As shown, the shooting ranges Ar1 to Ar4 of each camera 12a to 12d are represented by dashed lines or single-dot lines, respectively. Cameras 12a to 12d capture images by including the actuator 40 of the eVTOL 100, and acquire the captured images obtained through this capture. "Actuator" refers to a part that operates in relation to the generation of lift or thrust for the flight of the eVTOL 100. In this embodiment, "part that operates in relation to the generation of lift or thrust for the flight of the eVTOL 100" refers to a part capable of generating lift or thrust for the flight of the eVTOL 100, and a part that operates to generate lift or thrust for the flight of the eVTOL 100. Examples of actuator 40 include a rotor 130 and ailerons (not shown), elevators, rudders, EDS 110, and a pitch drive unit. Furthermore, the actuator 40 is not limited to the rotor 130 and ailerons, elevators, rudders, etc. (not shown), the EDS 110, and the pitch drive unit; it can be any type of component that operates in relation to the generation of lift or thrust for flight of the eVTOL 100.

[0030] The image processing unit 20 uses the captured image obtained in the shooting device 12 to generate an overhead image of the eVTOL 100 from above the main body 120.

[0031] In this embodiment, the overhead view display unit 50 is installed in the cockpit of the eVTOL 100. In this embodiment, the overhead view display unit 50 is composed of a liquid crystal display device. The overhead view display unit 50 displays an overhead view image generated by the image processing unit 20.

[0032] The control device 30 controls the overall operation of the aircraft monitoring system 10. For example... Figure 3 As shown, the control device 30 is configured as a computer including a storage unit 36, an input / output interface 37, and a CPU (Central Processing Unit). The storage unit 36 ​​has ROM (Read Only Memory) and RAM (Random Access Memory).

[0033] The input / output interface 37 is used to input / output command values ​​or output values ​​between the control device 30 and an external device. For example, the input / output interface 37 inputs an action confirmation start command from an external device. This external device could be a computer used for management and control, such as a server device that performs action confirmation control or records action confirmation results. This management and control computer could be, for example, a server device located in an air traffic control room, or a personal computer brought to the eVTOL 100's operating location by a maintenance worker performing action confirmation. Furthermore, action command information (such as a pitch angle command value relative to the pitch drive unit) input via the input / output interface 37 is sent from the control device 30 to the airframe control unit 60. Then, the airframe control unit 60 activates the action unit 40.

[0034] The CPU functions as the motion control unit 31, the motion normality judgment unit 32, and the obstacle determination unit 33 by executing the control program pre-stored in the storage unit 36.

[0035] The motion control unit 31 controls the shooting device 12. For example, the motion control unit 31 sends a shooting instruction to the shooting device 12.

[0036] The motion unit normality determination unit 32 causes the motion unit 40 to move by sending motion command information to the machine control unit 60. The motion unit normality determination unit 32 determines the normality of the motion of the motion unit 40 based on the motion command information and the captured image obtained by the imaging device 12.

[0037] The obstacle detection unit 33 determines whether there are obstacles within a predetermined distance range from the action unit 40. The determination of the presence or absence of obstacles will be described in detail later.

[0038] A-2. Action Confirmation Processing:

[0039] In this embodiment, when an action confirmation start command is input to the aircraft monitoring system 10 via the input / output interface 37, the following is executed: Figure 4 The action confirmation process shown is the process used to confirm the action of the action unit 40 via the aircraft monitoring system 10.

[0040] The motion control unit 31 sends a shooting instruction to the shooting device 12 (step S10). The shooting device 12 takes pictures of the motion unit 40 and its surroundings and acquires multiple surrounding images (step S12). These multiple surrounding images refer to multiple images taken by multiple cameras at approximately the same time.

[0041] The image processing unit 20 uses multiple surrounding images to generate an overhead image of the eVTOL 100 viewed from above the main body 120 (step S14).

[0042] The obstacle determination unit 33 determines whether there are obstacles within a predetermined distance range from the action unit 40 based on the overhead view generated in step S14 (step S16). For example, the obstacle determination unit 33 uses the overhead view to determine whether there are people or objects within 1m (meters) of the rotor.

[0043] If an obstacle is determined to exist (step S18: Yes), the process returns to before step S18. In other words, the process does not proceed to steps after S18 while an obstacle exists. If an obstacle is determined not to exist (step S18: No), the motion unit normality determination unit 32 causes the motion unit 40 to operate by sending motion command information to the machine control unit 60 (step S20).

[0044] The imaging device 12 takes pictures of the motion unit 40 and its surroundings a predetermined number of times at predetermined time intervals in chronological order (step S22). The image processing unit 20 uses the multiple images captured in step S22 to generate an overhead image of the eVTOL 100 viewed from above the main body 120 (step S24). In step S24, the image processing unit 20 generates the overhead image using multiple images captured by multiple cameras at approximately the same time. The image processing unit 20 generates the overhead image at approximately the same time intervals using the multiple images captured by multiple cameras in chronological order in step S22.

[0045] The motion unit normality determination unit 32 determines the normality of the motion of the motion unit 40 based on the motion command information and the overhead view generated in step S24 (step S26). Specifically, for example, when the pitch drive unit is instructed to set the pitch angle to 45 degrees, the unit determines whether the pitch angle of the motion unit 40 driven by the pitch drive unit is tilted to 45 degrees in multiple overhead views obtained in chronological order.

[0046] The motion unit normality determination unit 32 stores the determination result of the normality of the motion of the motion unit 40 in the storage unit 36 ​​(step S28). The determination result of the normality of the motion unit 40 is displayed on the overhead view display unit 50 (step S30). Specifically, for each motion unit, the result indicating normal or abnormal is displayed by text.

[0047] In this embodiment, the main body 120 corresponds to the body in the claims.

[0048] The aircraft monitoring system 10 of this embodiment described above includes an imaging device 12 mounted on the main body 120. It acquires images of the actuators 40 associated with the generation of lift or thrust for the flight of the eVTOL 100. Therefore, during pre-flight maintenance or when the eVTOL 100 has landed, it is possible to confirm whether there are any faults in the actuators associated with the generation of lift or thrust for the flight of the aircraft without visual inspection.

[0049] In addition, the image processing unit 20 uses the captured image obtained by the imaging device 12 to generate an overhead image of the eVTOL 100 from above the main body 120, and displays the overhead image on the overhead image display unit 50. Therefore, the operator or captain can intuitively confirm whether there is any malfunction in the operation of the operating unit 40 of the eVTOL 100.

[0050] Furthermore, when the motion unit 40 performs an action according to the motion command information, the motion unit 40 is photographed in chronological order and multiple photographed images are acquired. The normality of the action is determined using these multiple photographed images. Therefore, compared with the structure that uses a single photographed image, the accuracy of normality determination is improved.

[0051] Furthermore, before the action unit 40 is activated, the obstacle determination unit 33 determines whether there are any obstacles within a predetermined distance from the action unit 40 based on the surrounding images captured by the action unit 40 and its surroundings. Therefore, it can be linked to suppressing human error in safety confirmation and improve safety during inspection.

[0052] B. Other implementation methods:

[0053] (B1) In the aircraft monitoring system 10 of this embodiment, the imaging device 12 also captures images of the lighting device installed on the main body 120, and includes a lighting device normality determination unit. This lighting device normality determination unit further activates the lighting device by sending lighting command information to the airframe control unit 60, which activates the actuation unit 40, instructing the lighting device to turn on and off. Based on the lighting command information and the captured images obtained by capturing images of the lighting device, the unit determines the normality of the lighting device's operation. The lighting device normality determination unit can also perform at least one of the following two determinations: if the brightness of the lighting device in the captured image obtained after any time from the moment the lighting command information indicating that the lighting device is sent is greater than or equal to a predetermined first threshold, the lighting device is determined to be normal; and if the brightness of the lighting device in the captured image obtained after any time from the moment the lighting command information indicating that the lighting device is sent is less than or equal to a second threshold smaller than the first threshold, the lighting device is determined to be normal. Since the normality of the lighting device can be determined based on this quantitative value of the brightness of the lighting device, a quantitative check of normality can be performed.

[0054] (B2) In the aircraft monitoring system 10 of this embodiment, in addition to the overhead view, or as an alternative to the overhead view, it is also possible to display a captured image obtained by the capturing device 12, i.e., an image in the state before image processing to the overhead view. This makes it easier to locally display malfunctions in the operation of the operating unit 40 of the eVTOL 100. In this configuration, the overhead view display unit 50 is equivalent to the captured image display unit. Alternatively, a captured image display unit, different from the overhead view display unit 50, can be added to display captured images.

[0055] (B3) The aircraft monitoring system 10 of this embodiment is installed in the eVTOL 100, but is not limited to the eVTOL 100. For example, it can also be installed in any type of aircraft such as a jet or a helicopter. In addition, the eVTOL 100 is configured as a manned aircraft, but it can also be configured as an unmanned aircraft.

[0056] (B4) In the aircraft monitoring system 10 of this embodiment, the imaging device 12 is mounted on the main body 120 and acquires images of the actuator 40 and the area around the actuator 40 that are associated with the generation of lift or thrust for flight of the eVTOL 100. However, it is also possible to acquire images that only capture images of the actuator 40. In other words, the imaging device 12 only needs to acquire images that capture images of the actuator 40.

[0057] (B5) The aircraft monitoring system 10 of this embodiment includes an image processing unit 20, a top-down image display unit 50, a motion control unit 31, a motion unit normality judgment unit 32, and an obstacle determination unit 33, but these components may not be included. In other words, it is sufficient to include the imaging device 12. The imaging device 12 can also be used to capture images of the motion unit 40, and the operator or others can use the acquired images to judge the normality of the motion unit 40.

[0058] (B6) In the aircraft monitoring system 10 of this embodiment, the cameras 12a to 12d of the imaging device 12 are arranged above a support section (not shown), but they may also be arranged at a different position than the support section. They may also be distributed in parts of the main body 120 away from the support section. For example, the cameras may be distributed in the forward direction side and the backward direction side of the fuselage main body 121. In addition, multiple cameras may be concentrated on the tail 128.

[0059] (B7) In the aircraft monitoring system 10 of this embodiment, the shooting device 12 includes four cameras 12a to 12d, but is not limited to four, and may include one or more cameras of any number.

[0060] (B8) In the aircraft monitoring system 10 of this embodiment, the image processing unit 20 generates an overhead image of the eVTOL 100 viewed from above the main body 120, but this disclosure is not limited to this. Not only overhead images from above, but cameras can also be positioned to acquire overhead images from all directions. Cameras with the precision to detect damage such as dents or scratches on the aircraft surface can also be included, thereby enabling not only the confirmation of malfunctions in the actuators but also the implementation of aircraft damage detection.

[0061] (B9) In the action confirmation process of the aircraft monitoring system 10 in this embodiment, for each action, the result indicating normal or abnormal is displayed in the overhead image display unit 50 using text. However, for each action in the overhead image, a color-coded display indicating normal or abnormal may also be performed.

[0062] (B10) In the aircraft monitoring system 10 of this embodiment, the motion unit 40 is activated and motion confirmation processing is performed when the eVTOL 100 is stopped. However, motion confirmation processing can also be performed during flight of the eVTOL 100. Specifically, during flight of the eVTOL 100, if a malfunction is detected in any of the motion units 40, the motion control unit can instruct the imaging device 12 to take a picture of the location where the malfunction occurred, and display the picture result on the overhead view display unit 50, thereby confirming the malfunction condition.

[0063] This disclosure is not limited to the embodiments described above, and can be implemented through various structures without departing from the above-described spirit. For example, the technical features in each embodiment corresponding to the technical features described in the summary section can be appropriately replaced or combined to solve part or all of the above-described technical problems, or to achieve part or all of the above-described effects. Furthermore, the above-described technical features can be appropriately deleted unless they are described as essential structures in this specification.

[0064] The control device and method described in this disclosure can also be implemented using a dedicated computer, which is provided by comprising a processor and memory, the processor being programmed to perform one or more functions embodied in the computer program. Alternatively, the control device and method described in this disclosure can be implemented using a dedicated computer, which is provided by comprising a processor composed of one or more dedicated hardware logic circuits. Alternatively, the control device and method described in this disclosure can be implemented using one or more dedicated computers, which are constituted by a combination of a processor and memory programmed to perform one or more functions and a processor composed of one or more hardware logic circuits. Furthermore, the computer program can also be stored on a computer-readable non-transitory tangible storage medium as instructions to be executed by a computer.

Claims

1. An aircraft monitoring system that monitors a machine body of an aircraft in a landing transition state, characterized by comprising: a photographing device that is installed to the machine body and acquires at least a photographed image obtained by photographing a moving part associated with generation of lift or thrust for flight of the aircraft; a moving control part that controls the photographing device; and a moving part normality judgment part that causes the moving part to move by transmitting moving instruction information for the moving part to a machine body control part that causes the moving part to move, and judges normality of movement of the moving part based on the moving instruction information and the photographed image, the photographing device acquires a plurality of the photographed images in time series when the moving part moves according to the moving instruction information, the moving part normality judgment part judges the normality of movement using the plurality of the photographed images acquired by the photographing device, the moving part includes a pitch driving part, and the photographing device photographs the pitch driving part in time series when a command value of a pitch angle as the moving instruction information is instructed to the pitch driving part.

2. The aircraft monitoring system according to claim 1, characterized by comprising an image processing part, and the image processing part generates an overlooking image overlooking the aircraft from above the machine body using the photographed image.

3. The aircraft monitoring system according to claim 2, characterized by further comprising an overlooking image display part that displays the overlooking image.

4. The aircraft monitoring system according to claim 1, characterized by the photographed image is acquired by the photographing device photographing surroundings of the moving part in addition to the moving part, the aircraft monitoring system further comprises an obstacle determination part that determines whether or not there is an obstacle within a predetermined distance range from the moving part based on the surroundings image, and the moving part normality judgment part causes the moving part to move by transmitting the moving instruction information to the machine body control part in a case where the obstacle determination part determines that there is no obstacle.

5. The aircraft monitoring system according to any one of claims 1 to 4, characterized by the photographing device further photographs a lighting device installed to the machine body, the lighting device is further caused to move by transmitting lighting instruction information instructing lighting up and lighting out of the lighting device to the machine body control part that causes the moving part to move, the aircraft monitoring system further comprises a lighting device normality judgment part that judges normality of movement of the lighting device based on the lighting instruction information and a photographed image obtained by photographing the lighting device, and the lighting device normality judgment part performs at least one of the following two determinations: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ In a case where the brightness of the lighting device in the captured image obtained after an arbitrary time from the time when the lighting instruction information indicating lighting is transmitted to the lighting device is equal to or greater than a predetermined first threshold value, it is determined that the lighting device is normal; and In a case where the brightness of the lighting device in the captured image obtained after an arbitrary time from the time when the lighting instruction information indicating extinguishing is transmitted to the lighting device is equal to or less than a predetermined second threshold value that is smaller than the first threshold value, it is determined that the lighting device is normal.

6. The aircraft monitoring system according to any one of claims 1 to 4, wherein the aircraft monitoring system further includes a captured image display section that displays a captured image obtained by the capturing of the capturing device.

7. The aircraft monitoring system according to any one of claims 1 to 4, wherein the action control section instructs the capturing device to capture a failure occurrence position when a failure of the action section is detected.

Images