Detection system for detecting boarding and disembarking areas of an aircraft

Abstract

This detection system comprises: a passenger boarding bridge; a first camera and a second camera that are provided on the passenger boarding bridge; a height change mechanism; a controller; and an image processor that detects the boarding section of an aircraft from a first captured image captured by the first camera and a second captured image captured by the second camera. The image processor generates a first search area image from the first captured image and generates a second search area image from the second captured image. The first search area image has a wider area in the vertical direction than the second search area image. If the image processor detects the boarding section from the first search area image and does not detect the boarding section from the second search area image, the controller controls the height change mechanism to change the height position of the first camera and the second camera.

Description

[Technical Field]

[0001] The present disclosure relates to a detection system for detecting passenger access points on an aircraft. [Background technology]

[0002] Passenger boarding bridges are known as facilities that serve as pedestrian walkways for passengers between airport terminal buildings and aircraft. Passenger boarding bridges include a rotunda connected to the terminal building and supported so as to be freely rotatable horizontally; a tunnel section whose base end is connected to the rotunda and configured to be freely extendable; a cab that is rotatably mounted at the end of the tunnel section and attached to the aircraft boarding and disembarking section; and a drive column that is provided as a support leg near the end of the tunnel section. The drive column includes an elevator device that supports and moves the tunnel section up and down, and a traveling device that is provided below the elevator device and has a pair of traveling wheels. Proposals have been made to automate the movement of such passenger boarding bridges (see, for example, Patent Documents 1 and 2 listed below).

[0003] For example, Patent Document 1 describes a method in which a camera that photographs the boarding and disembarking area of ​​an aircraft is attached to the cab, and when the cab is in a designated waiting position, horizontal position information of the boarding and disembarking area is calculated based on an image of the area photographed by the camera, and a target position to which the cab should be moved in order to attach it to the boarding and disembarking area is calculated based on this horizontal position information, and the cab in the waiting position is moved toward the target position.

[0004] Patent Document 2 also describes a configuration in which a head unit (cab) connectable to an aircraft boarding / alighting door is equipped with two cameras, a first and a second. When a drive start input is given on a control panel, the travel drive unit starts wheel travel, and when the head unit reaches a few meters in front of the aircraft, the first and second cameras start capturing images of a first characteristic part and a second characteristic part of the aircraft. The document describes a method in which the position of a target point at the aircraft boarding / alighting door is calculated using the images captured by the first and second cameras, the relative position and relative angle of the head unit with respect to the aircraft boarding / alighting door are calculated, a control correction amount is calculated based on these, and various drive units are driven based on this to move the head unit toward the target point on the aircraft. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6720414 [Patent Document 2] Japanese Patent Publication No. 2020-175727 Summary of the Invention [Problem to be solved by the invention]

[0006] In order to automate the movement of such passenger boarding bridges, it is necessary to accurately detect the position of the boarding and disembarking areas of aircraft from images captured by a camera installed at the tip of the passenger boarding bridge. In particular, the position of the boarding and disembarking areas of aircraft can vary greatly depending on the type and size of the parked aircraft. Furthermore, airport gate layouts vary widely, and it is cumbersome to pre-set a mode for detecting the position of the boarding and disembarking areas for each combination of parked aircraft and gate layout. As such, there is room for improvement in the above-mentioned Patent Documents 1 and 2 in terms of accurately detecting the positions of the boarding and disembarking areas of multiple types of aircraft in response to various gate layouts.

[0007] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a detection system that can accurately detect the position of an aircraft's boarding and disembarking area in accordance with various gate arrangements, using images taken by a camera installed at the tip of a passenger boarding bridge. [Means for solving the problem]

[0008] A detection system according to an aspect of the present disclosure is a detection system for detecting boarding and disembarking sections of an aircraft, the detection system comprising: a passenger boarding bridge connected to a terminal building; a first camera and a second camera provided on the passenger boarding bridge; a height change mechanism capable of changing the height positions of the first camera and the second camera; a controller for controlling the operation of the height change mechanism; and an image processing device for detecting the boarding and disembarking section of the aircraft from a first photographed image taken by the first camera and a second photographed image taken by the second camera, wherein a first angle of a first virtual line connecting the optical center of the first camera and a predetermined position on the object side photographed by the first camera with respect to a horizontal plane is determined by the optical angle of the second camera. The first camera and the second camera are arranged so that they form an angle different from a second angle with respect to the horizontal plane of a second virtual line connecting the academic center and the specified position, the image processing device generates a first search area image from the first captured image and generates a second search area image from the second captured image, the first search area image having a wider area in the vertical direction than the second search area image, and the controller controls the height change mechanism to change the height positions of the first camera and the second camera when the image processing device detects the boarding and alighting area from the first search area image but does not detect the boarding and alighting area from the second search area image. [Effects of the Invention]

[0009] According to the present disclosure, the position of an aircraft's boarding and disembarking area can be accurately detected in accordance with various gate arrangements using images captured by a camera installed at the tip of a passenger boarding bridge. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic plan view showing an example of a passenger boarding bridge to which the detection system according to this embodiment is applied. [Figure 2] FIG. 2 is a diagram showing the positional relationship between the first camera and the second camera in this embodiment. [Figure 3] FIG. 3 is a block diagram showing a schematic configuration of a detection system applied to the passenger boarding bridge shown in FIG. [Figure 4] FIG. 4 is a flowchart showing the flow of the search process in this embodiment. [Figure 5] FIG. 5 is a flowchart showing the flow of the search process in this embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a captured image in this embodiment. [Figure 7] FIG. 7 is a conceptual diagram for setting the first and second regions in the photographed image shown in FIG. [Figure 8] FIG. 8 is a diagram showing an image after projective transformation of the first region in the first captured image in this embodiment, compared with an image before projective transformation. [Figure 9] FIG. 9 is a diagram showing an image after projective transformation of the second region in the second captured image in comparison with an image before projective transformation in this embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of how the shooting areas in the first search area image and the second search area image overlap in this embodiment. [Figure 11] FIG. 11 is a diagram showing the positional relationship between the first camera and the second camera in the first modification. [Figure 12] FIG. 12 is a diagram showing the positional relationship between the first camera and the second camera in the second modification. DETAILED DESCRIPTION OF THE INVENTION

[0011] Preferred embodiments of the present disclosure will be described below with reference to the drawings. Note that the same or corresponding elements will be designated by the same reference numerals throughout the drawings, and redundant explanations will be omitted. The drawings are schematic illustrations of the respective components for ease of understanding, and the shapes, dimensional ratios, and the like may not be accurately depicted. The present disclosure is not limited to the following embodiments.

[0012] FIG. 1 is a schematic plan view showing an example of a passenger boarding bridge to which the detection system according to this embodiment is applied.

[0013] The passenger boarding bridge 11 comprises a rotunda (base circular chamber) 4 connected to the boarding / alighting entrance of the airport terminal building 2, a tunnel section 5 whose base end is connected to the rotunda 4 and configured to be freely elevated and extendable in the longitudinal direction, and a cab (tip circular chamber) 6 connected to the tip of the tunnel section 5. The rotunda 4 is configured to be freely rotatable horizontally around a first rotation axis R4 extending vertically. The cab 6 is configured to be freely rotatable horizontally around a second rotation axis R6 extending in a direction perpendicular to the bottom surface of the cab 6.

[0014] The tunnel section 5 forms a walkway for passengers, and is configured by a plurality of cylindrical tunnels 5a, 5b that are nested together to allow for free extension and contraction in the longitudinal direction. Note that while the tunnel section 5 is illustrated here as being configured with two tunnels 5a, 5b, the tunnel section 5 may be configured with two or more tunnels. The base end of the tunnel section 5 is connected to the rotunda 4 so that it can swing freely (up and down) around a horizontal rotation axis, and is therefore connected to the rotunda 4 so that it can be freely raised and lowered.

[0015] Furthermore, a drive column 7 is provided as a support leg at the tip of the tunnel section 5 (tunnel 5a on the tip side). The drive column 7 may be attached to the cab 6. The drive column 7 is provided with a lifting device 8 that raises and lowers the cab 6 and the tunnel section 5. This allows the cab 6 and the tunnel section 5 to swing up and down with the rotunda 4 as the base point. As a result, the orientation of the second rotation axis R6, which is the rotation axis of the cab 6, changes depending on the inclination of the bottom surface of the cab 6 with respect to the horizontal plane. In other words, the second rotation axis R6 extends in a direction intersecting the horizontal plane, and its direction changes depending on the up and down swing of the cab 6 and the tunnel section 5.

[0016] The drive column 7 also includes a travel device 9 for rotating the tunnel section 5 around the rotunda 4 (around the first rotation axis R4) and extending or retracting the tunnel section 5. The travel device 9 is provided below the lifting device 8. The travel device 9 has, for example, two travel wheels that can be driven independently in forward and reverse directions. Forward or backward travel is possible by rotating the two travel wheels in the same direction (forward or reverse), and the travel direction (direction of the travel wheels) can be changed on the spot by rotating the two travel wheels in opposite directions. Note that in this embodiment, a configuration is illustrated in which the tunnel section 5 rotates around the first rotation axis R4 as the rotunda 4 itself rotates around the first rotation axis R4. However, instead, the rotunda 4 may be fixed to the terminal building 2, and the tunnel section 5 may be connected to the rotunda 4 so as to be rotatable around the first rotation axis R4.

[0017] The cab 6 has a connection part 6a at its tip that is connected to the boarding and disembarking sections D1, D2 of the aircraft 3. A closure, a bumper, a distance sensor, etc. are provided on the connection part 6a. The position of the connection part 6a (the direction of connection to the boarding and disembarking sections D1, D2) can be changed by rotating the cab 6 about the second rotation axis R6. Note that instead of rotating the cab 6, the connection part 6a may rotate relative to the cab 6 about the second rotation axis R6.

[0018] Additionally, cameras 21a and 21b are installed in the cab 6 for photographing the side of the aircraft 3. The cameras 21a and 21b are provided at the front end of the cab 6. In the example of FIG. 1, the first camera 21a is installed at the upper inside of the connection portion 6a in the cab 6. The second camera 21b is installed at the lower inside of the connection portion 6a in the cab 6. That is, the first camera 21a is disposed higher than the second camera 21b. Additionally, the first camera 21a is disposed at a position in the cab 6 in the fore-and-aft direction (first distance K1 from the connection portion 6a shown in FIG. 2) that is the same as the position in the cab 6 in the fore-and-aft direction of the second camera 21b (second distance K2 from the connection portion 6a shown in FIG. 2).

[0019] The positions of the first camera 21a and the second camera 21b may be any position other than inside the cab 6 as long as they are positions at the front end of the cab 6 that allow them to photograph the side of the aircraft 3. For example, the first camera 21a and the second camera 21b may be installed on the upper and lower exterior parts of the cab 6.

[0020] In this embodiment, the positional relationship between first camera 21a and second camera 21b is determined as follows. Fig. 2 is a diagram showing the positional relationship between the first camera and the second camera in this embodiment. As shown in Fig. 2, first camera 21a and second camera 21b are disposed so that a first angle δ1, made with respect to a horizontal plane H, of a first virtual line V1 connecting the optical center of first camera 21a and a predetermined position VO on the object side photographed by first camera 21a is different from a second angle δ2, made with respect to the horizontal plane H, of a second virtual line V2 connecting the optical center of second camera 21b and the predetermined position VO.

[0021] The predetermined position VO on the object side may be set to a predetermined three-dimensional position in a three-dimensional coordinate system with the first camera 21a as the origin, for example. Alternatively, the predetermined position VO on the object side may be set to a predetermined three-dimensional position in a three-dimensional coordinate system with a predetermined position on the apron EP as the origin, for example. The three-dimensional position of the predetermined position VO may be set assuming a detection position (a reference point of a door, described later) of the boarding and disembarking section D1 of the aircraft 3 when the boarding and disembarking section D1 is located in front of the connecting section 6a. In this embodiment, the predetermined position VO is set to a position below the cab 6, so that the first angle δ1 is greater than the second angle δ2.

[0022] In this embodiment, a search is performed for the boarding and disembarking area D1 of the aircraft 3 to which the passenger boarding bridge 11 should connect based on a first captured image G1 taken by a first camera 21a provided at the tip of the passenger boarding bridge 11 and a second captured image G2 taken by a second camera 21b.

[0023] Fig. 3 is a block diagram showing a schematic configuration of a detection system applied to the passenger boarding bridge shown in Fig. 1. As shown in Fig. 3, the detection system 20 in this embodiment includes the cameras 21a and 21b and an image processing device 23 that detects the boarding and disembarking section D1 of the aircraft 3 from the images G1 and G2 captured by the cameras 21a and 21b. The detection system 20 further includes a storage device 24 that stores data such as the captured images and an image processing program, and an output device 25 that outputs the detection results, etc. These components 21a, 21b, 23, 24, and 25 can exchange data with each other via a bus 26.

[0024] Also connected to the bus 26 is a controller 30 that controls the passenger boarding bridge 11. The controller 30 is provided, for example, in the cab 6 or the tunnel 5a at the forefront. The controller 30 controls the rotation of the cab 6 of the passenger boarding bridge 11 and the elevation and travel of the drive column 7. Furthermore, the controller 30 controls the rotation of the height change mechanism 36 and the shooting direction change mechanism 34 of the cameras 21a and 21b.

[0025] The height positions of the cameras 21a and 21b are adjustable. To this end, the detection system 20 includes a height changing mechanism 36 that can change the height positions of the cameras 21a and 21b. In this embodiment, the height changing mechanism 36 is the lifting device 8 of the drive column 7 that raises and lowers the cab 6. That is, the height of the cameras 21a and 21b is changed together with the cab 6 to which they are attached. Alternatively, the height changing mechanism 36 may be configured to change the height of the cameras 21a and 21b relative to the cab 6. For example, the cameras 21a and 21b may be attached to the outside of the cab 6 via height adjustment rails that extend in the height direction, and the height changing mechanism 36 may be capable of adjusting the positions of the cameras 21a and 21b on the height adjustment rails.

[0026] The cameras 21a and 21b are capable of changing the shooting direction (the direction of the shooting center axis L1) within a horizontal plane. To this end, the detection system 20 is provided with a shooting direction changing mechanism 34 that changes the shooting direction of the cameras 21a and 21b by rotating about a predetermined rotation axis that extends in a direction intersecting the horizontal plane.

[0027] The imaging direction change mechanism 34 may be configured as a camera rotation mechanism that rotates the cameras 21a, 21b relative to the cab 6. In this case, the imaging direction change mechanism 34 can change the imaging directions of the first camera 21a and the second camera 21b independently of each other. Alternatively, the imaging direction change mechanism 34 may be the cab 6. For example, the cameras 21a, 21b may be fixed to the cab 6, and the imaging directions of the cameras 21a, 21b (the orientation of the imaging center axis L1) may be changed by the cab 6 rotating around a second rotation axis R6 that extends in a direction intersecting the horizontal plane.

[0028] The controller 30 and the image processing device 23 are configured by a computer that performs various calculations and processes based on the data stored in the memory device 24. For example, the controller 30 and the image processing device 23 include a CPU, a main memory (RAM), a communication interface, etc. The controller 30 and the image processing device 23 may be configured by the same computer, or may be configured by different computers.

[0029] The output unit 25 outputs the results of calculations or processing performed by the controller 30 and the image processing device 23. The output unit 25 is configured, for example, by a monitor that displays the results of calculations or a communication device that transmits data to a server or a communication terminal via a communication network. The image processing device 23 and the storage unit 24 of the detection system 20 may be configured as a server connected to the cameras 21a, 21b via a communication network. That is, the server may acquire images captured by the cameras 21a, 21b, perform a search process (described later) on the server, and display the results on a monitor or a display unit of a communication terminal provided in an operation room of the passenger boarding bridge 11, for example.

[0030] The controller 30 and image processing device 23 grasp the position coordinates of each part of the passenger boarding bridge 11 in real time using a predetermined three-dimensional coordinate system, such as an XYZ Cartesian coordinate system. For example, the position coordinates of each part of the passenger boarding bridge 11 are expressed as absolute coordinates in a three-dimensional coordinate system in which the intersection of the first rotation axis R4 of the rotunda 4 and the plane of the apron EP is set as the origin (0,0,0) and the X, Y, and Z axes are orthogonal to each other. The X, Y, and Z coordinate values ​​of these position coordinates each indicate the distance (e.g., in mm) from the origin (0,0,0), which is the position of the first rotation axis R4 of the rotunda 4. The controller 30 and image processing device 23 express the positions of each part of the aircraft 3 and the passenger boarding bridge 11 as position coordinates using such a three-dimensional Cartesian coordinate system.

[0031] The image processing device 23 has a search area image generation unit 31, a search execution unit 32, and a search range determination unit 33 as functional blocks to perform the search process described below. These functional blocks include circuits, including integrated circuits, in part or in whole. Therefore, these configurations 31, 32, and 33 can be considered circuits. These functional blocks are hardware that performs the enumerated functions or hardware programmed to perform the enumerated functions. The hardware may be hardware disclosed in this specification or other known hardware that is programmed or configured to perform the enumerated functions. If the hardware is a processor, which is considered a type of circuit, these functional blocks are a combination of hardware and software, and the software is used to configure the hardware or processor. The image processing device 23 reads a program for performing the search process stored in the memory 24 and performs the search process described below.

[0032] The search process of the boarding / disembarking section D1 in this embodiment will be described below. Figures 4 and 5 are flowcharts showing the flow of the search process in this embodiment. The search process is performed when the aircraft 3 arrives at a predetermined arrival position. For this reason, the passenger boarding bridge 11 is located in an initial position (standby position) that avoids the aircraft 3, as shown in Figure 1.

[0033] The memory 24 stores data of search ranges that are set in advance for the first camera 21a and the second camera 21b. The search ranges are set as areas along the direction of the aircraft axis AL. Furthermore, the memory 24 stores the initial angles (data of the photographing direction) of the cameras 21a and 21b. The initial angles of the cameras 21a and 21b are set to angles at which, at the initial position of the passenger boarding bridge 11, at least one end of the predetermined search range in the direction of the aircraft axis AL is included within the angle of view of the cameras 21a and 21b.

[0034] Here, the search process in this embodiment is performed by sequentially moving from the front to the rear in the direction of the aircraft axis AL of the aircraft 3. For this reason, the initial angles of the cameras 21a and 21b are set to angles at the initial position of the passenger boarding bridge 11 such that the front ends of the search ranges set for the cameras 21a and 21b in the direction of the aircraft axis AL are included within the angles of view of the cameras 21a and 21b.

[0035] First, in the search process, the controller 30 controls the shooting direction change mechanism 34 so that the angles of the cameras 21a and 21b become the initial angles (step S1). Furthermore, the controller 30 controls the height change mechanism 36 so that the heights of the cameras 21a and 21b become the initial heights. The cameras 21a and 21b each capture images at the initial angles (step S2). The image processing device 23 acquires a first captured image G1 captured by the first camera 21a at the initial angle and a second captured image G2 captured by the second camera 21b at the initial angle. The acquired captured images G1 and G2 are stored in the memory 24. The image processing device 23 may perform predetermined image processing on the captured images G1 and G2 in advance, such as correction of distortion or brightness due to the camera lens. The positions in the captured images G1 and G2 are expressed as position coordinates in a two-dimensional coordinate system with one vertex (e.g., the upper left vertex) of the captured images G1 and G2 as the origin and mutually orthogonal U-axis and V-axis.

[0036] The search area image generation unit 31 divides first areas Eai (i = 1, 2, 3, ...) including a portion of the first captured image G1, and generates a first search area image Cai (see FIG. 8, described later) including a partial area of ​​the first captured image G1 partitioned based on the first areas Eai. To this end, the search area image generation unit 31 first sets a first area Eai on the first captured image G1 corresponding to a virtual rectangular frame Bi in a three-dimensional space, which is real space (step S3). The initial first area Ea1 in the search process is set to an area including the front end of the search range in the direction of the axis AL. Similarly, the search area image generation unit 31 divides second areas Ebi (i = 1, 2, 3, ...) including a portion of the second captured image G2, and generates a second search area image Cbi (see FIG. 9, described later) including a partial area of ​​the second captured image G2 partitioned based on the second areas Ebi.

[0037] FIG. 6 is a diagram showing an example of a captured image in this embodiment. FIG. 7 is a conceptual diagram for setting the first and second regions in the captured image shown in FIG. 6. While FIGS. 6 and 7 and the following description mainly describe the manner in which the first search area image Cai is generated from the first captured image G1, the manner in which the second search area image Cbi is generated from the second captured image G2 is similar. That is, the manner in which the second search area image Cbi is generated from the second captured image G2 can be expressed by replacing the first camera 21a, the first captured image G1, the first region Eai, and the first search area image Cai with the second camera 21b, the second captured image G2, the second region Ebi, and the second search area image Cbi, respectively, in FIGS. 6 and 7 and the following description. The first region Eai and the second region Ebi are set in the same region in real space (as regions corresponding to the same virtual rectangular frame Bi).

[0038] In this embodiment, the search area image generation unit 31 sets a virtual plane VP that is perpendicular to the horizontal plane (apron EP) and parallel to the aircraft axis AL in the first photographed image G1. Here, the aircraft axis AL is set in advance at a position a predetermined distance above the marshal line 13, assuming that the aircraft 3 will be parked on the marshal line 13 indicated on the apron EP. Note that in the present specification and claims, the forward and rear sides of the aircraft axis AL refer to the forward and rear sides of the aircraft 3 parked on the marshal line 13.

[0039] The imaginary plane VP is set at a position offset by an offset amount W from the aircraft axis AL to one side in the aircraft width direction (the side closer to the passenger boarding bridge 11). In other words, the imaginary plane VP is defined as a plane that includes a reference axis VL offset by the offset amount W from the aircraft axis AL to one side in the aircraft width direction and is perpendicular to the horizontal plane. The offset amount W is set in advance, taking into account the aircraft width of the aircraft 3, so that the distance between the imaginary plane VP and the outer surface of the boarding and disembarking section D1 is close (less than a predetermined distance).

[0040] The search area image generation unit 31 sets a virtual rectangular frame Bi in three-dimensional space on the virtual plane VP. For example, the virtual rectangular frame Bi may be configured as a square frame with a side length of 4 m on the virtual plane VP, centered at a reference point Pi on the reference axis VL. Since the shooting direction (shooting center axis L1) of the first camera 21a is often not perpendicular to the virtual plane VP, the virtual rectangular frame Bi often has a distorted quadrangle shape on the first captured image G1, as shown by the first area Eai in FIG. 6. There is a one-to-one correspondence between the three-dimensional coordinates Bi(xi, yi, zi) in the real space and the two-dimensional coordinates Eai(ui, vi) on the first captured image G1 captured by the first camera 21a. Transformation between the three-dimensional coordinate system in the real space and the two-dimensional coordinate system on the first captured image G1 is performed by perspective projection transformation.

[0041] It is desirable that the entire rectangular shape (first area) Eai on the first captured image G1 corresponding to the virtual rectangular frame Bi is contained within the captured image G1, but it is sufficient that at least a part of the rectangular shape is captured in the first captured image G1.

[0042] The search area image generation unit 31 performs a projective transformation on the virtual plane VP in the first area Eai (an area including a part of the first captured image G1) set in this manner so that the image becomes what it would look like when viewed from a direction perpendicular to the virtual plane VP (step S4).

[0043] Projective transformation will now be described in more detail. Fig. 8 is a diagram showing a comparison of an image after projective transformation of a first region in a first captured image with an image before projective transformation in this embodiment. Fig. 9 is a diagram showing a comparison of an image after projective transformation of a second region in a second captured image with an image before projective transformation in this embodiment. Figs. 8 and 9 illustrate an example in which a virtual rectangular frame Bi is set near the boarding and disembarking area D1. In Figs. 8 and 9, a part of the outer surface of the aircraft 3 is indicated by a two-dot chain line.

[0044] As described above, the first area Eai, which includes a portion of the first captured image G1, often has a distorted rectangular shape. In the first search area image Cai, which is the image after projective transformation, the virtual rectangular frame Bi has the same rectangular shape as the shape set in three-dimensional space. For example, if the virtual rectangular frame Bi is a square, the first area Eapi corresponding to the virtual rectangular frame Bi after projective transformation will also be a square. Accordingly, in the image after projective transformation, a portion of the aircraft 3 close to the virtual plane VP is corrected so as to approach an image viewed from a direction perpendicular to the aircraft axis AL of the aircraft 3. Note that only objects on the virtual plane VP are strictly corrected, so the further away from the virtual plane VP an object is from the virtual plane VP, the more correction error remains.

[0045] In FIG. 8, the photographing direction (photographing center axis L1) of the first camera 21a is inclined with respect to the camera axis AL. Therefore, the door, which is the passenger access area D1 included in the first area Eai, appears with its lower and upper ends inclined with respect to the horizon of the first photographed image G1. As will be described later, in searching for the passenger access area D1 (such as the AI ​​image recognition described above), the reference point for the door is set, for example, on the door sill or a reinforcing plate provided on the door sill. In this case, in the image used to search for the passenger access area D1, it is preferable that the lower end (door sill or reinforcing plate) of the passenger access area D1 (door) extends horizontally.

[0046] In this embodiment, as shown in Fig. 8, a first area Eai including a part of the first captured image G1 is projectively transformed so that a virtual rectangular frame Bi set on a virtual plane VP parallel to the vehicle axis AL and perpendicular to the horizontal plane (apron EP) is displayed as a rectangle. Furthermore, projective transformation suppresses changes in the size of the boarding and alighting area D1 due to the distance between the first camera 21a and the boarding and alighting area D1. That is, the size of the boarding and alighting area D1 after projective transformation can be made approximately the same regardless of the distance between the first camera 21a and the boarding and alighting area D1. Therefore, the boarding and alighting area D1 can be detected accurately in the image after projective transformation.

[0047] As shown in Figure 9, for the second captured image G2 captured by the second camera 21b, a second region Ebi including a portion of the second captured image G2 is similarly projectively transformed so that a virtual rectangular frame Bi set on a virtual plane VP parallel to the camera axis AL and perpendicular to the horizontal plane (apron EP) is displayed as a rectangle.

[0048] The search area image generation unit 31 adjusts the search area images Cai and Cbi by trimming the image after the projective transformation (step S5). The search area image generation unit 31 trims the image after the projective transformation so that the first search area image Cai, which is a partial area of ​​the first captured image G1, has a larger area in the vertical direction than the second search area image Cbi, which is a partial area of ​​the second captured image G2.

[0049] In this embodiment, as shown in Fig. 8, the first search area image Cai includes the entire first area Eapi after the projective transformation, and is set to an area where the length Wah between a pair of horizontal boundary lines Bah that define the up-down direction is longer than the length Wav between a pair of vertical boundary lines Bav that define the left-right direction. On the other hand, as shown in Fig. 9, the second search area image Cbi includes the entire second area Ebpi after the projective transformation, and is set to an area where the length Wbh between a pair of horizontal boundary lines Bbh that define the up-down direction is shorter than the length Wbv between a pair of vertical boundary lines Bbv that define the left-right direction.

[0050] Here, the vertical length Wah of the first search area image Cai is longer than the vertical length Wbh of the second search area image Cbi. Furthermore, the horizontal length Wbv of the second search area image Cbi is longer than the horizontal length Wav of the first search area image Cai. By trimming in this manner, even when using images G1 and G2 captured by the first camera 21a and the second camera 21b having the same angle of view, the first search area image Cai can be made to have a vertically larger area than the second search area image Cbi, making it easier to determine whether the height position is appropriate.

[0051] Furthermore, the search area image generation unit 31 performs a predetermined extrapolation process on the adjusted image. For example, the extrapolation process includes a process of filling pixels outside the range of the captured images G1 and G2 with a single color (e.g., black), a process of copying pixels at the edges, etc. Furthermore, the generated search area images Cai and Cbi are enlarged so that the entire virtual rectangular frame Bi appears as large as possible in the search area images Cai and Cbi.

[0052] The search execution unit 32 searches for a door that is a candidate for the boarding / alighting area D1 within each of the search area images Cai and Cbi generated as described above (step S6). The door search method is not particularly limited as long as it is an image recognition process that can detect a door, and for example, AI image recognition using a trained model generated by deep learning or the like can be used.

[0053] In the door image recognition process, the door and its reference point are detected based on the painted outline of the door and the shape of the reinforcing plate attached to the door sill, etc. The door reference point is set, for example, at the center of the door sill or the center of the reinforcing plate.

[0054] If the search execution unit 32 detects a door in the first search area image Cai and also detects a door in the second search area image Cbi (Yes in both steps S7 and S8), it calculates the three-dimensional coordinates of the reference point of the door (step S9). The three-dimensional coordinates of the door can be obtained, for example, by converting the two-dimensional coordinates on each search area image Cai, Cbi into two-dimensional coordinates on the original captured images G1, G2, and then converting the two-dimensional coordinates on the first captured image G1 and the two-dimensional coordinates on the second captured image G2 into three-dimensional coordinates in real space. Note that the three-dimensional coordinates in real space may be calculated from only the two-dimensional coordinates of either one of the search area images Cai, Cbi. In this case, for example, the calculation may be performed by approximating that the reference point of the door is on the virtual plane VP.

[0055] The search execution unit 32 determines whether the detected door is the boarding / alighting section D1 to which the passenger boarding bridge 11 should be attached, based on the three-dimensional coordinates of the detected door (step S10). A search range is stored in advance in the memory 24 so as to include all of the doors of various aircraft to which the passenger boarding bridge 11 should be attached. If the three-dimensional coordinates of the detected door are within the search range stored in the memory 24, the search execution unit 32 determines that the boarding / alighting section D1 has been detected. If it is determined that the boarding / alighting section D1 has been detected (Yes in step S10), the search execution unit 32 outputs a result of successful search (step S11).

[0056] If a door is detected in the first search area image Cai but not in the second search area image Cbi (Yes in step S7, No in step S8), the controller 30 controls the height change mechanism 36 to change the height positions of the first camera 21a and the second camera 21b.

[0057] In this embodiment, as described above, the first camera 21a is installed at the upper inside portion of the connection portion 6a of the cab 6. The second camera 21b is installed at the lower inside portion of the connection portion 6a of the cab 6. The reference point of the door for searching the boarding / disembarking area D1 is set to the door sill located at the lower end of the door or a reinforcing plate provided on the door sill. Therefore, if the height position of the second camera 21b, which is arranged at the lower inside portion of the connection portion 6a of the cab 6, is located higher than the height position of the reference point of the door at the boarding / disembarking area D1 of the aircraft 3, as shown in FIG. 9 , the lower part of the photographing range of the second camera 21b (second photographed image G2) is blocked by the floor surface X of the cab 6, and there is a risk that the reference point of the door will not be captured.

[0058] Therefore, if a door is detected in the first search area image Cai generated based on the first captured image G1 taken by the first camera 21a, but not in the second search area image Cbi generated based on the second captured image G2 taken by the second camera 21b, it is considered that the height position of the second camera 21b is located above the height position of the reference point of the door at the boarding and disembarking area D1 of the aircraft 3.

[0059] Therefore, the controller 30 controls the height changing mechanism 36 to change the height positions of the first camera 21a and the second camera 21b to lower positions. In this embodiment, the controller 30 causes the lifting device 8 of the drive column 7 to lower the cab 6 a predetermined distance. The amount of height change is set so that the images captured before and after the height change partially overlap each other. This eliminates the need to provide a separate mechanism for changing the height positions of the first camera 21a and the second camera 21b, and also reduces the amount of adjustment required to align the height position of the passenger boarding bridge 11 with the boarding and disembarking section D1 when connecting the connection section 6a on the passenger boarding bridge 11 side to the boarding and disembarking section D1 of the aircraft 3.

[0060] Here, since the first search area image Cai has a larger area in the vertical direction than the second search area image Cbi, the first search area image Cai can be an image in which, if the door is included within the horizontal range of the first search area image Cai, the reference point of the door is located within the first search area image Cai.

[0061] After the height change, the cameras 21a and 21b take images again (step S2). Thereafter, the above-described search process is similarly performed on the captured images G1 and G2 after the height change (steps S3 to S11).

[0062] If a door is not detected in the first search area image Cai (No in step S7) or if it is determined that the detected door is not the boarding / disembarking area D1 (No in step S10), the search area image generation unit 31 moves the first area Eai from one side to the other in the direction of the aircraft axis AL of the aircraft 3 to sequentially generate first search area images Cai, and moves the second area Ebi from one side to the other in the direction of the aircraft axis AL of the aircraft 3 to sequentially generate second search area images Cbi. In this embodiment, the search area image generation unit 31 moves the first area Eai and the second area Ebi from the front side to the rear side in the direction of the aircraft axis AL.

[0063] To achieve this, first, the search area image generation unit 31 shifts (moves) the virtual rectangular frame Bi a predetermined distance in three-dimensional space from one side to the other in the direction of the aircraft's axis AL. The search area image generation unit 31 calculates a first area Eai in the first photographed image G1 and a second area Ebi in the second photographed image G2 that correspond to the shifted virtual rectangular frame Bi. In this way, the search area image generation unit 31 shifts (moves) the first area Eai in the first photographed image G1 from one side to the other in the direction of the aircraft's axis AL, and shifts (moves) the second area Ebi in the second photographed image G2 from one side to the other in the direction of the aircraft's axis AL (step S13).

[0064] For example, a reference point Pi+1 is set at a position on the reference axis VL in three-dimensional space a predetermined distance rearward from the position of the reference point Pi in the direction of the axis AL, and a virtual rectangular frame Bi+1 is set with the reference point Pi+1 as its reference. The distance between the reference points Pi and Pi+1 is preferably equal to or less than the length of one side extending horizontally of the virtual rectangular frame Bi, and more preferably between 1 / 4 and 1 / 2 of the length of that side. This allows for multiple opportunities to detect the same door (approximately two or three times) during door search.

[0065] The search range determination unit 33 determines whether the shifted first region Eai and second region Ebi are within a predetermined search range, i.e., whether the entire search range has been searched (step S14). When determining whether the shifted first region Eai and second region Ebi are within the search range, the search range determination unit 33 may also determine whether the corresponding virtual rectangular frame Bi in three-dimensional space is within the search range. If the shifted first region Eai and second region Ebi are within the search range (Yes in step S14), the search range determination unit 33 further determines whether the shifted first region Eai is within the range of the first captured image G1 used in the previous search, and whether the shifted second region Ebi is within the range of the second captured image G2 used in the previous search (step S15).

[0066] If the shifted first area Eai is within the range of the first captured image G1 and the shifted second area Ebi is within the range of the second captured image G2 (Yes in step S15), the search area image generation unit 31 sets (updates) the shifted first area Eai as the first area Eai for generating the first search area image Cai, and sets (updates) the shifted second area Ebi as the second area Ebi for generating the second search area image Cbi (step S3).

[0067] The search area image generation unit 31 generates (updates) the first search area image Cai and the second search area image Cbi in the same manner as described above based on the updated first area Eai and second area Ebi. The search execution unit 32 searches for the boarding and disembarking area D1 in the updated first search area image Cai and second search area image Cbi. In this manner, the search execution unit 32 repeatedly searches for the boarding and disembarking area D1 using the multiple first search area images Cai that are sequentially generated while changing the first search area image Cai by shifting the first area Eai from the front side to the rear side along the vehicle axis AL. Similarly, the search execution unit 32 repeatedly searches for the boarding and disembarking area D1 using the multiple second search area images Cbi that are sequentially generated while changing the second search area image Cbi by shifting the second area Ebi from the front side to the rear side along the vehicle axis AL.

[0068] If the shifted first area Eai is within the search range but outside the range of the first captured image G1, or if the shifted second area Ebi is within the search range but outside the range of the second captured image G2 (No in step S15), the search range determination unit 33 sends an instruction signal to the controller 30 to change the shooting direction of the cameras 21a and 21b. The controller 30 controls the corresponding shooting direction change mechanism 34 to change the shooting direction of the cameras 21a and 21b based on the instruction signal (step S16). The amount of change in the shooting direction is set so that the captured images before and after the shooting direction change partially overlap each other.

[0069] 7, by rotating the cab 6 by a predetermined angle θ around the second rotation axis R6, the imaging center axis L1 of the cameras 21a, 21b is changed to an imaging center axis L1a that intersects with the reference axis VL of the virtual plane VP on the rear side in the direction of the axis AL compared to the imaging center axis L1. Note that, as described above, the method for changing the imaging direction of the cameras 21a, 21b is not limited to this, and for example, the first camera 21a and the second camera 21b themselves may be rotated around a predetermined rotation axis that extends in a direction intersecting with the horizontal plane relative to the cab 6.

[0070] First camera 21a and second camera 21b take images again after changing the shooting direction (step S2). After that, similarly, search processing is performed on the updated shot images (steps S3 to S11).

[0071] If the first area Eai or the second area Ebi after the shift falls outside the search range (No in step S14), or if the number of retries is less than a preset reference value (No in step S17), the search range determination unit 33 increments the number of retries by one and restarts the search process from scratch. That is, the cameras 21a and 21b are returned to their initial angles to capture images, and a search is performed based on each captured image G1 and G2. If the number of retries is equal to or greater than the reference value (Yes in step S17), the search execution unit 32 determines that the boarding and disembarking area D1 could not be detected, and outputs a result of search failure (step S18).

[0072] As described above, according to this embodiment, a first search area image Cai is generated from an image captured by the first camera 21a to detect and determine the boarding and disembarking area D1 of the aircraft 3, and a second search area image Cbi is generated from an image captured by the second camera 21b to detect and determine the boarding and disembarking area D1 of the aircraft 3.

[0073] If the boarding and alighting area D1 is detected in the first search area image Cai but not in the second search area image Cbi, the height positions of the first camera 21a and the second camera 21b are changed. Since the first search area image Cai has a larger area in the vertical direction than the second search area image Cbi, if the boarding and alighting area D1 can be detected in the first search area image Cai but not in the second search area image Cbi, the heights of the first camera 21a and the second camera 21b are not appropriate.

[0074] According to the above configuration, even if the height positions of the first camera 21a and the second camera 21b are not appropriate as positions for detecting the boarding and disembarking section D1 of the aircraft 3 due to the positional relationship between the aircraft 3 and the passenger boarding bridge 11, the height positions of the first camera 21a and the second camera 21b are automatically changed. This makes it possible to automatically change the height position of the search area for the boarding and disembarking section D1 to match the position of the boarding and disembarking section D1 of the aircraft 3. Therefore, it is possible to accurately detect the position of the boarding and disembarking section D1 of the aircraft 3 in accordance with various gate arrangements.

[0075] Furthermore, if the boarding and disembarking area D1 is not detected from the first search area image Cai, it is determined that the boarding and disembarking area D1 does not exist in the current search area (position of the virtual rectangular frame Bi) in the direction of the aircraft axis AL of the aircraft 3, and the areas Eai and Ebi are shifted.

[0076] 10 is a diagram illustrating an example of overlapping of the captured areas in the first search area image and the second search area image in this embodiment. In the example shown in Fig. 10, the determination area including the first search area image Cai and the second search area image Cbi includes a first determination area Q1, a second determination area Q2, and a third determination area Q3.

[0077] The first judgment area Q1 is an area where the upper area of ​​the first search area image Cai and the central area in the horizontal direction of the second search area image Cbi overlap with each other. If the boarding / alighting area D1 is detected in the first judgment area Q1, it is determined that the boarding / alighting area D1 to be searched for has been detected. The second judgment area Q2 is an area in the lower area of ​​the first search area image Cai that does not overlap with the second search area image Cbi. If the boarding / alighting area D1 is detected in the second judgment area Q2, it is determined that the height of the cameras 21a, 21b needs to be adjusted. The third judgment area Q3 is an area on both ends of the second search area image Cbi in the horizontal direction that does not overlap with the first search area image Cai. If the boarding / alighting area D1 is detected in the third judgment area Q3 or if the boarding / alighting area D1 is not detected in any of the judgment areas Q1, Q2, and Q3, it is determined that the search area needs to be shifted in the direction of the axis AL.

[0078] In this way, by branching the flow of the search process based on whether the boarding and disembarking area D1 is detected in any of the judgment areas Q1, Q2, Q3 contained in the first search area image Cai and the second search area image Cbi, or whether it is not detected in any of the judgment areas Q1, Q2, Q3, it is possible to automate the decision on how to change the search area.

[0079] Furthermore, from each of the captured images G1, G2 captured by the cameras 21a, 21b provided at the tip of the passenger boarding bridge 11, a plurality of search area images Cai, Cbi are generated, each divided into areas Eai, Ebi including a portion of the captured images G1, G2. The plurality of search area images Cai, Cbi are generated by moving the areas Eai, Ebi from one side to the other in the direction of the aircraft axis AL of the aircraft 3. Therefore, by repeatedly searching for the boarding and disembarking section D1 while changing the search area images Cai, Cbi using the plurality of search area images Cai, Cbi generated in sequence, it is possible to search for the boarding and disembarking section D1 in the entire captured images G1, G2 while moving the search area in the direction along the aircraft axis AL of the aircraft 3.

[0080] This allows the search for the boarding / alighting section D1 to be performed uniformly regardless of the type of aircraft 3 or the gate arrangement. This makes it possible to correctly detect the boarding / alighting section D1 to be installed even in gate arrangements that could not be handled by conventional position detection of the boarding / alighting section D1 that relies only on three-dimensional position. Moreover, when searching for the boarding / alighting section D1, it is possible to eliminate the need for cumbersome operations such as inputting information corresponding to the parked aircraft 3. In other words, the operator only needs to operate a button to start the search process, and the search process can be performed for various aircraft 3. of It can be executed.

[0081] Furthermore, according to this embodiment, the search area images Cai and Cbi are generated as images parallel to the aircraft axis AL by projective transformation. Therefore, even if the captured images G1 and G2 are captured from an oblique direction of the aircraft 3, it is possible to suppress distortion of the shape of the boarding and disembarking area D1 to be searched for in the search area images Cai and Cbi. This improves the accuracy of searching for the boarding and disembarking area D1 in the search area images Cai and Cbi.

[0082] Furthermore, according to this embodiment, the search area image generation unit 31 performs projective transformation on the photographed images G1, G2 in the areas Eai, Ebi each time the photographed images G1, G2 are divided into areas Eai, Ebi each including a part of the photographed images G1, G2. In this way, the photographed images G1, G2 are projectively transformed (and also cropped) for each area Eai, Ebi, and therefore it is possible to suppress a decrease in image resolution compared to when the entire photographed images G1, G2 are projectively transformed and then the projectively transformed images are divided into predetermined areas Eai, Ebi and enlarged.

[0083] Furthermore, according to this embodiment, if it is determined that the shifted areas Eai and Ebi are within the search range but not within the captured images G1 and G2, the shooting direction change mechanism 34 changes the shooting direction of the cameras 21a and 21b, and the cameras 21a and 21b capture images again. In this way, by capturing images multiple times while changing the shooting direction of the cameras 21a and 21b, it is possible to obtain multiple captured images G1 and G2 of the entire search range. Therefore, since the shooting range per captured image can be relatively narrow, the resolution of the search area images Cai and Cbi obtained from the captured images G1 and G2 can be increased. Alternatively, the search range can be set wider than the field of view of the cameras 21a and 21b.

[0084] [Variations] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various improvements, changes, and modifications are possible within the scope of the spirit of the present disclosure.

[0085] For example, in the above embodiment, the first camera 21a and the second camera 21b have the same angle of view, and when generating the search area images Cai and Cbi from the captured images G1 and G2, the first search area image Cai is adjusted (by changing the crop shape) so that it has a larger area in the vertical direction than the second search area image Cbi. However, this is not limiting. For example, the first camera 21a may have a wider angle of view than the second camera 21b. In this case, even if the first search area image Cai and the second search area image Cbi are generated from areas of the same area in the first captured image G1 and the second captured image G2, the area of ​​real space included in the first search area image Cai will be larger than the area of ​​real space included in the second search area image Cbi.

[0086] In addition, while the above embodiment illustrates an example in which the second search area image Cbi is adjusted to have a wider area in the horizontal direction than the first search area image Cai, this is not limitative. For example, the first search area image Cai and the second search area image Cbi may have the same width in the horizontal direction. Furthermore, the aspect ratio (Wah:Wav) of the first search area image Cai is not limited to the above embodiment (Wah>Wav). Similarly, the aspect ratio (Wbh:Wbv) of the second search area image Cbi is not limited to the above embodiment (Wbv>Wbh).

[0087] In addition, in the above embodiment, an example was given of a mode in which, when the boarding and alighting area D1 is not detected in the first search area image Cai, the search area images Cai and Cbi are shifted in the direction of the axis AL regardless of whether the boarding and alighting area D1 is detected in the second search area image Cbi (third judgment area Q3 in Figure 10).However, the subsequent processing may be different depending on whether the boarding and alighting area D1 is detected only in the second search area image Cbi and whether the boarding and alighting area D1 is not detected in either the first search area image Cai or the second search area image Cbi.

[0088] For example, the search area image generation unit 31 may shift the first area Eai and the second area Ebi in the direction of the axis AL differently when the boarding and alighting area D1 is detected only in the second search area image Cbi and when the boarding and alighting area D1 is not detected in either the first search area image Cai or the second search area image Cbi. In this case, for example, the shift amount of the first area Eai and the second area Ebi in the direction of the axis AL when the boarding and alighting area D1 is not detected in either the first search area image Cai or the second search area image Cbi may be larger than the shift amount when the boarding and alighting area D1 is detected only in the second search area image Cbi.

[0089] More specifically, if the boarding and alighting area D1 is detected only in the second search area image Cbi, i.e., if the boarding and alighting area D1 is detected in the third judgment area Q3, the search area image generation unit 31 may shift the first area Eai and the second area Ebi so that the boarding and alighting area D1 is located in the first judgment area Q1. Also, if the boarding and alighting area D1 is not detected in either the first search area image Cai or the second search area image Cbi, the search area image generation unit 31 may shift the first area Eai and the second area Ebi so that the area outside the third judgment area Q3 in the direction of the vehicle axis AL is located in the first judgment area Q1.

[0090] Furthermore, in the above embodiment, an example has been given in which the first camera 21a is provided at the same longitudinal position in the cab 6 as the second camera 21b but at a higher position than the second camera 21b, but this is not limiting. Fig. 11 is a diagram showing the positional relationship between the first camera and the second camera in Modification 1. As shown in Fig. 11, the first camera 21a and the second camera 21b may be arranged so that they are at the same height position but at different longitudinal positions in the cab 6. The longitudinal position of the first camera 21a in the cab 6 is arranged forward of the longitudinal position of the second camera 21b in the cab 6 (at a position closer to the connection portion 6a). In this case as well, a first angle δ1 formed with respect to the horizontal plane H of a first virtual line V1 connecting the optical center of first camera 21a and a predetermined position VO on the object side photographed by first camera 21a is different from a second angle δ2 formed with respect to the horizontal plane H of a second virtual line V2 connecting the optical center of second camera 21b and the predetermined position VO (the first angle δ1 is larger than the second angle δ2). Furthermore, first camera 21a and second camera 21b may be disposed at different height positions and at different positions in the fore-and-aft direction in cab 6.

[0091] In the above embodiment, the first camera 21a, which captures the first captured image G1 for generating the first search area image Cai having a wide area in the vertical direction, is disposed above (or in front of) the second camera 21b. However, as shown in FIG. 12, the first camera 21a may be disposed below (or behind) the second camera 21b. FIG. 12 is a diagram showing the positional relationship between the first camera and the second camera in Modification 2. The example in FIG. 12 differs from the above embodiment (the example in FIG. 2) in that the positions of the first camera 21a and the second camera 21b are swapped. In the example in FIG. 12, the predetermined position VO is set above the cab 6. Therefore, in this example as well, the first angle δ1 is greater than the second angle δ2.

[0092] For example, if the reference point for the door is set to the upper end of the door, and the height position of the second camera 21b disposed at the upper inside portion of the connecting portion 6a of the cab 6 is lower than the height position of the reference point for the door at the boarding / disembarking section D1 of the aircraft 3, the upper part of the photographing range of the second camera 21b (second photographed image G2) may be blocked by the ceiling of the cab 6, and the door reference point may not be captured. Therefore, if a door is detected in the first search area image Cai generated based on the first photographed image G1 photographed by the first camera 21a, but not in the second search area image Cbi generated based on the second photographed image G2 photographed by the second camera 21b, it is considered that the height position of the second camera 21b is lower than the height position of the reference point for the door at the boarding / disembarking section D1 of the aircraft 3. In this case, the controller 30 controls the height changing mechanism 36 to change the height positions of the cameras 21a and 21b to higher positions.

[0093] Furthermore, in the above embodiment, an example has been given in which the search is performed from the front to the rear in the direction of the aircraft axis AL of the aircraft 3, but this is not limiting, and the search may be performed from the rear to the front in the direction of the aircraft axis AL of the aircraft 3. For example, when connecting the connection section 6a of the passenger boarding bridge 11 to the boarding and disembarking section D2 located at the rear of the aircraft 3, it is preferable to perform the search from the rear to the front in the direction of the aircraft axis AL of the aircraft 3.

[0094] Furthermore, although the above embodiment has exemplified a system including one passenger boarding bridge 11, the detection system 20 may include two or more passenger boarding bridges. For example, the detection system 20 of the present disclosure can also be applied to a system including two passenger boarding bridges, a first boarding bridge and a second boarding bridge.

[0095] For example, the connection portion 6a of the cab 6 on the first boarding bridge is connected to the boarding / disembarking section D1 on the front side of the aircraft 3, and the connection portion 6a of the cab 6 on the second boarding bridge is connected to the boarding / disembarking section D2 on the rear side of the aircraft 3. In this case, in the search process for the boarding / disembarking section D1 to which the first boarding bridge is connected, the search is performed from the front side to the rear side in the direction of the aircraft axis AL of the predetermined search range for the first boarding bridge. That is, in the search process for the boarding / disembarking section D1 to which the first boarding bridge is connected, as described above, the areas Eai and Ebi are moved from the front side to the rear side in the direction of the aircraft axis AL on the captured images G1 and G2. Furthermore, if necessary, the shooting direction of the cameras 21a and 21b is changed from the front side to the rear side of the aircraft 3 and re-shooting is performed.

[0096] Meanwhile, in the search process for boarding and disembarking section D2 connected to the second boarding bridge, a search is performed from the rear side to the front side in the direction of aircraft axis AL within a predetermined search range for the second boarding bridge. That is, in the search process for boarding and disembarking section D2 connected to the second boarding bridge, areas Eai and Ebi are moved from the rear side to the front side in the direction of aircraft axis AL on the captured images G1 and G2. Furthermore, if necessary, the shooting direction of cameras 21a and 21b is changed from the rear side to the front side of the aircraft 3, and re-shooting is performed.

[0097] This makes it possible to preferentially detect the boarding and disembarking section D1 located at the front of the aircraft 3 based on the images G1, G2 taken by the cameras 21a, 21b installed on the first boarding bridge, and to preferentially detect the boarding and disembarking section D2 located at the rear of the aircraft 3 based on the images G1, G2 taken by the cameras 21a, 21b installed on the second boarding bridge. This reduces the possibility of detecting the same boarding and disembarking section for multiple boarding bridges. This makes it possible to accurately and quickly search for the boarding and disembarking section D1 to which the first boarding bridge should connect and the boarding and disembarking section D2 to which the second boarding bridge should connect.

[0098] Furthermore, the detection system 20 of the present disclosure can also be applied to a system including, for example, three boarding bridges, in which case it can be assumed that the aircraft 3 has boarding and disembarking sections on the first floor and second floor.

[0099] In this case, for example, to connect the first boarding bridge to the boarding section located forward in the direction of aircraft axis AL on the first floor, a first search is performed from forward to rearward in the direction of aircraft axis AL within a first search range. To connect the second boarding bridge to the boarding section located rearward in the direction of aircraft axis AL on the first floor, a second search is performed from rearward to forward in the direction of aircraft axis AL within a second search range. To connect the third boarding bridge to the boarding section on the second floor, a third search is performed from forward to rearward in the direction of aircraft axis AL within a third search range set higher than the first search range in the first search.

[0100] In addition, the search for the boarding and disembarking section D2 to which the second boarding bridge should be connected may also be performed from the front to the rear in the direction of the aircraft axis AL, similar to the search for the boarding and disembarking section D1 to which the first boarding bridge should be connected.

[0101] Furthermore, in the above embodiment, an example was given in which one controller 30 controls the operation of all of the passenger boarding bridge 11, the shooting direction change mechanism 34, and the height change mechanism 36, but different controllers may be provided for some or all of these components 11, 34, and 36.

[0102] [Disclosure Summary] Each of the following sections discloses a preferred embodiment.

[0103] [Item 1] A detection system according to an aspect of the present disclosure is a detection system for detecting boarding and disembarking sections of an aircraft, the detection system comprising: a passenger boarding bridge connected to a terminal building; a first camera and a second camera provided on the passenger boarding bridge; a height change mechanism capable of changing the height positions of the first camera and the second camera; a controller for controlling the operation of the height change mechanism; and an image processing device for detecting the boarding and disembarking section of the aircraft from a first photographed image taken by the first camera and a second photographed image taken by the second camera, wherein a first angle of a first virtual line connecting the optical center of the first camera and a predetermined position on the object side photographed by the first camera with respect to a horizontal plane is determined by the optical angle of the second camera. The first camera and the second camera are arranged so that they form an angle different from a second angle with respect to the horizontal plane of a second virtual line connecting the academic center and the specified position, the image processing device generates a first search area image from the first captured image and generates a second search area image from the second captured image, the first search area image having a wider area in the vertical direction than the second search area image, and the controller controls the height change mechanism to change the height positions of the first camera and the second camera when the image processing device detects the boarding and alighting area from the first search area image but does not detect the boarding and alighting area from the second search area image.

[0104] According to the above configuration, a first search area image for detecting and determining whether an aircraft's boarding and disembarking area is present is generated from an image captured by the first camera, and a second search area image for detecting and determining whether an aircraft's boarding and disembarking area is present is generated from an image captured by the second camera. If the boarding and disembarking area is detected in the first search area image but not in the second search area image, the height positions of the first camera and the second camera are changed. Since the first search area image has a larger area in the vertical direction than the second search area image, if the boarding and disembarking area can be detected in the first search area image but not in the second search area image, the heights of the first camera and the second camera are inappropriate. According to the above configuration, even if the height positions of the first camera and the second camera are inappropriate for detecting the aircraft's boarding and disembarking area due to the positional relationship between the aircraft and the passenger boarding bridge, the height positions of the first camera and the second camera are automatically changed. This allows the height position of the search area for the boarding and disembarking area to be automatically changed to match the position of the aircraft's boarding and disembarking area. Therefore, the position of the aircraft's boarding and disembarking area can be accurately detected in accordance with various gate arrangements.

[0105] [Item 2] In the detection system of item 1, the image processing device divides a first area including a portion of the first captured image to generate a first search area image including a portion of the first captured image divided based on the first area, and divides a second area including a portion of the second captured image to generate a second search area image including a portion of the second captured image divided based on the second area, wherein the first area and the second area are set to the same area in real space, and the portion of the first captured image may have a larger area in the vertical direction than the portion of the second captured image.

[0106] According to the above configuration, even if the first camera and the second camera have the same angle of view, it is possible to easily determine whether the height position is appropriate.

[0107] [Item 3] In the detection system of item 2, when the boarding and disembarking section is not detected from the first search area image, the image processing device sequentially generates the first search area images by moving the first area on the first photographed image from one side to the other in the aircraft axis direction of the aircraft, and sequentially generates the second search area images by moving the second area on the second photographed image from one side to the other in the aircraft axis direction of the aircraft, and repeats the search while changing the first search area images using the sequentially generated multiple first search area images, and sequentially generates the second search area images by changing the second search area images using the sequentially generated multiple second search area images. Boarding and alighting area The search may be repeated.

[0108] According to the above configuration, a plurality of first search area images are generated from a first captured image captured by a first camera, the first search area images being divided into first regions each including a portion of the first captured image. Similarly, a plurality of second search area images are generated from a second captured image captured by a second camera, the second search area images being divided into second regions each including a portion of the second captured image. The plurality of first search area images and the plurality of second search area images are generated by moving the first region or the second region from one side of the aircraft's axis to the other. Therefore, by repeatedly searching for boarding and disembarking areas while changing the search area image using the plurality of sequentially generated search area images, it is possible to search for boarding and disembarking areas throughout the entire captured image while moving the search area in a direction along the aircraft's axis. This allows for a uniform search for boarding and disembarking areas regardless of the type of aircraft or gate arrangement. Therefore, the location of an aircraft's boarding and disembarking areas can be accurately detected in accordance with various gate arrangements.

[0109] [Item 4] In the detection system of any one of items 1 to 3, the first camera may be disposed above the second camera.

[0110] [Item 5] In the detection system of any one of items 1 to 4, the passenger boarding bridge may include a cab having a connection portion connected to the boarding and disembarking portion of the aircraft, and a lifting device that raises and lowers the cab, the first camera and the second camera may be provided in the cab, and the height change mechanism may be the lifting device.

[0111] With the above configuration, there is no need to provide a separate mechanism for changing the height positions of the first and second cameras, and when connecting the connection part on the passenger boarding bridge to the boarding and disembarking part of the aircraft, the amount of adjustment required to align the height position of the passenger boarding bridge with the boarding and disembarking part can be reduced.

[0112] [Item 6] The detection system of item 3 may include a shooting direction change mechanism that changes the shooting direction of the first camera and the second camera by rotating around a predetermined rotation axis that extends in a direction intersecting a horizontal plane, and the image processing device determines whether the first area after movement is within a predetermined search range and within the first captured image, and if the controller determines that the first area after movement is within the search range but not within the first captured image, the controller may change the shooting direction of the first camera and the second camera using the shooting direction change mechanism and take images again using the first camera and the second camera.

[0113] According to the above configuration, by taking multiple photographs while changing the camera's shooting direction, it is possible to obtain multiple photographed images of the entire search range. Therefore, since the shooting range per photographed image can be made relatively narrow, the resolution of the search area image obtained from the photographed image can be increased. Alternatively, it is possible to set the search range wider than the field of view of the camera. [Explanation of symbols]

[0114] 2 Terminal Building 3 aircraft 6 Cab 8 Lifting device 11 Passenger Boarding Bridge 12 Second Boarding Bridge 20 Detection System 21a Camera 1 21b Second Camera 23 Image processing device 30 Controller 34 Shooting direction change mechanism 36 Height change mechanism D1, D2 boarding area

Claims

1. 1. A detection system for detecting a landing section of an aircraft, comprising: a passenger boarding bridge connected to the terminal building; a first camera and a second camera provided on the passenger boarding bridge; a height changing mechanism capable of changing the height positions of the first camera and the second camera; a controller for controlling the operation of the height change mechanism; an image processing device that detects a boarding and disembarking section of the aircraft from a first photographed image taken by the first camera and a second photographed image taken by the second camera, the first camera and the second camera are disposed such that a first angle of a first virtual line connecting an optical center of the first camera and a predetermined position on the side of an object photographed by the first camera with respect to a horizontal plane is different from a second angle of a second virtual line connecting an optical center of the second camera and the predetermined position with respect to the horizontal plane; the image processing device generates a first search area image from the first captured image and generates a second search area image from the second captured image; the first search area image has a larger area in the vertical direction than the second search area image, The controller controls the height change mechanism to change the height positions of the first camera and the second camera when the image processing device detects the boarding and alighting area from the first search area image but does not detect the boarding and alighting area from the second search area image.

2. the image processing device divides a first region including a portion of the first captured image to generate the first search area image including the partial region of the first captured image partitioned based on the first region, and divides a second region including a portion of the second captured image to generate the second search area image including the partial region of the second captured image partitioned based on the second region; the first region and the second region are set to the same region in real space; The detection system according to claim 1 , wherein the partial area of ​​the first captured image has a larger area in the vertical direction than the partial area of ​​the second captured image.

3. The image processing device includes: when the boarding and disembarking section is not detected from the first search area image, the first area is moved from one side in the aircraft axis direction of the aircraft on the first photographed image to the other side, thereby sequentially generating the first search area image, and the second area is moved from one side in the aircraft axis direction of the aircraft on the second photographed image to the other side, thereby sequentially generating the second search area image; The detection system described in claim 2, wherein the search for the boarding and alighting area is repeatedly performed by using a plurality of sequentially generated first search area images while changing the first search area image, and the search for the boarding and alighting area is repeatedly performed by using a plurality of sequentially generated second search area images while changing the second search area image.

4. The detection system according to claim 1 , wherein the first camera is positioned above the second camera.

5. the passenger boarding bridge comprises a cab having a connection portion connected to the boarding and disembarking portion of the aircraft, and a lifting device that raises and lowers the cab; the first camera and the second camera are provided in the cab, The detection system according to claim 1 , wherein the height changing mechanism is the lifting device.

6. an imaging direction changing mechanism that changes the imaging directions of the first camera and the second camera by rotating about a predetermined rotation axis that extends in a direction intersecting a horizontal plane; the image processing device determines whether the first area after movement is within a predetermined search range and within the first captured image; 4. The detection system of claim 3, wherein when the controller determines that the first area after movement is within the search range but not within the first captured image, the controller changes the capturing directions of the first camera and the second camera using the capturing direction change mechanism and captures images again using the first camera and the second camera.

Images