Boarding bridge all-round monitoring method, device and system and related equipment

Abstract

The invention provides a boarding bridge surround view monitoring method, device and system and related equipment, and relates to the technical field of image processing. The method comprises the steps that calibration parameters of multiple cameras are acquired, and the multiple cameras are used for collecting monitoring images of a to-be-monitored boarding bridge in multiple directions; constructing a dynamic projection lookup table according to the calibration parameters of the plurality of cameras; acquiring real-time image data of the plurality of cameras, performing image fusion processing based on the dynamic projection lookup table and a preset fusion rule, and generating a panoramic view for looking around and monitoring the boarding bridge; and displaying the panoramic view in the monitoring picture in real time. According to the invention, 360-degree all-round monitoring can be carried out on the external all-round view of the boarding bridge.

Description

Background Art

[0002] As a passage connecting the aircraft cabin door and the terminal building, the boarding bridge has a complex outline and multiple parts that can move, such as the rotation of the receiving gate, the extension and retraction of the passage, the rotation of the passage around the center of the rotating platform, the lifting mechanism driving the lifting and lowering of the receiving gate and the pitching of the passage, etc. The bottom of the boarding bridge can be connected to various equipment such as aircraft power supply and aircraft air conditioning. Due to the complex outline of the boarding bridge itself and the presence of various bridge hanging equipment, the boarding bridge is prone to collision during docking and bridge withdrawal. There have been many accidents worldwide where boarding bridges have scratched aircraft engines, scratched wings, and collided with adjacent bridges. In order to avoid collisions, some safety detection sensors are usually added, and cameras are added to monitor dangerous areas to reduce the risks brought by the operator's visual blind spots. In addition to the cameras installed under the rotating platform or the inner channel to observe the status under the boarding bridge, additional cameras will be added to the wheel frame area in front of the wheel frame, cameras for wing anti-collision protection will be added on the right side of the receiving gate and the service ladder platform, and cameras for illuminating the wind speed tube will be added on the left side of the receiving gate, etc. The number of cameras is increasing.

[0003] With the development of boarding bridge technology, a remote operation mode has emerged. The operator's operating position has been transferred from the local operating console of the boarding bridge to the remote operating console in the remote operating room. After the operating position leaves the local operating console, the areas that the operator could originally visually view on-site through the front window, side windows, and rearview mirrors must also be observed through new cameras, such as the areas on the left and right sides of the aisle, the movable floor of the pick-up gate, and the area directly in front of the pick-up gate. By increasing the number of cameras, the problem of the operator's visual blind spots can be solved to a certain extent. However, the excessive number of camera images also causes the operator to monitor multiple cameras in real time, which makes it easy for the operator to become defocused, reducing the anti-collision effect.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present invention provides a method, device, system and related equipment for surround monitoring of a boarding bridge, which at least to a certain extent overcomes the problem in the related art that too many monitoring images may cause operators to lose focus, thereby reducing the anti-collision effect.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a method for surround monitoring of a boarding bridge is provided, comprising: obtaining calibration parameters of a plurality of cameras, wherein the plurality of cameras are used to collect monitoring images of a boarding bridge to be monitored in multiple directions; constructing a dynamic projection lookup table according to the calibration parameters of the plurality of cameras; obtaining real-time image data of the plurality of cameras, performing image fusion processing based on the dynamic projection lookup table and pre-set fusion rules, and generating a panoramic view for surround monitoring of the boarding bridge; and displaying the panoramic view in real time in the monitoring screen.

[0008] In some exemplary embodiments of the present disclosure, based on the aforementioned scheme, real-time image data of multiple cameras are acquired, image fusion processing is performed based on the dynamic projection lookup table and pre-set fusion rules, and a panoramic view for surround monitoring of the boarding bridge is generated, including: acquiring real-time image data of multiple cameras, and obtaining a mapping image with overlapping areas based on the dynamic projection lookup table; performing image fusion processing on the mapping image based on pre-set fusion rules, and generating a panoramic view for surround monitoring of the boarding bridge.

[0009] In some exemplary embodiments of the present disclosure, based on the aforementioned scheme, the fusion rules include: fusing the surveillance images taken by the cameras arranged on both sides of the outer channel of the boarding bridge and the surveillance images taken by the cameras arranged at the bottom of the outer channel into one surveillance image; fusing the surveillance images taken by the cameras arranged on both sides of the boarding bridge's pick-up port into one surveillance image; splicing the fused surveillance images of the outer channel of the boarding bridge and the fused surveillance images of the boarding bridge's pick-up port into one surveillance image through image splicing technology; or, fusing the surveillance images taken by the cameras arranged on both sides of the outer channel of the boarding bridge into one surveillance image; fusing the surveillance images taken by the cameras arranged on both sides of the boarding bridge's pick-up port into one surveillance image; splicing the fused surveillance images of the outer channel of the boarding bridge and the fused surveillance images of the boarding bridge's pick-up port into one surveillance image through image splicing technology.

[0010] In some exemplary embodiments of the present disclosure, based on the aforementioned scheme, calibration parameters of multiple cameras are obtained, including: after adjusting the boarding bridge to a preset posture and a preset position, calibrating the internal parameters of the multiple cameras; based on the internal parameters of the multiple cameras, calibrating the external parameters of the multiple cameras relative to the boarding bridge and the ground.

[0011] In some exemplary embodiments of the present disclosure, based on the aforementioned scheme, a dynamic projection lookup table is constructed according to the calibration parameters of multiple cameras, including: obtaining images generated by multiple cameras shooting the same scene area; performing distortion correction on the images based on the calibrated internal parameters of the multiple cameras to obtain a distortion correction map; performing perspective transformation on the distortion correction map based on the calibrated external parameters of the multiple cameras to obtain a ground projection map; determining the fusion boundary and fusion angle of adjacent ground projection maps to obtain a bird's-eye view; using a partitioned projection method to project the near-area image and the far-area image corresponding to the ground projection map to the corresponding areas of a pre-set 3D model, respectively, wherein the near-area image and the far-area image corresponding to the ground projection map are represented as a bird's-eye view obtained by converting images captured at areas near and far from the camera; performing multi-perspective observation on the 3D model and generating a corresponding dynamic lookup table.

[0012] In some exemplary embodiments of the present disclosure, based on the aforementioned scheme, real-time image data of multiple cameras are acquired, and image fusion processing is performed based on the dynamic projection lookup table and pre-set fusion rules to generate a panoramic view for surround monitoring of the boarding bridge. The method also includes: acquiring real-time image data of multiple cameras, performing image processing on the real-time image data to obtain image data for image fusion processing, wherein the image processing includes at least one or more of the following: image denoising processing, image edge extraction, image histogram processing, and image transformation processing.

[0013] According to another aspect of the present disclosure, there is also provided a device for surround monitoring of a boarding bridge, including: a parameter calibration module, used to obtain calibration parameters of multiple cameras, wherein the multiple cameras are used to collect monitoring images of multiple directions of the boarding bridge to be monitored; a dynamic projection lookup table construction module, used to construct a dynamic projection lookup table according to the calibration parameters of the multiple cameras; a panoramic view generation module, used to obtain real-time image data of multiple cameras, perform image fusion processing according to the dynamic projection lookup table and pre-set fusion rules, and generate a panoramic view for surround monitoring of the boarding bridge; a panoramic view display module, used to display the panoramic view in real time in the monitoring screen.

[0014] According to another aspect of the present disclosure, a boarding bridge surround monitoring system is also provided, including: a camera system for acquiring real-time image data of multiple cameras; an image processing system for acquiring calibration parameters of multiple cameras, wherein the multiple cameras are used to collect monitoring images of multiple directions of the boarding bridge to be monitored; constructing a dynamic projection lookup table according to the calibration parameters of the multiple cameras; performing image fusion processing on the real-time image data of the multiple cameras based on the dynamic projection lookup table and pre-set fusion rules to generate a panoramic view for surround monitoring of the boarding bridge; and a monitoring screen display system for displaying the panoramic view in real time.

[0015] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned boarding bridge surround monitoring methods by executing the executable instructions.

[0016] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for monitoring the surrounding view of an aerobridge described in any one of the above is implemented.

[0017] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the computer program implements any one of the above-mentioned boarding bridge surround monitoring methods.

[0018] The embodiments of the present invention provide a method, device, system and related equipment for surround monitoring of a boarding bridge. By arranging multiple cameras on the outside of the boarding bridge and obtaining calibration parameters of the multiple cameras, a dynamic projection lookup table is constructed based on the calibration parameters of the multiple cameras. The real-time image data of the multiple cameras are subjected to image fusion processing based on the dynamic projection lookup table and pre-set fusion rules. A panoramic view for surround monitoring of the boarding bridge is generated and displayed in real time on the monitoring screen. The operator can easily judge the situation around the boarding bridge by observing the external display.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A schematic diagram showing a system structure of a method for monitoring a boarding bridge surround view in an embodiment of the present disclosure;

[0022] Figure 2 A flow chart of a method for monitoring a boarding bridge around view in an embodiment of the present disclosure is shown;

[0023] Figure 3 A schematic diagram of the structure of a boarding bridge in an embodiment of the present disclosure is shown;

[0024] Figure 4 A schematic diagram of camera installation in an embodiment of the present disclosure is shown;

[0025] Figure 5 A schematic diagram showing the relative movement of a boarding bridge in an embodiment of the present disclosure is shown;

[0026] Figure 6 A schematic diagram showing an example of a display screen of an aerobridge in an embodiment of the present disclosure is shown;

[0027] Figure 7 A schematic diagram of the bottom area of ​​a boarding bridge in an embodiment of the present disclosure is shown;

[0028] Figure 8 A schematic diagram showing an example of a display screen of an aerobridge in an embodiment of the present disclosure is shown;

[0029] Fig. 9 A schematic diagram of a boarding bridge surround monitoring device according to an embodiment of the present disclosure is shown;

[0030] Fig.10 A schematic diagram of a boarding bridge surround monitoring system according to an embodiment of the present disclosure is shown;

[0031] Fig.11 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0033] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0034] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0035] Figure 1 FIG. 2 shows an exemplary application system architecture diagram to which the boarding bridge surround monitoring method in the embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture may include a terminal device 101 , a network 102 and a server 103 .

[0036] The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.

[0037] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0038] The terminal device 101 can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0039] Optionally, the client of the application installed in different terminal devices 101 is the same, or the client of the same type of application based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile client, a PC client, etc.

[0040] The server 103 may be a server that provides various services, such as a background management server that provides support for the device operated by the user using the terminal device 101. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device.

[0041] Optionally, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0042] Those skilled in the art will know that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration, and any number of terminal devices, networks and servers may be provided according to actual needs, and the embodiments of the present disclosure do not limit this.

[0043] Under the above system architecture, a method for surround monitoring of a boarding bridge is provided in an embodiment of the present disclosure, and the method can be executed by any electronic device with computing and processing capabilities.

[0044] In some embodiments, the surround monitoring method for the boarding bridge provided in the embodiments of the present disclosure can be executed by the terminal device of the above-mentioned system architecture; in other embodiments, the surround monitoring method for the boarding bridge provided in the embodiments of the present disclosure can be executed by the server in the above-mentioned system architecture; in other embodiments, the surround monitoring method for the boarding bridge provided in the embodiments of the present disclosure can be implemented by the terminal device and the server in the above-mentioned system architecture through interaction.

[0045] First of all, in response to the above-mentioned problems, a method for surround monitoring of a boarding bridge is provided in an embodiment of the present disclosure, which can be applied to any boarding bridge, ship boarding bridge or other similar equipment that requires surround monitoring. Compared with the related art, there is a visual blind spot problem when the operator observes the boarding bridge when it moves, and the number of cameras needs to be increased. However, the excessive number of camera screens also causes the operator to monitor multiple cameras in real time, which makes the operator easily defocused, reducing the anti-collision effect. The embodiment of the present disclosure arranges multiple cameras on the outside of the boarding bridge, constructs a dynamic projection lookup table based on the calibration parameters of multiple cameras, performs image fusion processing on the real-time image data obtained from multiple cameras, and generates a panoramic view for surround monitoring of the boarding bridge. The number of display screens can be appropriately reduced, and the operator can easily judge the situation around the boarding bridge by observing the external display.

[0046] Figure 2 A flow chart of a method for monitoring a boarding bridge surround view in an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the boarding bridge surround monitoring method provided in the embodiment of the present disclosure includes the following steps:

[0047] S202, obtaining calibration parameters of a plurality of cameras, wherein the plurality of cameras are used to capture monitoring images of a plurality of positions of the boarding bridge to be monitored.

[0048] It should be noted that the camera in the embodiment of the present disclosure can be any image acquisition device capable of shooting high-definition images, such as a device with a high-definition camera; in addition, the calibration parameters of the camera include internal parameters and external parameters, wherein the internal parameters describe the properties of the camera itself and are independent of the position of the camera relative to the object being photographed. They are mainly used to convert pixel coordinates into normalized coordinates in the camera coordinate system, and the internal parameters include at least: the focal length of the camera, the principal point offset, the radial distortion coefficient, the tangential distortion coefficient, the pixel aspect ratio, and the eccentric distortion; the external parameters describe the position and posture of the camera relative to the world coordinate system, and are used to convert points in the camera coordinate system to the world coordinate system, and the external parameters include at least: the rotation matrix and the translation vector.

[0049] In some embodiments, the focal length of the camera in the embodiments of the present disclosure represents the focal length of the camera lens, which is expressed in pixel units, and usually has two values ​​corresponding to the focal lengths in the x-axis and y-axis directions respectively; the principal point offset refers to the coordinate position of the image center (i.e., the optical center or the principal point) on the image plane. Ideally, it should be located at the center of the image, but there may be deviations in practice; the radial distortion coefficient describes the degree of image deformation caused by the curvature of the lens. The radial distortion will cause the straight line at the edge of the image to look curved; the tangential distortion coefficient is caused by the asymmetry in the lens manufacturing process, which is manifested as the image plane tilting relative to the ideal position; the pixel aspect ratio describes the proportional relationship between the width and height of each pixel. If the sensor is a square pixel, the ratio is 1; if not, this parameter needs to be considered to ensure correct mapping; eccentric distortion is a relatively rare parameter used to describe the image offset caused by improper lens installation; the rotation matrix represents the rotation angle of the camera coordinate system relative to the world coordinate system, which can describe the direction of the camera; the translation vector is a 3D vector that represents the position of the origin of the camera coordinate system relative to the origin of the world coordinate system, and describes the position of the camera.

[0050] In some embodiments, the embodiments of the present disclosure can accurately map the two-dimensional coordinates in the image captured by the camera back to the real position in the three-dimensional space through the above calibration method.

[0051] S204, constructing a dynamic projection lookup table according to calibration parameters of multiple cameras.

[0052] It should be noted that the dynamic projection lookup table in the embodiment of the present disclosure is obtained by reconstructing the three-dimensional scene through multi-perspective images to achieve accurate mapping between the two-dimensional image and the three-dimensional space. The dynamic projection lookup table can be directly queried when needed instead of recalculating it each time, thereby improving image processing efficiency.

[0053] S206, acquiring real-time image data from multiple cameras, performing image fusion processing based on a dynamic projection lookup table and pre-set fusion rules, and generating a panoramic view for surround monitoring of the boarding bridge.

[0054] In some embodiments, the fusion rules in the embodiments of the present disclosure include: fusing the surveillance images taken by the cameras installed on both sides of the outer channel of the boarding bridge and the surveillance images taken by the cameras installed at the bottom of the outer channel into one surveillance image; fusing the surveillance images taken by the cameras installed on both sides of the boarding bridge's pick-up port into one surveillance image; splicing the fused surveillance images of the outer channel of the boarding bridge and the fused surveillance images of the boarding bridge's pick-up port into one surveillance image through image splicing technology; or, fusing the surveillance images taken by the cameras installed on both sides of the outer channel of the boarding bridge into one surveillance image; fusing the surveillance images taken by the cameras installed on both sides of the boarding bridge's pick-up port into one surveillance image; splicing the fused surveillance images of the outer channel of the boarding bridge and the fused surveillance images of the boarding bridge's pick-up port into one surveillance image through image splicing technology. It should be noted that in the overlapping area of ​​the surveillance images captured by the cameras on both sides of the outer channel and the bottom of the outer channel, because of the existence of service ladders, ladder platforms and other structures for personnel passage or cargo transportation, the content projected to the ground observed by the cameras is not exactly the same. Therefore, it is necessary to adopt a semi-transparent processing method for the boarding bridge itself, and display the content displayed by the cameras at the top and bottom of the channel at the same time to ensure the best monitoring effect.

[0055] In some embodiments, the disclosed embodiments fuse the surveillance images collected by the cameras on both sides of the pick-up port into one image; the surveillance images after the channel fusion and the surveillance images after the pick-up port fusion are combined into a 2D image using stitching technology; the camera installed in front of the pick-up port and the camera installed near the rotating platform do not participate in the image fusion and stitching, and the original 3D image is used for display; the above fusion and display requirements ensure that the situation around the channel and the wheel frame and the bottom of the pick-up port can be quickly understood through the 2D image. The perspective of the rotating platform camera and the camera facing the pick-up port is mainly forward-facing, and the use of 3D original image display can better understand the information in the height direction.

[0056] In some embodiments, the method for monitoring the boarding bridge around view in the embodiment of the present disclosure includes, first, obtaining calibration parameters of multiple cameras; second, constructing a dynamic projection lookup table according to the calibration parameters of multiple cameras; then, obtaining real-time image data of multiple cameras, performing image fusion processing based on the dynamic projection lookup table and the pre-set fusion rules, and generating a panoramic view for monitoring the boarding bridge around view; finally, performing image fusion processing on the monitoring images collected by multiple cameras based on the dynamic projection lookup table, and generating a panoramic view for monitoring the boarding bridge around view. Compared with the related art, the operator has a visual blind spot problem when observing the boarding bridge when it moves, and the number of cameras needs to be increased. However, the excessive number of camera images also causes the operator to monitor multiple cameras in real time, and the operator is prone to defocusing, which reduces the anti-collision effect. In the embodiment of the present disclosure, multiple cameras are arranged outside the boarding bridge, and a dynamic projection lookup table is constructed based on the calibration parameters of multiple cameras. The real-time image data of multiple cameras obtained are subjected to image fusion processing to generate a panoramic view for monitoring the boarding bridge around view, and the display screen is appropriately reduced. The operator can easily judge the situation around the boarding bridge by observing the external display.

[0057] In some embodiments, the boarding bridge in the embodiment of the present disclosure includes: an outer channel, a pick-up port, and a rotating platform. Before obtaining the calibration parameters of multiple cameras, the boarding bridge surround monitoring method in the embodiment of the present disclosure also includes: setting a first camera, a second camera, a third camera, and a fourth camera on both sides of the outer channel and near the top of the outer channel; setting a fifth camera and a sixth camera on both sides of the outer channel and near the bottom of the outer channel; setting a seventh camera and an eighth camera on both sides of the pick-up port and near the top of the pick-up port; setting a ninth camera on the side of the pick-up port facing the boarding user and near the top of the pick-up port; and setting a tenth camera on one side of the rotating platform and near the bottom of the rotating platform. Specifically, as Figure 3 As shown, the external area of ​​the boarding bridge in the embodiment of the present disclosure is generally divided into a receiving port, a passage and a rotating platform, wherein the passage includes an inner passage and an outer passage. If the boarding bridge in the embodiment of the present disclosure is a three-section bridge, it also includes a middle passage.

[0058] In more detail, Figure 3 As shown, the rotating platform in the disclosed embodiment is the connecting part between the boarding bridge and the terminal building (or fixed bridge), and its bottom is fixed on the column connected to the apron foundation. It is the hinge support center of the horizontal rotation movement center and the lifting movement of the boarding bridge. Because it is connected to the terminal building, it is a fixed part and has no collision risk and does not need to be monitored.

[0059] In some embodiments, Figure 3As shown, there is relative movement between the inner channel and the middle channel (only available in 3-section bridge) and the outer channel in the embodiment of the present disclosure, and the gap between the channels is relatively small. If the camera is installed on the outer wall of the inner channel or the middle channel, the channel will cause interference between the channel side wall and the camera when it is extended or retracted. Taking all factors into consideration, the camera used to monitor the inner channel and the middle channel (only available in 3-section bridge) in the embodiment of the present disclosure is usually installed at the bottom of the rotating platform / inner channel of the boarding bridge, so that the bottom area of ​​the inner channel and the middle channel (only available in 3-section bridge) and the overall protection along the height direction of the channel can be achieved.

[0060] In some embodiments, the outer passage and pick-up area in the disclosed embodiment may install cameras at the top of the side of the passage and pick-up area, so that the collision risk in the entire height direction of the boarding bridge can be monitored from a bird's-eye view.

[0061] In some embodiments, the disclosed embodiments also require that a camera be installed at the bottom of the channel to compensate for the blind spot of the camera that monitors the wheel frame area from the turntable.

[0062] In some embodiments, the disclosed embodiments require that a forward-looking camera be installed at the front of the pick-up port in order to observe the status of the area in front of the pick-up port.

[0063] In some embodiments, Figure 4 As shown, the embodiment of the present disclosure completes the installation of the camera according to the above description. Specifically, four cameras are installed at the top of the two side walls of the outer channel, namely, the first camera 401, the second camera 402, the third camera 403 and the fourth camera 404; for a boarding bridge with a complex outer channel contour, such as a service ladder, a luggage chute and other protruding structures, it is necessary to adjust the camera installation position to avoid the obstruction of the protruding obstacles. If the boarding bridge is long, 6 or more cameras can also be installed; at the bottom of the outer channel, one camera is installed at each of the front and rear positions of the wheel frame, namely, the fifth camera 405 and the sixth camera 406, which are used to monitor the front and rear positions of the wheel frame below the channel and the service ladder below. Obstacles; a camera is installed at the top of the left and right sides of the pick-up port, namely the seventh camera 407 and the eighth camera 408, which are used to observe the areas on the left and right sides of the pick-up port, so as to meet the real-time monitoring of the pick-up port when it is rotated to any angle; a ninth camera 409 facing forward is installed on the side of the pick-up port facing the boarding user, that is, in front, to observe the area in front of the pick-up port; a tenth camera 410 is installed at the rotating platform position to observe pedestrians on the ground, obstacles, and collision risks on both sides of the inner channel and the middle channel (only three-section bridges have) from the rotating platform to the wheel frame; all the above cameras together cover a 360° area around the channel, the pick-up port and the bottom of the channel.

[0064] In some embodiments, the disclosed embodiments obtain real-time image data from multiple cameras, perform image fusion processing based on a dynamic projection lookup table and pre-set fusion rules, and generate a panoramic view for surround monitoring of the boarding bridge, including: obtaining real-time image data from multiple cameras, and obtaining a mapping image with overlapping areas based on a dynamic projection lookup table; performing image fusion processing on the mapping image based on pre-set fusion rules to generate a panoramic view for surround monitoring of the boarding bridge. Specifically, the overlapping area is the basis for image fusion and blind spot-free display after image splicing. It should be noted that the disclosed embodiments obtain a mapping image with overlapping areas based on a dynamic projection lookup table, such as Figure 4 As shown, there are overlapping areas between the surveillance images of the first camera 401, the second camera 402 and the sixth camera 406; there are overlapping areas between the surveillance images of the sixth camera 406 and the fifth camera 405; there are overlapping areas between the surveillance images of the fifth camera 405 and the third camera 403, the fourth camera 404; there are also overlapping areas between the third camera 403, the fourth camera 404 and the seventh camera 407, the eighth camera 408.

[0065] In some embodiments, the embodiments of the present disclosure obtain calibration parameters of multiple cameras, including: after adjusting the boarding bridge to a preset posture and a preset position, calibrating the internal parameters of the multiple cameras; based on the internal parameters of the multiple cameras, calibrating the external parameters of the multiple cameras relative to the boarding bridge and the ground.

[0066] In some embodiments, the present disclosure can calibrate the camera through the following calibration methods:

[0067] Checkerboard calibration method: Use known geometric shapes (such as a checkerboard) as reference objects and calculate the internal and external parameters of the camera by taking multiple photos at different angles.

[0068] Self-calibration technology: Under certain specific conditions, the system can automatically adjust and optimize internal and external parameters based on the data it collects.

[0069] Professional calibration tools: such as laser scanners, precision measuring instruments, etc., provide calibration results with higher accuracy.

[0070] In some embodiments, the embodiments of the present disclosure construct a dynamic projection lookup table based on the calibration parameters of multiple cameras, including: obtaining images generated by multiple cameras shooting the same scene area; performing distortion correction on the images based on the calibrated internal parameters of the multiple cameras to obtain a distortion correction map; performing perspective transformation on the distortion correction map based on the calibrated external parameters of the multiple cameras to obtain a ground projection map; determining the fusion boundary and fusion angle of adjacent ground projection maps to obtain a bird's-eye view; using a partitioned projection method to project the near-area image and the far-area image corresponding to the ground projection map to the corresponding areas of a pre-set 3D model, wherein the near-area image and the far-area image corresponding to the ground projection map are represented as a bird's-eye view obtained by converting images captured at areas near and far from the camera; performing multi-perspective observation of the 3D model and generating a corresponding dynamic lookup table. Specifically, the embodiment of the present disclosure performs distortion correction and perspective transformation on the image, determines the fusion boundary and fusion angle of adjacent ground projection images, obtains a bird's-eye view, and uses a partitioned projection method to project the near-area image and the far-area image corresponding to the ground projection image to the corresponding areas of a pre-set 3D model, respectively. Finally, the 3D model is observed from multiple perspectives and a corresponding dynamic lookup table is generated, so that the current static multi-perspective panoramic image can be subsequently generated using the obtained dynamic lookup table.

[0071] In some embodiments, the disclosed embodiment acquires real-time image data from multiple cameras, performs image fusion processing based on a dynamic projection lookup table and pre-set fusion rules, and before generating a panoramic view for surround monitoring of the boarding bridge, the boarding bridge surround monitoring method in the disclosed embodiment also includes: acquiring real-time image data from multiple cameras, performing image processing on the real-time image data to obtain image data for image fusion processing, wherein the image processing includes at least one or more of the following: image denoising processing, image edge extraction, image histogram processing, and image transformation processing. In more detail, during the operation of the boarding bridge, the extension and retraction of the channel, the change in the height of the lifting columns, and the change in the pitch of the channel will cause the camera to tilt relative to the ground, thereby causing the image projected to the ground to be offset and distorted. In order to maintain a better display effect, real-time camera posture correction is required through algorithms; the camera image needs to perform basic operations including digital image processing, such as denoising, edge extraction, histogram processing, etc., as well as establishing image matching templates and transforming the image (such as Fourier transform, wavelet transform, etc.); select the appropriate algorithm when stitching the images: select the appropriate stitching algorithm according to the specific application scenario and requirements. For example, for applications that need to maintain feature details, you can choose a feature matching algorithm; for applications that require a smooth transition, you can choose a fusion algorithm.

[0072] In some embodiments, when the boarding bridge receiving gate in the disclosed embodiments rotates, the relative position between the camera at the receiving gate and the passage of the boarding bridge will change, requiring real-time adjustment of the screen display.

[0073] In some embodiments, the embodiments of the present disclosure select positioning reference points for screen display, specifically including: for the channel camera screen and the reception port camera screen, the position relative to the apron ground will change with the telescopic movement of the channel and the rotational movement of the channel around the center of the rotating platform and the rotation of the reception port, while the camera at the rotating platform position is fixed relative to the inner channel; the camera looking forward from the reception port is fixed relative to the reception port.

[0074] In some embodiments, Figure 5 As shown, for the cameras of the passage and the pick-up gate, considering that the rotation angle range of the pick-up gate relative to the passage is relatively large (such as 90° left turn and 45° right turn), in order to ensure the stability of the boarding bridge image, refer to the conventional 360° image display scheme of the car, define the rear end of the outer passage of the boarding bridge as the end face, as the bottom boundary of the video image, the passage part of the boarding bridge is perpendicular to the bottom of the image, and the angle of the pick-up gate changes with the change of the pick-up gate angle. The display position of the passage image needs to be converted according to the current bridge body angle and the passage length. The display of the pick-up gate image needs to synchronously consider the bridge body angle, the passage length and the pick-up gate angle, as shown in Figure 6 As shown, the status information of the boarding bridge provided by the boarding bridge PLC is updated in real time. After the PLC collects the current bridge body angle, channel length and receiving port angle information, it is sent to the system corresponding to the boarding bridge surround monitoring method through communication, which can also be called a 360° surround system. After receiving the position information, the 360° surround system performs calculations based on the current position information, adjusts the coordinates of the picture, re-splices and other operations. This series of operations will affect the real-time nature of the picture to a certain extent. For the safety of the operation after the picture, some dangerous situations cannot be handled in time. In order to avoid the adverse effects of the lag of the picture, Kalman filtering and other related technologies are used to realize the estimation of the state information of the boarding bridge and realize the early acquisition of the position information state. The specific implementation method includes: the boarding bridge transmits the current position information while transmitting the current movement direction and movement speed information. Through the received direction and speed information, the 360° surround system can predict the change trend of the position information related to the boarding bridge through internal calculation, so as to make calculation adjustments in advance, further improve the accuracy and efficiency of image processing, and perform secondary verification after receiving the position information fed back by the boarding bridge next time.

[0075] For cameras near the rotating platform and the cameras facing the front of the receiving port, the image display is centered on the camera, which is convenient and efficient.

[0076] In some embodiments, when performing image fusion processing, the embodiments of the present disclosure use a dynamic lookup table method to improve processing speed and efficiency and reduce the loss and distortion of image information. Image fusion processes stitching gaps and brightness differences to make the stitched image look more natural. The aligned images are subjected to weighted averaging, multi-band fusion or other techniques to reduce stitching marks and improve the overall image quality. In order to improve stitching efficiency, the details or smooth transitions of the original image are not retained.

[0077] At the same time, the embodiments of the present disclosure can also integrate the detection function of AI through the computing controller, incorporate active safety functions, detect surrounding personnel or typical obstacles, and further improve operational safety through alarm signal interfaces and video screen reminders.

[0078] In some embodiments, a service ladder or ladder platform is configured on the boarding bridge channel in the embodiments of the present disclosure, and aircraft air conditioning, telescopic air duct or 400HZ and other equipment are configured below the channel, such as Figure 7 As shown in the figure, the main features of this type of boarding bridge are as follows: Due to the influence of the service ladder, the situation at the bottom of the service ladder cannot be reflected when looking down from the top of the channel, and the camera at the bottom of the channel cannot reflect the situation above the service ladder. For scenes with many devices under the round platform, when using 2D view display, due to the occlusion of the equipment, a large shadow area will be formed after being projected to the ground, which cannot truly reflect the actual situation.

[0079] For the above-mentioned types of boarding bridges, the camera installation plan remains basically the same, but the picture fusion and display plan are adjusted as follows: the cameras on both sides of the outer channel are fused into one picture separately; the cameras at the bottom of the outer channel are synthesized into one picture by editing; the cameras on both sides of the terminal are fused into one picture separately.

[0080] The camera fusion images on both sides of the external channel and the fusion image of the receiving port are synthesized into one image using splicing technology. The ninth camera at the receiving port and the tenth camera near the rotating platform do not participate in the image fusion.

[0081] The above fusion and display requirements, such as Figure 8As shown, the 2D picture from a bird's-eye view is used to monitor the full height of the boarding bridge's outer contour to prevent collisions with surrounding objects. At the same time, it can also better reflect the status of personnel on the service ladder and ladder platform. The camera at the bottom of the channel uses a 3D view to edit the camera images before and after the wheel frame, which can avoid the numerous equipment at the bottom of the channel (such as 400HZ power supply, PCA aircraft air conditioning duct, etc.) affecting the 2D projection effect, and can more realistically reflect the situation near the wheel frame, at the bottom of the service ladder, at the bottom of the channel, and at the bottom of the aircraft receiving port. The view angle of the rotating platform camera and the camera facing forward at the aircraft receiving port is mainly forward-facing, and the use of 3D original image display can better understand the information in the height direction.

[0082] Based on the same inventive concept, the disclosed embodiment also provides a boarding bridge surround monitoring device, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0083] Fig. 9 A schematic diagram of a boarding bridge surround monitoring device in an embodiment of the present disclosure is shown, and the device includes:

[0084] The parameter calibration module 901 is used to obtain calibration parameters of multiple cameras, wherein the multiple cameras are used to collect monitoring images of multiple positions of the boarding bridge to be monitored;

[0085] A dynamic projection lookup table construction module 902 is used to construct a dynamic projection lookup table according to calibration parameters of multiple cameras;

[0086] A panoramic view processing module 903 is used to obtain real-time image data from multiple cameras, perform image fusion processing based on a dynamic projection lookup table and preset fusion rules, and generate a panoramic view for surround monitoring of the boarding bridge;

[0087] The panoramic view display module 904 is used to display the panoramic view in real time in the monitoring screen.

[0088] In some embodiments, the boarding bridge surround monitoring device in the embodiments of the present disclosure obtains calibration parameters of multiple cameras through a parameter calibration module; constructs a dynamic projection lookup table according to the calibration parameters of multiple cameras through a dynamic projection lookup table construction module; obtains real-time image data of multiple cameras through a panoramic view processing module, performs image fusion processing based on the dynamic projection lookup table and pre-set fusion rules, and generates a panoramic view for surround monitoring of the boarding bridge; performs image fusion processing on the monitoring images captured by multiple cameras based on the dynamic projection lookup table through a panoramic view display module, and generates a panoramic view for surround monitoring of the boarding bridge. Compared with the related art, the operator has a visual blind spot problem when observing the boarding bridge when it moves, and the number of cameras needs to be increased. However, the excessive number of camera images also causes the operator to monitor multiple cameras in real time, which makes the operator easily defocused, reducing the anti-collision effect. The embodiment of the present disclosure arranges multiple cameras outside the boarding bridge, constructs a dynamic projection lookup table based on the calibration parameters of the multiple cameras, performs image fusion processing on the real-time image data obtained from the multiple cameras, generates a panoramic view for monitoring the boarding bridge, and appropriately reduces the display screen. The operator can easily judge the situation around the boarding bridge by observing the external display.

[0089] In some embodiments, the panoramic view processing module in the embodiments of the present disclosure is also used to acquire real-time image data from multiple cameras, and obtain a mapping image with overlapping areas based on a dynamic projection lookup table; perform image fusion processing on the mapping image based on pre-set fusion rules to generate a panoramic view for surround monitoring of the boarding bridge.

[0090] In some embodiments, the fusion rules in the embodiments of the present disclosure include: fusing the surveillance images taken by the cameras installed on both sides of the outer channel of the boarding bridge and the surveillance images taken by the cameras installed at the bottom of the outer channel into one surveillance image; fusing the surveillance images taken by the cameras installed on both sides of the boarding bridge's pick-up port into one surveillance image; splicing the fused surveillance images of the outer channel of the boarding bridge and the fused surveillance images of the boarding bridge's pick-up port into one surveillance image through image splicing technology; or, fusing the surveillance images taken by the cameras installed on both sides of the outer channel of the boarding bridge into one surveillance image; fusing the surveillance images taken by the cameras installed on both sides of the boarding bridge's pick-up port into one surveillance image; splicing the fused surveillance images of the outer channel of the boarding bridge and the fused surveillance images of the boarding bridge's pick-up port into one surveillance image through image splicing technology.

[0091] In some embodiments, the parameter calibration module in the embodiment of the present disclosure is also used to calibrate the internal parameters of multiple cameras after the boarding bridge is adjusted to a preset posture and a preset position; based on the internal parameters of the multiple cameras, calibrate the external parameters of the multiple cameras relative to the boarding bridge and the ground.

[0092] In some embodiments, the dynamic projection lookup table construction module in the embodiments of the present disclosure is also used to obtain images generated by multiple cameras shooting the same scene area; based on the calibrated internal parameters of the multiple cameras, the images are distortion corrected to obtain a distortion correction map; based on the calibrated external parameters of the multiple cameras, the distortion correction map is perspective transformed to obtain a ground projection map; the fusion boundaries and fusion angles of adjacent ground projection maps are determined to obtain a bird's-eye view; the near-area image and the far-area image corresponding to the ground projection map are projected to the corresponding areas of a pre-set 3D model using a partitioned projection method, wherein the near-area image and the far-area image corresponding to the ground projection map are represented as a bird's-eye view obtained by converting images captured at areas near and far from the camera; the 3D model is observed from multiple perspectives and a corresponding dynamic lookup table is generated.

[0093] In some embodiments, the boarding bridge surround monitoring device in the embodiments of the present disclosure further includes: an image processing module, which is used to obtain real-time image data from multiple cameras, perform image fusion processing based on a dynamic projection lookup table and pre-set fusion rules, and before generating a panoramic view for surround monitoring of the boarding bridge, obtain real-time image data from multiple cameras, perform image processing on the real-time image data to obtain image data for image fusion processing, wherein the image processing includes at least one or more of the following: image denoising processing, image edge extraction, image histogram processing, and image transformation processing.

[0094] Based on the same inventive concept, the disclosed embodiment also provides a boarding bridge surround monitoring system, as described in the following embodiment. Since the principle of solving the problem in the system embodiment is similar to that in the above method embodiment, the implementation of the system embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0095] Fig.10 A schematic diagram of a boarding bridge surround monitoring system in an embodiment of the present disclosure is shown, and the system includes:

[0096] Camera system 1001, used to obtain real-time image data from multiple cameras;

[0097] The image processing system 1002 is used to obtain calibration parameters of multiple cameras, wherein the multiple cameras are used to collect monitoring images of multiple positions of the boarding bridge to be monitored; construct a dynamic projection lookup table according to the calibration parameters of the multiple cameras; perform image fusion processing on the real-time image data of the multiple cameras based on the dynamic projection lookup table and a pre-set fusion rule to generate a panoramic view for monitoring the boarding bridge;

[0098] The monitoring screen display system 1003 is used to display the panoramic view in real time.

[0099] In some embodiments, the boarding bridge surround monitoring system in the disclosed embodiment obtains images of multiple cameras through a camera system, obtains calibration parameters of multiple cameras used to collect monitoring images of multiple positions of the boarding bridge to be monitored through an image processing system; secondly, a dynamic projection lookup table is constructed according to the calibration parameters of multiple cameras, and image fusion processing is performed on the real-time image data of multiple cameras based on the dynamic projection lookup table and the pre-set fusion rules to generate a panoramic view for surround monitoring of the boarding bridge; finally, the panoramic view is displayed in real time through the monitoring screen display system. Compared with the related art, the operator has a visual blind spot problem when observing the boarding bridge when it moves, and the number of cameras needs to be increased. However, the excessive number of camera images also causes the operator to monitor multiple cameras in real time, and the operator is prone to defocusing, which reduces the anti-collision effect. In the disclosed embodiment, multiple cameras are arranged outside the boarding bridge, a dynamic projection lookup table is constructed based on the calibration parameters of multiple cameras, and image fusion processing is performed on the real-time image data of multiple cameras to generate a panoramic view for surround monitoring of the boarding bridge, and the display screen is appropriately reduced. The operator can easily judge the situation around the boarding bridge by observing the external display.

[0100] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above method embodiments. It should be noted that the above modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0101] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0102] Refer to the following Fig.11 1100 according to this embodiment of the present disclosure is described. Fig.11 The electronic device 1100 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0103] like Fig.11 As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include but are not limited to: at least one processing unit 1110, at least one storage unit 1120, and a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110).

[0104] The storage unit stores program codes, which can be executed by the processing unit 1110, so that the processing unit 1110 performs the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 1110 can perform the following steps of the above method embodiment:

[0105] The calibration parameters of multiple cameras are obtained, wherein the multiple cameras are used to collect monitoring images of the boarding bridge to be monitored in multiple directions; a dynamic projection lookup table is constructed according to the calibration parameters of the multiple cameras, and the dynamic projection lookup table is used to map the two-dimensional pixel points in the monitoring images collected by each camera into a three-dimensional space; the monitoring images collected by the multiple cameras are subjected to image fusion processing based on the dynamic projection lookup table to generate a panoramic view for surround monitoring of the boarding bridge.

[0106] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 11201 and / or a cache storage unit 11202 , and may further include a read-only storage unit (ROM) 11203 .

[0107] The storage unit 1120 may also include a program / utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0108] Bus 1130 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0109] The electronic device 1100 may also communicate with one or more external devices 1140 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1100, and / or communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1160. As shown, the network adapter 1160 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0110] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0111] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer program product, which includes: a computer program, which implements the method in the above embodiment when executed by a processor.

[0112] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0113] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0114] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0115] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0116] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0117] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0118] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0119] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0120] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for monitoring a boarding bridge, characterized in that: include: Acquire calibration parameters of a plurality of cameras, wherein the plurality of cameras are used to collect monitoring images of a plurality of positions of the boarding bridge to be monitored; Constructing a dynamic projection lookup table according to calibration parameters of multiple cameras; Acquire real-time image data from multiple cameras, perform image fusion processing based on the dynamic projection lookup table and pre-set fusion rules, and generate a panoramic view for monitoring the boarding bridge; The panoramic view is displayed in real time on the monitoring screen.

2. The boarding bridge surround monitoring method according to claim 1, characterized in that: Acquire real-time image data of multiple cameras, perform image fusion processing based on the dynamic projection lookup table and the preset fusion rules, and generate a panoramic view for monitoring the boarding bridge, including: Acquire real-time image data of multiple cameras, and obtain a mapping image with overlapping areas based on the dynamic projection lookup table; The mapping images are subjected to image fusion processing based on a preset fusion rule to generate a panoramic view for monitoring the boarding bridge.

3. The boarding bridge surround monitoring method according to claim 1, characterized in that: The fusion rules include: The monitoring images captured by the cameras arranged at both sides of the outer passage of the boarding bridge and the monitoring images captured by the cameras arranged at the bottom of the outer passage are merged into one monitoring image; Merging the surveillance images captured by the cameras arranged on both sides of the boarding bridge's receiving gate into one surveillance image; The monitoring picture after the external channel of the boarding bridge is merged and the monitoring picture after the boarding bridge's pick-up gate is merged are spliced ​​into one monitoring picture by using a picture splicing technology; or, Merging the monitoring images captured by the cameras arranged on both sides of the outer passage of the boarding bridge into one monitoring image; Merging the surveillance images captured by the cameras arranged on both sides of the boarding bridge's receiving gate into one surveillance image; The monitoring picture after the external channel of the boarding bridge is merged and the monitoring picture after the reception port of the boarding bridge is merged are spliced ​​into one monitoring picture through picture splicing technology.

4. The boarding bridge surround monitoring method according to claim 1, characterized in that: Get calibration parameters of multiple cameras, including: After the boarding bridge is adjusted to a preset posture and a preset position, internal parameters of multiple cameras are calibrated; Based on the internal parameters of the multiple cameras, the external parameters of the multiple cameras relative to the boarding bridge and the ground are calibrated.

5. The boarding bridge surround monitoring method according to claim 1, characterized in that: A dynamic projection lookup table is constructed based on the calibration parameters of multiple cameras, including: Acquire images generated by multiple cameras shooting the same scene area; Performing distortion correction on the image based on the calibrated internal parameters of the multiple cameras to obtain a distortion correction map; Performing perspective transformation on the distortion correction image based on the calibrated external parameters of the multiple cameras to obtain a ground projection image; Determine the fusion boundary and fusion angle of adjacent ground projection images to obtain a bird's-eye view; The near-area image and the far-area image corresponding to the ground projection map are projected to the corresponding areas of the preset 3D model respectively by using a partition projection method, wherein the near-area image and the far-area image corresponding to the ground projection map are represented as the bird's-eye view obtained by converting the images captured at the areas far from the camera; The 3D model is observed from multiple perspectives and a corresponding dynamic lookup table is generated.

6. The boarding bridge surround monitoring method according to claim 1, characterized in that: Before acquiring real-time image data of a plurality of cameras and performing image fusion processing based on the dynamic projection lookup table and a preset fusion rule to generate a panoramic view for monitoring the boarding bridge, the method further comprises: Real-time image data of multiple cameras are acquired, and image processing is performed on the real-time image data to obtain image data for image fusion processing, wherein the image processing includes at least one or more of the following: image denoising processing, image edge extraction, image histogram processing, and image transformation processing.

7. A boarding bridge surround monitoring device, characterized in that: include: A parameter calibration module, used to obtain calibration parameters of multiple cameras, wherein the multiple cameras are used to collect monitoring images of multiple positions of the boarding bridge to be monitored; A dynamic projection lookup table construction module, used to construct a dynamic projection lookup table according to calibration parameters of multiple cameras; A panoramic view generation module, used to obtain real-time image data from multiple cameras, perform image fusion processing according to the dynamic projection lookup table and pre-set fusion rules, and generate a panoramic view for monitoring the boarding bridge; The panoramic view display module is used to display the panoramic view in real time in the monitoring screen.

8. A boarding bridge surround monitoring system, characterized in that: include: A camera system for acquiring real-time image data from multiple cameras; An image processing system is used to obtain calibration parameters of multiple cameras, wherein the multiple cameras are used to collect monitoring images of multiple directions of the boarding bridge to be monitored; construct a dynamic projection lookup table according to the calibration parameters of the multiple cameras; perform image fusion processing on the real-time image data of the multiple cameras based on the dynamic projection lookup table and a pre-set fusion rule to generate a panoramic view for monitoring the boarding bridge; The monitoring screen display system is used to display the panoramic view in real time.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the boarding bridge surround monitoring method as described in any one of claims 1 to 6 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the boarding bridge surround monitoring method described in any one of claims 1 to 6 is implemented.

Images