Aircraft cabin door recognition and positioning method and system, electronic equipment and storage medium
Patent Information
- Application Number
- CN202510095030.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-21
- Publication Date
- 2025-05-13
Smart Images
Figure CN119992037A_ABST
Abstract
Description
Background Art
[0002] With the development and progress of technology, the automatic docking technology of boarding bridges has begun to be applied to the docking of boarding bridges and cabin doors. When the boarding bridge automatically docks, it is first necessary to identify the aircraft cabin door.
[0003] Traditional methods rely mainly on image recognition technology to identify and dock aircraft doors of different models. These methods usually capture images of the aircraft door area through cameras, and then use image processing algorithms to identify features such as door markings or door gaps. Once these features are identified, they are used to calculate the specific location of the door, thereby guiding the boarding bridge for docking.
[0004] However, the paint on the aircraft fuselage may vary greatly depending on the airline, aircraft model or personalized needs. Some paints will cover or blur key features such as door markings or door gaps, making it difficult for image recognition algorithms to accurately identify them. In addition, changes in lighting conditions (such as strong light, backlight, shadows, etc.) may also make features such as door markings or door gaps difficult to identify, resulting in failure or misidentification of door recognition. In view of this, how to solve the inaccurate identification and positioning of aircraft doors is a problem that needs to be solved urgently.
[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0006] The present disclosure provides an aircraft cabin door identification and positioning method, system, electronic device and storage medium, which at least to a certain extent overcome the problem of inaccurate aircraft cabin door identification and positioning in the related art.
[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.
[0008] According to one aspect of the present disclosure, a method for identifying and locating an aircraft door is provided, comprising: collecting scanning data of a target area on an aircraft fuselage, wherein the target area is an area including an observation window and a porthole, the observation window is a window arranged on the aircraft door, and the porthole is a window arranged outside the aircraft door and within a preset distance range from the observation window; identifying and extracting observation window feature information of the observation window and porthole feature information of the porthole from the scanning data of the target area; determining an aircraft model according to the observation window feature information and the porthole feature information to obtain fuselage layout information, wherein the fuselage layout information at least includes layout information of the aircraft door on the aircraft fuselage relative to the observation window and / or the porthole; determining relative posture information between the aircraft door and a boarding bridge according to the observation window feature information, the porthole feature information and the fuselage layout information, wherein the relative posture information is used to control the docking of the boarding bridge with the aircraft door.
[0009] In some embodiments, the collecting of scanning data of a target area on an aircraft fuselage includes: collecting two-dimensional image data, depth image data, and three-dimensional laser scanning data of the target area on the aircraft fuselage; and synchronously aligning the two-dimensional image data, depth image data, and three-dimensional laser scanning data of the target area on the aircraft fuselage to obtain the scanning data of the target area on the aircraft fuselage.
[0010] In some embodiments, the identifying and extracting observation window feature information of the observation window and porthole feature information of the porthole from the scanning data of the target area includes: extracting image edge contours from the two-dimensional image data of the target area to obtain the two-dimensional image edge contours of the observation window and the porthole; extracting three-dimensional spatial measurement features from the depth image data and / or three-dimensional laser scanning data of the target area to obtain three-dimensional spatial measurement information of the observation window and the porthole; fusing the two-dimensional image edge contour of the observation window and the three-dimensional spatial measurement information to obtain observation window feature information of the observation window; fusing the two-dimensional image edge contour of the porthole and the three-dimensional spatial measurement information to obtain the porthole feature information of the porthole.
[0011] In some embodiments, the extracting of image edge contours from the two-dimensional image data of the target area to obtain the two-dimensional image edge contours of the observation window and the porthole includes: constructing a first deep learning model; the first deep learning model is trained based on the two-dimensional image data of the target area on the aircraft fuselage; the two-dimensional image data of the target area is input into the first deep learning model, and the two-dimensional image edge contours of the observation window and the porthole are output; the extracting of three-dimensional spatial measurement features from the depth image data and / or three-dimensional laser scanning data of the target area to obtain three-dimensional spatial measurement information of the observation window and the porthole includes: constructing a second deep learning model; the second deep learning model is trained based on the depth image data and three-dimensional laser scanning data of the target area on the aircraft fuselage; the depth image data and / or three-dimensional laser scanning data of the target area is input into the second deep learning model, and the three-dimensional spatial measurement information of the observation window and the porthole is output.
[0012] In some embodiments, determining the aircraft model to obtain fuselage layout information based on the observation window feature information and the porthole feature information includes: matching the observation window feature information and the porthole feature information with a pre-established aircraft model database to determine the aircraft model and obtaining the fuselage layout information from the database.
[0013] In some embodiments, determining the relative position information of the aircraft cabin door and the boarding bridge according to the observation window feature information, the porthole feature information and the fuselage layout information includes: determining the spatial coordinates of the observation window and the porthole relative to the boarding bridge according to the observation window feature information and the porthole feature information; determining the corner point position of the aircraft cabin door according to the observation window feature information, the porthole feature information and the fuselage layout information; and calculating the relative position information of the aircraft cabin door and the boarding bridge based on the corner point position of the aircraft cabin door and the spatial coordinates of the observation window and the porthole relative to the boarding bridge.
[0014] In some embodiments, the target area also includes the area of the cockpit, and the cockpit is a window arranged at the front end of the aircraft cockpit; the method also includes: extracting cockpit feature information of the cockpit from the scanning data of the target area; determining the aircraft model according to the observation window feature information, the porthole feature information and the cockpit feature information to obtain the fuselage layout information; determining the relative position information of the aircraft cabin door and the boarding bridge according to the observation window feature information, the porthole feature information, the cockpit feature information and the fuselage layout information.
[0015] According to another aspect of the present disclosure, there is also provided an aircraft door identification and positioning system, comprising: an information acquisition module and a data processing module, wherein the information acquisition module and the data processing module are communicatively connected; the information acquisition module is used to acquire scanning data of a target area on an aircraft fuselage, wherein the target area is an area including an observation window and a porthole, the observation window is a window arranged on the aircraft door, and the porthole is a window arranged outside the aircraft door and within a preset distance range from the observation window; the data processing module is used to identify and extract observation window feature information of the observation window and porthole feature information of the porthole from the scanning data of the target area; according to the observation window feature information and the porthole feature information, determine the aircraft model to obtain fuselage layout information, wherein the fuselage layout information at least includes layout information of the aircraft door on the aircraft fuselage relative to the observation window and / or the porthole; according to the observation window feature information, the porthole feature information and the fuselage layout information, determine the relative posture information of the aircraft door and the boarding bridge, wherein the relative posture information is used to control the boarding bridge to dock with the aircraft door.
[0016] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned aircraft cabin door identification and positioning methods by executing the executable instructions.
[0017] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the aircraft cabin door identification and positioning method described in any one of the above is implemented.
[0018] According to another aspect of the present disclosure, a computer program product is provided, including: a computer program or instructions, wherein when the computer program or instructions are executed by a processor, any one of the above-mentioned aircraft cabin door identification and positioning methods is implemented.
[0019] In an embodiment of the present disclosure, a method for identifying and locating an aircraft door is provided, comprising: collecting scanning data of a target area on an aircraft fuselage, wherein the target area is an area including an observation window and a porthole, the observation window is a window arranged on the aircraft door, and the porthole is a window arranged outside the aircraft door and within a preset distance range from the observation window; identifying and extracting observation window feature information of the observation window and porthole feature information of the porthole from the scanning data of the target area; determining the aircraft model according to the observation window feature information and the porthole feature information to obtain fuselage layout information, wherein the fuselage layout information at least includes layout information of the aircraft door on the aircraft fuselage relative to the observation window and / or the porthole; determining the relative posture information of the aircraft door and the boarding bridge according to the observation window feature information, the porthole feature information and the fuselage layout information, wherein the relative posture information is used to control the boarding bridge to dock with the aircraft door. In this way, by collecting scanning data of the target area on the aircraft fuselage, accurately identifying the feature information of the observation window and the porthole, and combining the fuselage layout information, the aircraft door can still be accurately identified under complex coating and variable lighting conditions. In this way, by comprehensively using the characteristic information of the observation windows and portholes as well as the fuselage layout information, the effect of accurately identifying and locating the aircraft cabin door in a complex environment is achieved.
[0020] Furthermore, by improving the accuracy of aircraft door recognition, the relative position information of the aircraft door and the boarding bridge can be quickly determined, thereby achieving high-precision and high-efficiency docking.
[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0023] Figure 1 A schematic diagram showing an exemplary system structure of a method for identifying and locating an aircraft cabin door according to an embodiment of the present disclosure;
[0024] Figure 2 A flow chart of a method for identifying and locating an aircraft door in an embodiment of the present disclosure is shown;
[0025] Figure 3 A schematic diagram showing a target area on an aircraft fuselage according to an embodiment of the present disclosure;
[0026] Figure 4AA schematic diagram showing a two-dimensional image of a target area on an aircraft fuselage in an embodiment of the present disclosure is shown;
[0027] Figure 4B A schematic diagram of a three-dimensional point cloud image of a target area on an aircraft fuselage in an embodiment of the present disclosure is shown;
[0028] Figure 5 A flow chart of an optional aircraft door identification and positioning method in an embodiment of the present disclosure is shown;
[0029] Figure 6 A flow chart of an optional aircraft door identification and positioning method in an embodiment of the present disclosure is shown;
[0030] Figure 7 A flow chart of an optional aircraft door identification and positioning method in an embodiment of the present disclosure is shown;
[0031] Figure 8 A flow chart of an optional aircraft door identification and positioning method in an embodiment of the present disclosure is shown;
[0032] Fig. 9 A flow chart of an optional aircraft door identification and positioning method in an embodiment of the present disclosure is shown;
[0033] Fig.10 A flow chart of another aircraft door identification and positioning method according to an embodiment of the present disclosure is shown;
[0034] Fig.11 A flow chart of another aircraft door identification and positioning method according to an embodiment of the present disclosure is shown;
[0035] Fig.12 A flow chart of an aircraft door identification and positioning system according to an embodiment of the present disclosure is shown;
[0036] Fig.13 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0038] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0039] The publicly provided aircraft cabin door identification and positioning method is described in detail below with reference to the accompanying drawings and embodiments.
[0040] Figure 1 FIG. 1 is a schematic diagram showing an exemplary system architecture to which the aircraft cabin door identification and positioning method in the embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture may include a terminal device 101 , a network 102 and a server 103 .
[0041] The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.
[0042] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.
[0043] The terminal device 101 may be any scanning data acquisition electronic device, which is used to acquire scanning data of a target area on the fuselage of the aircraft, such as scanning data of an aircraft door and its surrounding environment, which may specifically include observation windows and portholes of the aircraft. Scanning data acquisition electronic devices include, but are not limited to, high-definition cameras, infrared cameras, thermal imagers, 360-degree panoramic cameras, drone-mounted cameras, depth cameras, laser scanners, industrial-grade digital cameras, portable image acquisition devices, and intelligent surveillance cameras.
[0044] Optionally, different terminal devices 101 can ensure data collection at the same time point through a synchronization mechanism to ensure consistency of subsequent processing. The terminal device 101 may also include a built-in pre-processing unit for preliminary processing and compression of data to reduce the transmission load. In this embodiment, different terminal devices 101 have their own advantages. Combining the advantages of different image acquisition devices, all-round and high-precision monitoring of aircraft doors and their surroundings can be achieved. This can not only improve the accuracy and efficiency of door recognition, but also provide comprehensive and accurate data support for subsequent boarding bridge docking.
[0045] Optionally, the terminal device 101, such as a high-definition camera 1011, is used to capture two-dimensional image data of a high-resolution image of a target area on the fuselage of the aircraft. The infrared camera 1012 can capture infrared radiation emitted by an object, and is suitable for image acquisition at night or under low light conditions, ensuring the clarity and visibility of the image of the target area on the fuselage of the aircraft. The depth camera 1013 can generate depth data of the target area on the fuselage of the aircraft by measuring the distance between the object and the camera, thereby constructing a three-dimensional model of the aircraft door and its surrounding environment. The laser scanner 1014 calculates the distance by emitting a laser beam and measuring the time it takes for it to reflect back, and can be used to accurately measure the position, size and relative position relationship of the target area on the fuselage of the aircraft with other objects, providing accurate data for subsequent maintenance, inspection and modification.
[0046] The server 103 may be a server that provides various services, such as a background management server that provides support for the device operated by the user using the terminal device 101. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device.
[0047] Optionally, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.
[0048] Those skilled in the art will know that Figure 1 The number of image acquisition devices, networks and servers in the embodiment is only for illustration, and any number of image acquisition devices, networks and servers may be provided according to actual needs. The embodiments of the present disclosure are not limited to this.
[0049] Under the above system architecture, an embodiment of the present disclosure provides a method for identifying and locating an aircraft door, which can be executed by any electronic device with computing and processing capabilities.
[0050] In some embodiments, the aircraft door identification and positioning method provided in the embodiments of the present disclosure may be executed by the server in the above-mentioned system architecture; in other embodiments, the aircraft door identification and positioning method provided in the embodiments of the present disclosure may be implemented by the terminal device and the server in the above-mentioned system architecture through interaction.
[0051] Figure 2 A flow chart of a method for identifying and locating an aircraft door in an embodiment of the present disclosure is shown. Figure 2 As shown, the aircraft cabin door identification and positioning method provided in the embodiment of the present disclosure includes the following:
[0052] S202, collecting scanning data of the target area on the aircraft fuselage.
[0053] The fuselage of an aircraft refers to the main structure of the aircraft, including the cockpit, passenger cabin, cargo hold, etc. The specific part or range specified on the fuselage of the aircraft where image data needs to be collected. In this example, the target area includes the observation window and porthole. The observation window is a window set on the aircraft cabin door, usually used by flight attendants or ground personnel to observe the external situation of the aircraft. The porthole is a window set on the outside of the aircraft cabin, within a preset range from the observation window, usually used by passengers to view the external scenery or emergency escape.
[0054] Scanning data is data information captured by an imaging device (such as a camera or a laser radar) and stored and transmitted in digital form. When defining a target area, one or more windows belonging to the target area can be determined by setting the distance between the observation window and the porthole. Generally speaking, the target area includes the porthole closest to the aircraft door.
[0055] In this embodiment, the target area including the observation window and the porthole is determined according to the structure and design of the aircraft fuselage. An imaging device (such as a high-resolution camera) is used to shoot above or to the side of the target area to collect scanning data including the observation window and the porthole. Figure 3 A schematic diagram of a target area on an aircraft fuselage is provided for an embodiment of the present disclosure. Figure 3As shown, the target area is an area including an observation window 10 and a porthole 20. The observation window 10 is a window arranged on the aircraft cabin door, and the porthole 20 is a window arranged outside the aircraft cabin door and within a preset distance range from the observation window.
[0056] Specifically, the terminal device 101 (such as a visual camera, a depth camera or a laser scanner) can be used to scan the target area on the fuselage of the aircraft including the observation window and the porthole to obtain high-resolution two-dimensional image data and three-dimensional point cloud data of the area. For example, the terminal device 101 is installed at a fixed position near the boarding bridge. When the aircraft A is parked in place, the terminal device 101 starts working and takes a clear picture. Figure 4A A two-dimensional image schematic diagram of a target area on an aircraft fuselage is provided for an embodiment of the present disclosure, which includes an observation window on the aircraft cabin door and portholes adjacent thereto (i.e., windows within a preset distance range from the observation window). Figure 4B A schematic diagram of a three-dimensional point cloud image of a target area on an aircraft fuselage provided in an embodiment of the present disclosure.
[0057] S204, identifying and extracting observation window feature information of the observation window and porthole feature information of the porthole from the scan data of the target area.
[0058] In this embodiment, the characteristic information of the observation window and the porthole is automatically identified and extracted from the image data obtained in the first step by a computer vision algorithm. The characteristic information of the observation window and the porthole may specifically include visual features such as the shape, size, and texture of the observation window and the porthole, and the spatial position relationship based on the depth information may also be extracted.
[0059] Specifically, the image of the target area on the aircraft fuselage obtained in S202 is first preprocessed, for example, filtering techniques (such as Gaussian filtering, median filtering, etc.) are used to remove noise in the image and improve image quality, and contrast enhancement techniques (such as histogram equalization) can also be used to improve image details and enhance image contrast.
[0060] The observation window features may include the shape (such as rectangle, circle, etc.), size, position (position coordinates relative to the image or structure), edge definition, etc. of the window. The porthole features also include shape, size, position, and possibly the style and color of the window frame.
[0061] Continuing with the above example of aircraft A, in the acquired image, the server 103 can accurately distinguish the circular observation window and the flat rectangular porthole. For each window, its contour boundary, center coordinates, width, height and other specific parameters can be obtained. In addition, the relative distance and angle relationship between the two windows can also be calculated.
[0062] In this embodiment, by introducing the feature information of the observation window and the porthole, the reliance on a single feature (such as the door marking strip) is reduced. Even in cases where direct features such as the door marking strip are difficult to identify due to complex paint or poor lighting, the aircraft door can be assisted in identification by observing and comparing the relative position, shape and other features of the observation window and the porthole.
[0063] S206, determining the aircraft model according to the observation window feature information and the porthole feature information to obtain fuselage layout information.
[0064] In this embodiment, the fuselage layout information at least includes the layout information of the aircraft cabin door relative to the observation window and / or porthole on the aircraft fuselage. The specific layout information of the aircraft is determined by using the feature information extracted in S204 and combining it with the known aircraft model database. That is, the model of the aircraft currently photographed is confirmed by the observation window feature information and the porthole feature information, and then the aircraft model database is searched to obtain the exact position and layout information of each component (such as cabin door, observation window, porthole) on the corresponding model.
[0065] It should be noted that the fuselage layout information of the aircraft fuselage can be stored in the aircraft model database. Specifically, a unified data structure is formulated for each aircraft model to ensure the accuracy and consistency of the information. The dimensions of various doors of each aircraft model, such as the width and height of the boarding door, cargo door, service door, etc., are recorded in detail, and the specific position, size and shape characteristics of the aircraft observation window and porthole and their relative position relationship are recorded. Efficient storage and query of data can be achieved by using technologies such as relational databases or non-relational databases.
[0066] In some embodiments, the aircraft model is determined based on the observation window feature information and the porthole feature information to obtain the fuselage layout information, specifically including: matching the observation window feature information and the porthole feature information with a pre-established aircraft model database to determine the aircraft model and obtain the fuselage layout information from the database.
[0067] In this embodiment, following the example of S204, after the circular observation window and the flat rectangular porthole have been accurately distinguished, based on the extracted features of the observation window and the porthole, it is confirmed through the aircraft model database that aircraft A is an A-800 aircraft with a standard configuration, and the internally stored aircraft model database is further queried to obtain layout information of the aircraft door relative to other parts of the fuselage, specifically including the layout of the aircraft door relative to the observation window and the porthole, and may also include the specific layout between other parts of the aircraft fuselage.
[0068] It is understandable that with the rapid development of the aviation industry and the continuous introduction of new aircraft, the aircraft model database supports the addition and update of model information. When a new aircraft model is launched, the database can promptly record the relevant information of the new aircraft model, including the size of the cabin door, the location, size and shape characteristics of the observation window and porthole, etc. For the aircraft models that have been recorded, the database can update the relevant information according to their improvement or upgrade.
[0069] S208, determining the relative position information between the aircraft door and the boarding bridge according to the observation window feature information, the porthole feature information and the fuselage layout information.
[0070] In this embodiment, the relative posture information is used to control the docking of the boarding bridge and the aircraft door. Specifically, all feature points are converted from the local coordinate system of each sensor to a unified world coordinate system to ensure that data from different sources can correspond correctly. Then, based on the fuselage layout information, the aircraft door and its surrounding structures are reconstructed in three-dimensional space to form an accurate three-dimensional model. Based on the three-dimensional model, the position coordinates (x, y, z) of the aircraft door relative to the boarding bridge are calculated. Similarly, the attitude parameters (roll, pitch, yaw) of the aircraft door relative to the boarding bridge, that is, the rotation angles around the three axes, can be solved through the three-dimensional model.
[0071] Continuing with the above example, an A-800 aircraft is known through S206, and the coordinates of all feature points around the cabin door are known. By mapping these coordinates to the coordinate system where the boarding bridge is located, the three-dimensional coordinates (x, y, z) and rotation angles (roll, pitch, yaw) of the aircraft cabin door relative to the boarding bridge can be calculated. Then, the boarding bridge can adjust its position and direction to ensure that its front end is aligned with the aircraft cabin door to achieve safe and smooth docking.
[0072] In an optional embodiment, since different image acquisition devices have different types of data, the structural features of the target area can be determined more comprehensively through the synchronous alignment of multi-source data, thereby significantly improving the recognition accuracy of target areas on the aircraft fuselage, such as aircraft doors and their surroundings. Figure 5 A flowchart of an optional aircraft door identification and positioning method provided in an embodiment of the present disclosure. Figure 5 As shown, the scanning data of the target area on the fuselage of the aircraft provided by the embodiment of the present disclosure may include:
[0073] S502, collecting two-dimensional image data, depth image data and three-dimensional laser scanning data of the target area on the aircraft fuselage.
[0074] In this embodiment, since the aircraft cabin door observation window is usually a standard specification of round, rectangular with rounded corners or waist-shaped hole, the size is fixed. The aircraft porthole is usually in the shape of a waist-shaped hole, and the size is also fixed. Therefore, a high-resolution visual camera (such as an industrial-grade CCD or CMOS camera) can be used to shoot the aircraft fuselage part including the observation window and the porthole for subsequent identification. It is understandable that it is necessary to ensure that the camera is equipped with an appropriate lens to cover the entire target area, and adjust the exposure parameters to adapt to different lighting conditions. It should be noted that the above camera configuration process can be achieved in an automated manner, and the specific adjustment method is not limited in this disclosure.
[0075] In addition, the aircraft cabin door observation window is on the aircraft cabin door, and there will be a certain depth difference with the aircraft cabin door surface. Similarly, the aircraft porthole also has a depth difference with the fuselage surface. On this basis, a depth camera and / or laser scanner can be used to collect the depth difference data of the observation window and the porthole.
[0076] Specifically, the depth camera can not only capture color or grayscale images, but also provide the depth information corresponding to each pixel. The depth camera works synchronously with the visual camera to record the three-dimensional point cloud or depth map of the target area. The laser scanner can emit a laser beam and measure the time difference or phase change of the reflected light to accurately determine the distance of the surface of the object. The laser scanner performs a full-scale scan around the target area to generate a detailed three-dimensional coordinate point cloud.
[0077] S504, synchronously aligning the two-dimensional image data, the depth image data and the three-dimensional laser scanning data of the target area on the aircraft fuselage to obtain the scanning data of the target area on the aircraft fuselage.
[0078] In order to ensure that the data from different sensors can correctly correspond and effectively fuse, synchronization and alignment processing is required. First, the hardware synchronization interface or software timestamp mechanism can be used to ensure that all sensors collect data at the same time point, or at least have a high enough time resolution, so as to achieve time synchronization. Then, according to the coordinate data of different sensors themselves, all sensors are externally geometrically calibrated to determine their relative position relationship and achieve geometric calibration. Finally, the data from different sensors are converted into a unified world coordinate system. For visual cameras, the pixel coordinates in the image can be converted into world coordinates; for depth cameras and laser scanners, the measured distance values need to be mapped into the same spatial frame.
[0079] In an optional embodiment, for the alignment of two-dimensional image data and three-dimensional image data, a feature detection algorithm can be used to extract key points from the two-dimensional image data, and then find corresponding points in the depth image and the laser scanning data.
[0080] In an optional embodiment, the above synchronous alignment process can also construct a multimodal fusion model, such as a multimodal fusion model based on a Bayesian network or a neural network, which integrates the information provided by each image acquisition device to output a more accurate scanning data description of the target area.
[0081] In this embodiment, the consistency and accuracy of the data collected by different image acquisition devices are ensured through alignment processing, reducing the error accumulation problem that may be caused by using a certain type of sensor alone. In addition, in this embodiment, even under complex lighting conditions or facing different types of aircraft paint, due to the fusion of multiple types of data, it can still maintain high stability and reliability.
[0082] In an alternative embodiment, Figure 6 A flowchart of an optional aircraft door identification and positioning method provided in an embodiment of the present disclosure. Figure 6 As shown, identifying and extracting observation window feature information of the observation window and porthole feature information of the porthole from the scan data of the target area may include:
[0083] S602, extracting image edge contours from the two-dimensional image data of the target area to obtain two-dimensional image edge contours of the observation window and the porthole.
[0084] In this embodiment, image edge detection techniques, such as Sobel operator, Prewitt operator, Canny edge detection algorithm, etc., can be used to process the two-dimensional image data of the target area. These algorithms can detect the edges in the image, that is, the positions where the grayscale value changes dramatically, so as to obtain the edge contours of the two-dimensional images of the observation window and the porthole. It is also possible to detect the edge of a binary image, that is, in a binary image, the edge of each pixel with a value of 0 or 255 is detected. The contour of the image can also be extracted by morphological processing. This method is based on the structural morphology of the image and extracts the contour through morphological operations of expansion and corrosion. The above are only some examples of edge detection algorithms, and the specific edge detection algorithm is not limited in the present disclosure.
[0085] S604, extracting three-dimensional spatial measurement features from the depth image data and / or three-dimensional laser scanning data of the target area to obtain three-dimensional spatial measurement information of the observation window and the porthole.
[0086] In this embodiment, a depth camera and / or a 3D laser scanner are used to obtain depth image data or 3D point cloud data in the target area, that is, the 3D spatial measurement information of the observation window and the cabin door, the porthole and the fuselage in the target area is obtained by the depth camera and / or the 3D laser scanner. The 3D spatial measurement information includes depth difference information and scale information, and the scale information specifically refers to the size or size information of the scanned object (such as the observation window, porthole, cockpit window and fuselage / cabin door, etc.) in the 3D space, including but not limited to the length, width, height of the object and the relative size relationship between them.
[0087] It should be noted that both the depth image data and the three-dimensional point cloud data can reflect the depth difference data and the three-dimensional space coordinates. In the actual application process, you can choose to use the depth image data or the three-dimensional cloud point data. In order to further ensure accuracy, the two can also be used in combination. This disclosure does not specifically limit this.
[0088] Therefore, first, the depth image data and / or three-dimensional point cloud data are preprocessed, and the depth image or three-dimensional point cloud data is denoised and filtered. If multiple depth cameras or three-dimensional laser scanners are used, the acquired data needs to be aligned to ensure that the multiple depth cameras or three-dimensional laser scanners are in the same coordinate system.
[0089] Then, for each pixel in the depth image, its corresponding three-dimensional spatial coordinates are calculated according to the depth value. For three-dimensional point cloud data, the three-dimensional coordinates of each point are directly obtained. The depth difference between the observation window, porthole and surrounding structures (such as cabin doors and fuselage) is calculated to distinguish different objects and identify their boundaries. By measuring the distance between pixels in the depth image or the distance between points in the three-dimensional point cloud, the length, width, height and other size information of objects such as observation windows and portholes are calculated. At the same time, the relative size relationship between these objects can also be analyzed.
[0090] In the scenario where the depth image data and the 3D point cloud data are used together, the depth image data and the 3D point cloud data can also be fused through point cloud registration and data fusion algorithms to obtain more comprehensive 3D spatial metric information. Furthermore, the extracted 3D spatial metric information is post-processed, such as smoothing and deduplication, to improve the accuracy and reliability of the data.
[0091] S606, fusing the two-dimensional image edge contour and the three-dimensional space measurement information of the observation window to obtain observation window feature information of the observation window.
[0092] In this embodiment, a data fusion algorithm, such as weighted averaging, Kalman filtering, etc., is used to fuse the edge contour of the two-dimensional image and the three-dimensional spatial metric feature to obtain complete feature information of the observation window.
[0093] S608, fusing the two-dimensional image edge contour of the porthole with the three-dimensional spatial measurement information to obtain porthole feature information of the porthole.
[0094] Similarly, the same or similar data fusion algorithm is used to fuse the two-dimensional image edge contour and three-dimensional spatial metric features of the porthole to obtain complete feature information of the porthole.
[0095] In this embodiment, under different lighting conditions, viewing angles and occlusion conditions, the two-dimensional image edge contour and the three-dimensional space measurement feature may show different stabilities. By combining the two-dimensional image edge contour and the three-dimensional space measurement feature, the features of the observation window and the porthole can be more comprehensively described, thereby improving the accuracy of recognition.
[0096] Figure 7 A flowchart of an optional aircraft door identification and positioning method provided in an embodiment of the present disclosure. Figure 7 As shown, extracting the image edge contour from the two-dimensional image data of the target area to obtain the two-dimensional image edge contour of the observation window and the porthole may include:
[0097] S702, construct a first deep learning model.
[0098] In this embodiment, the first deep learning model is trained based on two-dimensional image data of a target area on the fuselage of the aircraft. It is understandable that the two-dimensional image data of the target area is two-dimensional image data including the aircraft observation window and porthole.
[0099] The specific training process of the first deep learning model is as follows:
[0100] First, we collect a large amount of 2D image data of aircraft observation windows and portholes. These data are diverse, including images from different angles, different lighting conditions, and different resolutions to ensure the generalization ability of the model.
[0101] Then, the collected image data were manually annotated to mark the edge contours of the observation windows and portholes.
[0102] Finally, the labeled image data is input into the deep learning model for training. During the training process, the model will learn how to identify and extract edge contour features in the image. By adjusting the model's hyperparameters and using data enhancement techniques, the performance of the model is optimized to obtain the first deep learning model, which enables the first deep learning model to more accurately extract the edge contours of the observation window and porthole.
[0103] S704, input the two-dimensional image data of the target area into the first deep learning model, and output the two-dimensional image edge contours of the observation window and the porthole.
[0104] In this embodiment, by introducing the first deep learning model, the edge contours in the image can be automatically extracted, thereby improving processing efficiency. Moreover, the model, which has been trained and optimized with a large amount of data, can more accurately identify and extract the edge contours of the observation window and porthole, thereby reducing the error caused by manual intervention. In addition, since a variety of data is used for training, the model has a stronger adaptability to new image data and can accurately extract edge contours under different conditions.
[0105] Figure 8 A flow chart of an optional aircraft door identification and positioning method provided in an embodiment of the present disclosure.
[0106] S802, construct a second deep learning model.
[0107] The second deep learning model is trained based on the depth image data and three-dimensional laser scanning data of the target area on the aircraft fuselage. The depth image data and three-dimensional laser scanning data of the target area are the depth image data and three-dimensional laser scanning data of the aircraft observation window and porthole.
[0108] Training the second deep learning model also requires the following data:
[0109] First, collect depth image data and 3D laser scanning data containing aircraft observation windows and portholes. These data should have high precision and accuracy to ensure that the model can learn the correct 3D spatial measurement features.
[0110] Then, the collected data is manually labeled to mark the 3D spatial position and size of the observation windows and portholes. The labeled data is input into the deep learning model for training. During the training process, the model learns how to identify and extract metric features in 3D space. By adjusting the model architecture, adding regularization terms, using more advanced deep learning algorithms, etc., the performance of the model is optimized to obtain a second deep learning model, which can more accurately extract 3D spatial metric information.
[0111] S804, input the target area depth image data and / or three-dimensional laser scanning data into the second deep learning model, and output the three-dimensional spatial measurement information of the observation window and the porthole.
[0112] It should be noted that since the second deep learning model is trained based on the depth image data and three-dimensional laser scanning data of the target area on the aircraft fuselage, the second deep learning model can extract three-dimensional spatial measurement features through depth image data, three-dimensional spatial measurement features through three-dimensional laser scanning data, or three-dimensional spatial measurement features through depth image data and three-dimensional laser scanning data.
[0113] In this embodiment, by introducing the first deep learning model, the edge contours in the image can be automatically extracted, thereby improving processing efficiency. Moreover, the model, which has been trained and optimized with a large amount of data, can more accurately identify and extract the edge contours of the observation window and porthole, thereby reducing the error caused by manual intervention. In addition, since a variety of data is used for training, the model has a stronger adaptability to new image data and can accurately extract edge contours under different conditions.
[0114] In the embodiments of the present disclosure, the above deep learning models can be used individually or simultaneously, and the present disclosure does not impose any restrictions on this.
[0115] In an optional embodiment, there are multiple possible implementations for determining the relative position information between the aircraft door and the boarding bridge based on the observation window feature information, the porthole feature information and the fuselage layout information. Fig. 9 A flowchart of an optional aircraft door identification and positioning method provided in an embodiment of the present disclosure. Fig. 9 As shown, determining the relative position information between the aircraft door and the boarding bridge according to the observation window feature information, the porthole feature information and the fuselage layout information may include:
[0116] S902, determining the spatial coordinates of the observation window and the porthole relative to the boarding bridge according to the observation window characteristic information and the porthole characteristic information of the aircraft.
[0117] In this embodiment, the observation window and porthole of the aircraft have their own characteristic information. After obtaining the observation window characteristic information and porthole characteristic information of the aircraft, the three-dimensional reconstruction technology can be used to set the boarding bridge as a reference system, analyze the positional relationship of the observation window characteristic information and porthole characteristic information in space, and thus calculate the spatial coordinates of the observation window and porthole relative to the boarding bridge.
[0118] S904, determining the corner point position of the aircraft door according to the observation window feature information, the porthole feature information and the fuselage layout information.
[0119] In this embodiment, the corner position of the aircraft door refers to the turning point or intersection of the door outline, such as the coordinate positions of the four vertices of the door frame, which are used to determine the precise position and shape of the door. Since the position, shape and other features of the observation window and porthole are related to the overall layout of the fuselage, by analyzing the spatial coordinates of the observation window and porthole relative to the boarding bridge, combined with the fuselage layout information, the coordinate positions of the corner points of the turning point or intersection of the aircraft door outline (such as the four vertices of the door frame) can be found, thereby determining the precise position and shape of the door.
[0120] In some embodiments, image distortion or perspective changes can be eliminated through geometric transformations (such as affine transformations or perspective transformations), and the positions of the corner points of the hatch can be corrected to ensure their accuracy.
[0121] S906, based on the corner position of the aircraft cabin door and the spatial coordinates of the observation window and the porthole relative to the boarding bridge, the relative position information of the aircraft cabin door and the boarding bridge is obtained by solving.
[0122] In this embodiment, the corner position of the aircraft door can determine the approximate outline position of the door in space. The spatial coordinates of the observation window and the porthole relative to the boarding bridge provide more information on the positional relationship between the structure around the door and the boarding bridge.
[0123] Through these coordinate data, mathematical algorithms (such as spatial geometric transformation and other related methods) can be used to calculate the position (such as translation) and attitude (such as rotation angle) of the aircraft door relative to the boarding bridge in three-dimensional space. The position reflects the distance and direction relationship between the centers or specific points of the two; the attitude reflects whether the door is facing, tilted, or tilted at a certain angle to the boarding bridge, so as to obtain accurate relative posture information of the two.
[0124] Specifically, the corner position of the aircraft door and the spatial coordinates of the observation window and porthole relative to the boarding bridge are converted into world coordinates (or airport coordinate system) to convert the corner position of the aircraft door and the coordinate data of the boarding bridge into the same coordinate system. Use a spatial coordinate conversion algorithm (such as rigid body transformation, affine transformation, etc.) to calculate the relative position and direction between the aircraft door and the boarding bridge. Specifically, it can be achieved by solving a set of linear equations or optimization problems to minimize the position and direction errors between the two. Finally, according to the results of the spatial coordinate conversion, the relative posture information of the aircraft door and the boarding bridge is solved. This includes information such as the position offset and rotation angle of the door relative to the boarding bridge.
[0125] In an alternative embodiment, Fig.10 Another aircraft door identification and positioning flow chart provided in the embodiment of the present disclosure (S202-S208 in Fig.10 Not shown, refer to Figure 1 ). Combined Fig.10 As shown, the method also includes:
[0126] S210, sending the relative position information of the aircraft cabin door and the boarding bridge to the boarding bridge control system, so that the boarding bridge control system controls the boarding bridge to dock with the aircraft cabin door according to the received relative position information of the aircraft cabin door and the boarding bridge.
[0127] In this embodiment, by accurately determining the relative position information of the aircraft cabin door and the boarding bridge, the docking accuracy between the two can be significantly improved.
[0128] In summary, the disclosed embodiment collects scan data of the target area including observation windows and portholes on the aircraft fuselage, and accurately extracts the observation window feature information and porthole feature information therefrom, and then determines the fuselage layout information of the aircraft fuselage based on these feature information, so that the aircraft cabin door and its related features can still be accurately identified under complex painting and changing lighting conditions. In this way, by comprehensively using the feature information of the observation window and porthole and the fuselage layout information, the effect of accurately identifying and locating the aircraft cabin door in a complex environment is achieved.
[0129] In an optional embodiment, in combination Figure 1 As shown, the target area may also include the area of the cockpit window 30 , which is a viewing window disposed at the front end of the cockpit of the aircraft.
[0130] Fig.11 A flowchart of another aircraft door identification and positioning method provided by an embodiment of the present disclosure. Figure 1 and Fig.11 The method further comprises:
[0131] S1102, extracting the driving window feature information of the driving window from the scanning data of the target area.
[0132] In this embodiment, first, the scanned data of the target area is preprocessed, including denoising, contrast enhancement, etc., to improve the accuracy of subsequent feature extraction. Image processing algorithms (such as edge detection, contour extraction, feature point matching, etc.) or deep learning models are applied to extract feature information such as the precise position, size, shape, etc. of the driver's window from the preprocessed image.
[0133] S1104, determining the aircraft model according to the observation window feature information, the porthole feature information and the cockpit window feature information to obtain fuselage layout information.
[0134] The feature information of the observation window, porthole and cockpit window is integrated to form a more comprehensive aircraft feature description. The integrated feature information is matched with the fuselage layout information in the aircraft model database, and the aircraft model and the corresponding fuselage layout information are determined by comparing the similarity between the features. The specific method is referred to S206, and this embodiment will not be described in detail here.
[0135] S1106, determining the relative position information between the aircraft door and the boarding bridge according to the observation window feature information, the porthole feature information, the cockpit feature information and the fuselage layout information.
[0136] After determining the fuselage layout information of the aircraft, the precise position of the aircraft door in the image is calculated by combining the position information of the observation window, porthole and cockpit window in the image, and the relative position relationship between each window and the cabin door provided in the fuselage layout information. Spatial coordinate conversion: The calculated cabin door position information is converted into spatial coordinates, and aligned and converted with the spatial coordinates of the boarding bridge to obtain the relative posture information of the aircraft cabin door and the boarding bridge. The specific implementation method refers to S208, and this embodiment will not be described in detail here.
[0137] In this embodiment, the cockpit window is a prominent feature on the aircraft, and its position, size, shape and other information are relatively stable under different lighting, angles and resolutions. Therefore, the introduction of the cockpit window can further increase the robustness of recognition. Combining the multi-feature information of the observation window, porthole and cockpit window can more effectively distinguish different models of aircraft, reduce the misrecognition rate caused by similar features, and thus improve the accuracy of recognition.
[0138] Based on the same inventive concept, the present disclosure also provides an aircraft cabin door identification and positioning system, as described in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.
[0139] Fig.12 A schematic diagram of an aircraft door identification and positioning system according to an embodiment of the present disclosure is shown. Fig.12 As shown, the system includes: an information acquisition module 1201 and a data processing module 1202, and the information acquisition module 1201 and the data processing module 1202 are communicatively connected.
[0140] Among them, the information acquisition module 1201 is used to collect scanning data of a target area on the aircraft fuselage, wherein the target area is an area including an observation window and a porthole, the observation window is a window set on the aircraft cabin door, and the porthole is a window set outside the aircraft cabin door and within a preset distance range from the observation window; the data processing module 1202 is used to identify and extract observation window feature information of the observation window and porthole feature information of the porthole from the scanning data of the target area; determine the aircraft model according to the observation window feature information and the porthole feature information to obtain the fuselage layout information, wherein the fuselage layout information at least includes the layout information of the aircraft cabin door on the aircraft fuselage relative to the observation window and / or the porthole; determine the relative posture information of the aircraft cabin door and the boarding bridge according to the observation window feature information, the porthole feature information and the fuselage layout information, wherein the relative posture information of the aircraft cabin door and the boarding bridge is used to control the docking of the boarding bridge with the aircraft cabin door.
[0141] In some embodiments, the information acquisition module 1201 is specifically used to: collect two-dimensional image data, depth image data and three-dimensional laser scanning data of the target area on the aircraft fuselage; and fuse the two-dimensional image data, depth image data and three-dimensional laser scanning data of the target area on the aircraft fuselage to obtain scanning data of the target area on the aircraft fuselage.
[0142] In some embodiments, the data processing module 1202 is specifically used to: extract image edge contours from the two-dimensional image data of the target area to obtain the two-dimensional image edge contours of the observation window and the porthole; extract three-dimensional spatial measurement features from the depth image data and / or three-dimensional laser scanning data of the target area to obtain three-dimensional spatial measurement information of the observation window and the porthole; fuse the two-dimensional image edge contour of the observation window and the three-dimensional spatial measurement information to obtain observation window feature information of the observation window; fuse the two-dimensional image edge contour of the porthole and the three-dimensional spatial measurement information to obtain porthole feature information of the porthole.
[0143] In some embodiments, the data processing module 1202 is also used to: construct a first deep learning model; the first deep learning model is trained based on two-dimensional image data containing a target area on the aircraft fuselage; extract image edge contours from the two-dimensional image data of the target area to obtain two-dimensional image edge contours of the observation window and the porthole, including: inputting the two-dimensional image data of the target area into the first deep learning model, and outputting the two-dimensional image edge contours of the observation window and the porthole; extracting three-dimensional spatial measurement features from the depth image data and / or three-dimensional laser scanning data of the target area to obtain three-dimensional spatial measurement information of the observation window and the porthole, including: constructing a second deep learning model; the second deep learning model is trained based on the depth image data and three-dimensional laser scanning data containing the target area on the aircraft fuselage; inputting the depth image data and / or three-dimensional laser scanning data of the target area into the second deep learning model, and outputting three-dimensional spatial measurement information of the observation window and the porthole.
[0144] In some embodiments, the data processing module 1202 is specifically used to: match the observation window feature information and the porthole feature information with a pre-established aircraft model database to determine the aircraft model and obtain the fuselage layout information from the database.
[0145] In some embodiments, the data processing module 1202 is specifically used to: determine the spatial coordinates of the observation window and the porthole relative to the boarding bridge based on the observation window feature information and the porthole feature information; determine the corner point position of the aircraft door based on the observation window feature information, the porthole feature information and the fuselage layout information; and calculate the relative position information of the aircraft door and the boarding bridge based on the corner point position of the aircraft door and the spatial coordinates of the observation window and the porthole relative to the boarding bridge.
[0146] In some embodiments, the data processing module 1202 is also used to: send the relative position information of the aircraft cabin door and the boarding bridge to the boarding bridge control system, so that the boarding bridge control system controls the docking of the boarding bridge and the aircraft cabin door according to the received relative position information of the aircraft cabin door and the boarding bridge.
[0147] In some embodiments, the target area also includes the area of the cockpit, which is a window located at the front end of the cockpit of the aircraft. The data processing module 1202 is also used to: extract the cockpit feature information of the cockpit from the scanning data of the target area; determine the aircraft model according to the observation window feature information, the porthole feature information and the cockpit feature information to obtain the fuselage layout information; determine the relative position information of the aircraft door and the boarding bridge according to the observation window feature information, the porthole feature information, the cockpit feature information and the fuselage layout information.
[0148] It should be noted that the various modules in the above system embodiment are the same as the corresponding examples and application scenarios implemented in the method embodiment, but are not limited to the contents disclosed in the above method embodiment. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.
[0149] Those skilled in the art will appreciate that various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."
[0150] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present disclosure, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned aircraft cabin door identification and positioning methods by executing the executable instructions. Since the principle of solving the problem in the electronic device embodiment is similar to that in the above-mentioned method embodiment, the implementation of the electronic device embodiment can refer to the implementation of the above-mentioned method embodiment, and the repeated parts will not be repeated.
[0151] Refer to the following Fig.13 1300 according to this embodiment of the present disclosure is described. Fig.13 The electronic device 1300 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0152] like Fig.13As shown, the electronic device 1300 is in the form of a general computing device. The components of the electronic device 1300 may include but are not limited to: at least one processing unit 1310, at least one storage unit 1320, and a bus 1330 connecting different system components (including the storage unit 1320 and the processing unit 1310).
[0153] The storage unit stores a program code, and the program code can be executed by the processing unit 1310, so that the processing unit 1310 executes the various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the processing unit 1310 can execute the above method embodiment as follows: collect scanning data of a target area on the fuselage of the aircraft, wherein the target area is an area including an observation window and a porthole, the observation window is a window set on the aircraft cabin door, and the porthole is a window set outside the aircraft cabin door and within a preset distance range from the observation window; identify and extract observation window feature information of the observation window and porthole feature information of the porthole from the scanning data of the target area; determine the aircraft model according to the observation window feature information and the porthole feature information to obtain the fuselage layout information, wherein the fuselage layout information at least includes the layout information of the aircraft cabin door, the observation window and the porthole on the aircraft fuselage; determine the relative posture information of the aircraft cabin door and the boarding bridge according to the observation window feature information, the porthole feature information and the fuselage layout information, wherein the relative posture information is used to control the docking of the boarding bridge with the aircraft cabin door.
[0154] The storage unit 1320 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 13201 and / or a cache storage unit 13202 , and may further include a read-only storage unit (ROM) 13203 .
[0155] The storage unit 1320 may also include a program / utility 13204 having a set (at least one) of program modules 13205, such program modules 13205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0156] Bus 1330 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0157] The electronic device 1300 may also communicate with one or more external devices 1340 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1300, and / or communicate with any device that enables the electronic device 1300 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1350. Furthermore, the electronic device 1300 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1360. As shown, the network adapter 1360 communicates with other modules of the electronic device 1300 via a bus 1330. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0158] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0159] Based on the same inventive concept, the embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned aircraft door identification and positioning methods is implemented. Since the principle of solving the problem in the embodiment of the computer-readable storage medium is similar to that in the above-mentioned method embodiment, the implementation of the embodiment of the computer-readable storage medium can refer to the implementation of the above-mentioned method embodiment, and the repeated parts will not be repeated.
[0160] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0161] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0162] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0163] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0164] Based on the same inventive concept, a computer program product is also provided in the embodiments of the present disclosure, including a computer program product, including: a computer program or an instruction, wherein when the computer program or the instruction is executed by a processor, the aircraft cabin door identification and positioning method in any one of the above method embodiments is implemented. Since the principle of solving the problem in the computer program product embodiment is similar to that in the above method embodiment, the implementation of the computer program product embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.
[0165] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0166] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0167] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0168] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.
Claims
1. A method for identifying and locating an aircraft door, characterized in that: include: Collecting scanning data of a target area on the fuselage of the aircraft, wherein the target area is an area including an observation window and a porthole, the observation window is a window arranged on the cabin door of the aircraft, and the porthole is a window arranged outside the cabin door of the aircraft and within a preset distance range from the observation window; Identifying and extracting observation window feature information of the observation window and porthole feature information of the porthole from the scan data of the target area; Determine the aircraft model according to the observation window characteristic information and the porthole characteristic information to obtain fuselage layout information, wherein the fuselage layout information at least includes layout information of aircraft doors on the aircraft fuselage relative to the observation window and / or the porthole; The relative position information between the aircraft door and the boarding bridge is determined based on the observation window feature information, the porthole feature information and the fuselage layout information, wherein the relative position information is used to control the docking of the boarding bridge and the aircraft door.
2. The aircraft door identification and positioning method according to claim 1, characterized in that: The collecting of scanning data of the target area on the aircraft fuselage includes: Collect 2D image data, depth image data and 3D laser scanning data of target areas on the aircraft fuselage; The two-dimensional image data, the depth image data and the three-dimensional laser scanning data of the target area on the aircraft fuselage are synchronously aligned to obtain the scanning data of the target area on the aircraft fuselage.
3. The aircraft door identification and positioning method according to claim 2, characterized in that: The identifying and extracting the observation window feature information of the observation window and the porthole feature information of the porthole from the scan data of the target area includes: Extracting image edge contours from the two-dimensional image data of the target area to obtain two-dimensional image edge contours of the observation window and the porthole; Extracting three-dimensional spatial measurement features from the depth image data and / or three-dimensional laser scanning data of the target area to obtain three-dimensional spatial measurement information of the observation window and the porthole; The two-dimensional image edge contour of the observation window and the three-dimensional space measurement information are fused to obtain the observation window feature information of the observation window; The two-dimensional image edge contour and three-dimensional spatial metric information of the porthole are fused to obtain the porthole feature information of the porthole.
4. The aircraft door identification and positioning method according to claim 3, characterized in that: The step of extracting the image edge contour from the two-dimensional image data of the target area to obtain the two-dimensional image edge contour of the observation window and the porthole includes: Constructing a first deep learning model; the first deep learning model is trained based on two-dimensional image data containing a target area on the aircraft fuselage; Inputting the two-dimensional image data of the target area into the first deep learning model, and outputting the two-dimensional image edge contours of the observation window and the porthole; The step of extracting three-dimensional spatial measurement features from the depth image data and / or three-dimensional laser scanning data of the target area to obtain three-dimensional spatial measurement information of the observation window and the porthole includes: Constructing a second deep learning model; the second deep learning model is trained based on the depth image data and the three-dimensional laser scanning data of the target area on the aircraft fuselage; The target area depth image data and / or three-dimensional laser scanning data are input into the second deep learning model, and the three-dimensional spatial measurement information of the observation window and the porthole is output.
5. The aircraft door identification and positioning method according to claim 1, characterized in that: Determining the aircraft model according to the observation window characteristic information and the porthole characteristic information to obtain fuselage layout information includes: The observation window characteristic information and the porthole characteristic information are matched with a pre-established aircraft model database to determine the aircraft model and obtain fuselage layout information from the database.
6. The aircraft door identification and positioning method according to claim 1, characterized in that: The determining, according to the observation window feature information, the porthole feature information and the fuselage layout information, the relative position information between the aircraft door and the boarding bridge comprises: Determine the spatial coordinates of the observation window and the porthole relative to the boarding bridge according to the observation window characteristic information and the porthole characteristic information; Determining the corner point position of the aircraft door according to the observation window feature information, the porthole feature information and the fuselage layout information; Based on the corner point position of the aircraft cabin door and the spatial coordinates of the observation window and the porthole relative to the boarding bridge, the relative position information of the aircraft cabin door and the boarding bridge is calculated.
7. The aircraft door identification and positioning method according to claim 1, characterized in that: The target area also includes the area of the driving window, which is a window arranged at the front end of the cockpit of the aircraft; The method further comprises: Extracting driving window feature information of the driving window from the scan data of the target area; Determine the aircraft model according to the observation window feature information, the porthole feature information and the cockpit window feature information to obtain fuselage layout information; The relative position information between the aircraft door and the boarding bridge is determined according to the observation window feature information, the porthole feature information, the cockpit window feature information and the fuselage layout information.
8. An aircraft door identification and positioning system, characterized in that: include: An information acquisition module and a data processing module, wherein the information acquisition module and the data processing module are communicatively connected; The information acquisition module is used to acquire scanning data of a target area on the aircraft fuselage, wherein the target area is an area including an observation window and a porthole, the observation window is a window arranged on the aircraft cabin door, and the porthole is a window arranged outside the aircraft cabin door and within a preset distance range from the observation window; The data processing module is used to identify and extract observation window feature information of the observation window and porthole feature information of the porthole from the scanning data of the target area; determine the aircraft model according to the observation window feature information and the porthole feature information to obtain fuselage layout information, wherein the fuselage layout information at least includes layout information of the aircraft door on the aircraft fuselage relative to the observation window and / or the porthole; determine the relative posture information of the aircraft door and the boarding bridge according to the observation window feature information, the porthole feature information and the fuselage layout information, wherein the relative posture information is used to control the docking of the boarding bridge with the aircraft door.
9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the aircraft cabin door identification and positioning method as described in any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the aircraft door identification and positioning method according to any one of claims 1 to 7 is implemented.