Abnormity detection method and device of airport visual parking guidance system, and related equipment

Abstract

The invention provides an anomaly detection method and device for an airport visual berth guidance system and related equipment, and relates to the technical field of airport visual berth guidance. The method comprises the following steps: acquiring first detection data acquired in a first time period; the first detection data comprises a reference object, and the position of the reference object relative to the ground is unchanged; acquiring second detection data acquired in a second time period; the second detection data comprises a reference object, and the second time period is a time period after the first time period; extracting first feature information of the reference object from the first detection data, and extracting second feature information of the reference object from the second detection data; and determining whether the airport visual parking guidance system is abnormal based on the first feature information and the second feature information. As the reference object is fixed relative to the ground position, more contents can be obtained when the feature information of the reference object is extracted, and the accuracy and reliability of anomaly detection of the airport visual berth guiding system can be improved.

Description

Background Art

[0002] The airport Visual Docking Guidance System (VDGS) realizes precise guidance and monitoring of aircraft. However, during the actual operation of the airport visual docking guidance system, due to various complex factors such as extreme weather, airport ground subsidence, and accidental physical impacts, it may undergo relative movement with the ground, resulting in the deviation of its parameters from the set expected values.

[0003] In related technologies, the methods for detecting abnormalities in the airport visual docking guidance system mainly rely on selecting a limited number of reference points at specific azimuth angles as reference points. However, due to the reliance on a small number of reference points, it is vulnerable to external factors such as climate and site obstacles, resulting in inaccurate detection. Moreover, changes in climate and site conditions may interfere with the detection process, causing false positives or false negatives, thereby reducing the accuracy and reliability of the airport visual docking guidance system and affecting the accurate calculation of aircraft pose information by the airport visual docking guidance system.

[0004] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure. Therefore, it may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0005] The present disclosure provides an abnormality detection method, device, and related equipment for an airport visual docking guidance system, which improve the accuracy and reliability of abnormality detection for the airport visual docking guidance system.

[0006] Other features and advantages of the present disclosure will become apparent through the following detailed description, or will be partially learned through the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, there is provided an abnormality detection method for an airport visual docking guidance system, including: obtaining first detection data collected in a first time period; the first detection data includes a reference object, and the relative position of the reference object with respect to the ground remains unchanged; obtaining second detection data collected in a second time period; the second detection data includes the reference object, and the second time period is a time period occurring after the first time period; extracting first feature information of the reference object from the first detection data, and extracting second feature information of the reference object from the second detection data; determining whether the airport visual docking guidance system is abnormal based on the first feature information and the second feature information.

[0008] In some embodiments, extracting the first feature information of the reference object from the first detection data and extracting the second feature information of the reference object from the second detection data includes: extracting and creating a first model of the reference object from the first detection data; extracting a second model of the reference object from the second detection data; comparing and registering the first model and the second model of the reference object to obtain a spatial transformation matrix; determining whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information includes: determining whether the airport visual berth guidance system is abnormal based on the spatial transformation matrix.

[0009] In some embodiments, when the reference object is a guiding line, extracting the first feature information of the reference object from the first detection data and extracting the second feature information of the reference object from the second detection data; extracting a first plane equation of the plane where the guiding line is located from the first detection data; extracting a second plane equation of the plane where the guiding line is located from the second detection data; determining whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information includes: determining whether the airport visual berth guidance system is abnormal based on the first plane equation and the second plane equation.

[0010] In some embodiments, the guiding line includes two intersecting sub-guiding lines. Extracting a first plane equation of the plane where the guiding line is located from the first detection data includes: respectively extracting the equations of the two intersecting sub-guiding lines from the first detection data; determining a first plane equation of the plane where the guiding line is located based on the equations of the two intersecting sub-guiding lines; extracting a second plane equation of the plane where the guiding line is located from the second detection data includes: respectively extracting the equations of the two intersecting sub-guiding lines from the second detection data; determining a second plane equation of the plane where the guiding line is located based on the equations of the two intersecting sub-guiding lines.

[0011] In some embodiments, determining whether the airport visual berth guidance system is abnormal based on the first plane equation and the second plane equation includes: determining a normal vector of the first plane based on the first plane equation; determining a normal vector of the second plane based on the second plane equation; determining an offset vector based on the normal vector of the first plane and the normal vector of the second plane; determining whether the airport visual berth guidance system is abnormal based on the offset vector.

[0012] In some embodiments, determining whether the airport visual berth guidance system is abnormal based on the offset vector includes: determining a rotation vector based on the normal vector of the first plane and the normal vector of the second plane, where the rotation vector includes an angle and a rotation axis.

[0013] In some embodiments, the first detection data includes first point cloud data and / or first image data. Obtaining the first detection data collected in the first time period includes: using a lidar to scan the apron containing the reference object in the first time period to obtain first point cloud data; and / or, using an image acquisition device to capture the apron containing the reference object in the first time period to obtain first image data. The second detection data includes second point cloud data and / or second image data. Obtaining the second detection data collected in the second time period includes: using a lidar to scan the apron containing the reference object in the second time period to obtain second point cloud data; and / or, using an image acquisition device to capture the apron containing the reference object in the second time period to obtain second image data.

[0014] In some embodiments, when the first detection data is first point cloud data, extracting the first feature information of the reference object from the first detection data includes: screening out first point cloud data clusters with intensities greater than a preset threshold from the first point cloud data; using a filtering technique to extract the first feature information of the reference object from the first point cloud data clusters. When the second detection data is second point cloud data, extracting the second feature information of the reference object from the second detection data includes: screening out second point cloud data clusters with intensities greater than a preset threshold from the second point cloud data; using a filtering technique to extract the second feature information of the reference object from the second point cloud data clusters. Wherein, the filtering technique includes at least one of the following: morphological filtering, deep learning-based filtering, and clustering-based filtering.

[0015] According to another aspect of the present disclosure, there is also provided an abnormality detection device for an airport visual berth guidance system. The device includes: a first acquisition module for acquiring first detection data collected in a first time period; the first detection data contains a reference object, and the relative position of the reference object to the ground remains unchanged; a second acquisition module for acquiring second detection data collected in a second time period; the second detection data contains the reference object, and the second time period is a time period after the first time period; an extraction module for extracting first feature information of the reference object from the first detection data and extracting second feature information of the reference object from the second detection data; a determination module for determining whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information.

[0016] According to another aspect of the present disclosure, there is also provided an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the abnormality detection method of the airport visual berth guidance system according to any one of the above through executing the executable instructions.

[0017] According to another aspect of the present disclosure, there is also provided a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the abnormal detection method of the airport vision berth guidance system described in any one of the above is implemented.

[0018] According to another aspect of the present disclosure, there is also provided a computer program product, including: a computer program or instruction. When the computer program or instruction is executed by a processor, the abnormal detection method of the airport vision berth guidance system described in any one of the above is implemented.

[0019] An abnormal detection method, device and related equipment of an airport vision berth guidance system provided in an embodiment of the present disclosure. The method includes: obtaining first detection data collected in a first time period; the first detection data includes a reference object, and the relative position of the reference object to the ground remains unchanged; obtaining second detection data collected in a second time period; the second detection data includes the reference object, and the second time period is a time period occurring after the first time period; extracting first feature information of the reference object from the first detection data, and extracting second feature information of the reference object from the second detection data; determining whether the airport vision berth guidance system is abnormal based on the first feature information and the second feature information. Since the relative position of the reference object to the ground is fixed, more content can be obtained when extracting its feature information, which can improve the accuracy and reliability of the abnormal detection of the airport vision berth guidance system.

[0020] Furthermore, since the position of the reference object is fixed, its feature information is relatively stable and less affected by external factors such as climate and site. Therefore, the present disclosure has strong robustness to changes in external factors such as climate and site, which helps to reduce false positives and false negatives, and further improves the stability and reliability of the abnormal detection of the airport vision berth guidance system.

[0021] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 A schematic diagram of the system architecture showing an abnormal detection method of an airport vision berth guidance system in an embodiment of the present disclosure;

[0024] Figure 2The flowchart of an abnormal detection method for an airport visual berth guidance system in an embodiment of the present disclosure is shown;

[0025] Figure 3 The schematic diagram of the relative position between an airport visual berth guidance system and a guiding line in an embodiment of the present disclosure is shown;

[0026] Figure 4 The flowchart of another abnormal detection method for an airport visual berth guidance system in an embodiment of the present disclosure is shown;

[0027] Figure 5 The flowchart of a method for determining the abnormality of an airport visual berth guidance system in an embodiment of the present disclosure is shown;

[0028] Figure 6 The schematic diagram of an apron plan in an embodiment of the present disclosure is shown;

[0029] Figure 7 The flowchart of a method for extracting guiding line features in an embodiment of the present disclosure is shown;

[0030] Figure 8 The schematic diagram of an abnormal detection device for an airport visual berth guidance system in an embodiment of the present disclosure is shown;

[0031] Figure 9 The structural block diagram of an electronic device in an embodiment of the present disclosure is shown. Detailed implementation manners

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0033] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0034] As shown above, the airport visual berth guidance system realizes the precise guidance and monitoring of aircraft. LiDAR is usually used to detect objects and measure distances, and the visual berth guidance system relies on LiDAR to provide the position, speed and attitude information of the aircraft. In the actual application process, when LiDAR works on site, it will be affected by various objective factors, such as extreme weather, airport ground subsidence, accidental physical impact, etc. These factors will cause relative movement between LiDAR and the ground, resulting in its deviation from the three-dimensional coordinate position set at the factory. The accurate calculation of the pose information of the aircraft when it enters the port depends on the accurate position of LiDAR. If the installation accuracy of the radar is not high, for example, there is a deviation angle or height difference, the emission direction of the laser beam will be inaccurate, resulting in errors in the measured distance, position and other data, thus affecting the accurate acquisition of the aircraft pose information and ultimately possibly affecting the accuracy of the aircraft docking. Therefore, the LiDAR in the VDGS system needs to be detected irregularly. If the deviation reaches a certain degree, the system parameters need to be repaired or recalibrated to ensure the normal operation of the system and accurately calculate the pose information of the aircraft when it enters the port.

[0035] In the related technology, the specific method for detecting the deviation of LiDAR is as follows: First, a small number of reference points (such as two points p1 and p2) at a specific azimuth angle are selected as reference points, and the coordinates of the reference points are recorded and saved. The relative distance between the LiDAR distance and the reference points can be obtained through the coordinates of this pair of reference points. During detection, the real-time coordinates of the same-name reference points (the current p1' and p2') are re-acquired according to the azimuth angle of the previous reference points. Then, the real-time coordinates of the same-name reference points are compared with the coordinates of the reference points and the corresponding relative distances. If the difference exceeds the set threshold, the radar deviates, that is, the airport visual berth guidance system is abnormal.

[0036] However, in the related technology, only relying on the coordinates of a small number of reference points to judge the abnormality of the airport visual berth guidance system, due to the complex on-site situation, the climate at different times during calibration and detection (such as rain and snow causing water accumulation and snow accumulation on the apron, and the appearance of mirrors affecting laser scanning), temporary on-site obstacles, etc., will all interfere with the accuracy of radar deviation detection. If a small number of reference points are in the affected area, the data accuracy will be severely damaged, resulting in low accuracy and reliability of the detection results. Moreover, the ordinary method can only judge whether the radar has a large deviation and cannot detect detailed information such as the deviation azimuth.

[0037] It has been found through research that fixed objects on the apron (such as high mast lights, floor-mounted aircraft air conditioners, distribution kiosks, fixed bridges, bridgeheads, guiding lines, etc.), as key indication signs in important facilities such as airports, have obvious features and high reliability. Taking fixed objects as important identifiers can provide a more stable and reliable basis for judging system anomalies in complex situations, making up for the deficiency of judging only based on the coordinates of a small number of reference points. For example, for the guiding lines on the apron, in order to ensure their long-term effectiveness and efficiency, the guiding lines are generally sprayed with high-quality special coatings or other special paint materials that meet industry standards. These materials not only have excellent weather resistance and wear resistance, but also show unique signal characteristics in radar scans. Since the coatings and materials used for the guiding lines are different from those of the ordinary apron surface, there are significant differences in the absorption rate and reflectivity of the laser pulses emitted by the radar. This difference enables the lidar to clearly distinguish the guiding lines from the surrounding ordinary surface during scanning. In lidar data, the guiding lines usually show stronger signal intensity due to their high reflectivity to laser pulses. In contrast, the signal intensity of the non-sprayed area around is weaker. This difference in signal intensity provides a reliable basis for subsequent data processing and the calculation of offset vectors.

[0038] In view of this, the embodiments of the present disclosure achieve the anomaly detection of the airport visual berth guidance system by extracting the characteristic information of the reference object and relying on a large amount of point cloud information. Since more reference information is introduced, the accuracy and reliability of the airport visual berth guidance system can be improved.

[0039] It should be noted that in the above example, the visual berth guidance system realizes the precise guidance and monitoring of the aircraft through lidar technology. In the actual application process, the visual berth guidance system can also use other sensors such as image acquisition devices to achieve the same function. Specifically, it can achieve the precise guidance and monitoring of the aircraft through a multispectral camera or a color camera. In some embodiments, it can also adopt any combination of lidar, multispectral camera and color camera. The present disclosure does not limit this. For the convenience of description, in the following embodiments, lidar, multispectral camera, color camera, etc. are collectively referred to as detection devices. In addition, the present disclosure does not limit the number of detection devices.

[0040] The following will describe in detail the specific implementation manners of the embodiments of the present disclosure with reference to the accompanying drawings.

[0041] Figure 1 The exemplary application system architecture diagram in which the anomaly detection method of the airport visual berth guidance system in the embodiments of the present disclosure can be applied is shown. As Figure 1 shown, the system architecture may include a terminal device 101, a network 102, and a server 103.

[0042] Network 102 is a medium for providing a communication link between the terminal device 101 and the server 103, which can be a wired network or a wireless network.

[0043] Optionally, the above-mentioned wireless network or wired network uses standard communication technologies and / or protocols. The network is usually the Internet, but can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent the data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec), etc. can be used to encrypt all or some of the links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0044] The terminal device 101 can be various electronic devices, including but not limited to smartphones, tablets, laptop computers, desktop computers, smart speakers, smart watches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0045] Optionally, the clients of the application programs installed in different terminal devices 101 are the same, or clients of the same type of application programs based on different operating systems. Depending on the different terminal platforms, the specific form of the client of the application program can also be different. For example, the client of the application program can be a mobile client, a PC client, etc.

[0046] The server 103 can be a server that provides various services, such as a back-end management server that supports the devices for the operations performed by the user using the terminal device 101. The back-end management server can analyze and process data such as requests received, and feedback the processing results to the terminal device.

[0047] Optionally, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0048] Those skilled in the art can understand that Figure 1 the numbers of the terminal devices, networks, and servers in are merely illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers. The embodiments of the present disclosure do not limit this.

[0049] Under the above system architecture, an anomaly detection method for an airport visual berth guidance system is provided in an embodiment of the present disclosure. This method can be executed by any electronic device with computing and processing capabilities.

[0050] In some embodiments, the anomaly detection method for the airport visual berth guidance system provided in the embodiments of the present disclosure can be executed by the terminal device of the above system architecture; in other embodiments, the anomaly detection method for the airport visual berth guidance system provided in the embodiments of the present disclosure can be executed by the server in the above system architecture; in other embodiments, the anomaly detection method for the airport visual berth guidance system provided in the embodiments of the present disclosure can be implemented by the terminal device and the server in the above system architecture through interaction.

[0051] Figure 2 shows a flowchart of an anomaly detection method for an airport visual berth guidance system in an embodiment of the present disclosure. As Figure 2 shown, the anomaly detection method for the airport visual berth guidance system provided in the embodiments of the present disclosure includes the following steps:

[0052] S202, obtaining first detection data collected in a first time period; the first detection data includes a reference object, and the relative position of the reference object with respect to the ground remains unchanged.

[0053] In this embodiment, the first time period refers to a specific time interval for defining the start and end times of data collection. Here, the first time period specifically refers to the time when the airport visual berth guidance system determines the reference data. The first detection data refers to the data about the reference object on the airport apron collected by the airport visual berth guidance system at the first time, and specifically may include the reference object type, reference object coordinates, relative distance of the reference object, etc. The reference object refers to a fixed object on the airport apron, and specifically may include high mast lights, floor-mounted aircraft air conditioners, distribution kiosks, fixed bridges, bridgeheads, guiding lines, etc. on the airport apron.

[0054] In some embodiments, in the scenario of offset detection of an airport visual berthing guidance system, guiding lines for guiding aircraft to dock are provided in the apron area. These guiding lines provide visual references for pilots to ensure that aircraft can dock accurately and safely at designated positions. Since the guiding lines are sprayed with high-quality special coatings, they are different from ordinary apron surface materials. This results in significant differences in the absorption rate and reflectivity of the laser pulses emitted by the radar for the guiding lines. Therefore, the differences between the guiding lines and ordinary areas can be utilized to collect richer and more reliable detection data through the airport visual berthing guidance system. When the reference object is a guiding line, the first detection data at least includes the guiding line for guiding the aircraft, and this guiding line is used to calibrate the standard parameters of the airport visual berthing guidance system, so that the airport visual berthing guidance system guides the aircraft to the target stop position based on the standard parameters. The target stop position is the final position that the aircraft needs to reach.

[0055] Figure 3 This is a schematic diagram of the relative positions of an airport visual berthing guidance system and guiding lines provided by an embodiment of the present disclosure. It should be noted that Figure 3 The example in Figure 3 shows a type of guiding line in the airport scenario, also called a parking guidance line, including a stop line and a guiding line. Among them, the stop line is an identification line for indicating the stop position of the aircraft, and the guiding line plays a role in guiding the aircraft. As shown in

[0056] Specifically, the airport visual berthing guidance system includes tools or devices for detecting relevant information. For example, the airport visual berthing guidance system can include lidar, sensors, image acquisition devices, etc., and its function is to collect data on the surrounding environment or specific targets. For example, lidar can be used to collect point cloud data of fixed objects (such as guiding lines, high mast lights, distribution kiosks, etc.) on the airport apron. Point cloud data usually includes the spatial coordinates (x, y, z) of each measurement point, as well as possible reflection intensity information. The spatial coordinates and reflection intensity information of each measurement point constitute a discrete point set in three-dimensional space. The image acquisition device can collect image data of the environment and identify guiding lines through visual information. Therefore, the first detection data can include first point cloud data and / or first image data.

[0057] In some embodiments, obtaining the first detection data collected by the airport visual berthing guidance system in the first time period includes: scanning the apron containing the reference object with lidar in the first time period to obtain first point cloud data; and / or, photographing the apron containing the reference object with an image acquisition device in the first time period to obtain first image data.

[0058] S204, obtaining second detection data collected in the second time period, where the second detection data includes the reference object.

[0059] In this embodiment, due to the existence of some objective reasons, such as extreme weather, airport surface subsidence, or accidental physical impact, etc., it may cause the position of the airport visual berth guidance system to shift or the parameters to be abnormal. At this time, the detection data collected by the airport visual berth guidance system can reflect these abnormalities. The second detection data at least includes reference objects on the airport apron, and the second time period is a time period that occurs after the first time period.

[0060] It should be noted that the reference objects collected in the second detection data are the same as those in the first detection data. It can be understood that the number of reference objects selected can be one or more, and this embodiment does not limit this.

[0061] For example, in combination with Figure 3 the example, when the reference object is a guiding line, the red dotted line indicates the position after the radar shift and the position of the guiding line corresponding to the radar after the shift. It should be noted that the position of the guiding line indicated by the red dotted line is the theoretically calculated guiding line position by the radar based on its current position and angle, rather than the actual position of the guiding line in the real world. Since the radar may have shifted, these two positions may not be the same. What the red dotted line marks is exactly this theoretically guiding line position based on the current state of the radar.

[0062] In some embodiments, the second detection data may include second point cloud data and / or second image data. Obtaining the second detection data collected in the second time period includes: scanning the apron containing the reference object with a lidar during the second time period to obtain second point cloud data; and / or, photographing the apron containing the reference object with an image acquisition device during the second time period to obtain second image data.

[0063] S206, extract the first feature information of the reference object from the first detection data, and extract the second feature information of the reference object from the second detection data.

[0064] In this embodiment, the first feature information refers to the feature information of the reference object extracted from the first detection data, such as position, length, direction, curvature, etc. The second feature information refers to the specific features of the reference object extracted from the first detection data, such as position, length, direction, curvature, etc.

[0065] For example, for a straight guiding line, the least squares method is used to fit the straight line equation Ax + By + C = 0, that is, the sum of the squares of the distances from numerous points in the point cloud to the fitted straight line is minimized, so as to determine the values of A, B, and C, and accurately describe the straight line. For circular or cylindrical objects (such as high mast lights), the diameter and center position of the circular or cylindrical object can be calculated for definition. Specifically, by analyzing the distribution law of the point cloud in space, the two points with the farthest distance can be found to determine the diameter, and the center position can be obtained from the centroid of the point cloud distribution or through a specific algorithm (such as fitting the center of the circle based on the circumferential points), so as to clarify the geometric characteristics of the object.

[0066] Since the first detection data and the second detection data obtained in S202 and S204 contain the reference object, a large amount of information characterizing the features of the reference object can be extracted from the detection data, so as to obtain sufficient reference information. Compared with only referring to a few reference points in the related art, the obtained feature information is more reliable.

[0067] S208, determine whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information.

[0068] In this embodiment, the relevant information of the reference object included in the first feature information represents the standard parameters of the airport visual berth guidance system. The second feature information is the relevant information of the reference object obtained by the airport visual berth guidance system collecting data again during the offset detection. Due to possible radar offset, the second feature information of the reference object may be different from the first feature information. By comparing the first feature information with the second feature information and analyzing the differences between the two, it can be determined whether the airport visual berth guidance system is abnormal. It should be noted that the differences between the two may include the position offset, shape change, direction adjustment, etc. of the reference object. If the difference is within a reasonable range, it is considered that the airport visual berth guidance system is normal; if the difference is too large, it is considered that the airport visual berth guidance system may have shifted or malfunctioned. The relationship between the two is based on the first feature information as the standard, and the second feature information is used to test whether the system is abnormal due to factors such as radar offset.

[0069] For example, when the reference object is a guiding line, the offset can be obtained by calculating the relative position difference in space between the first characteristic information and the second characteristic information of the guiding line. If the offset exceeds a preset threshold (critical threshold), the airport visual berth guidance system is considered abnormal. It should be noted that the setting of the threshold needs to be based on actual situations, including factors such as the accuracy requirements of the airport visual berth guidance system and the influence degree of the external environment. When the offset is not serious (less than the preset offset threshold), the relative reference system between the radar and the ground can be corrected by recalibrating the coordinate system in the software to ensure the accuracy of the azimuth recognition after the aircraft enters the port. When the offset is relatively serious (greater than or equal to the preset offset threshold), the device needs to be repaired or adjusted according to the offset vector to restore its normal function.

[0070] In some embodiments, multiple lidars and / or multiple image acquisition devices can also be used for joint calibration detection. By using multi-sensor fusion technology, the accuracy and reliability of calibration and detection can be improved to meet the scenarios of large airports or multi-radar systems.

[0071] In some embodiments, during the operation of the airport visual berth guidance system, the characteristic information of the reference object is regularly detected and analyzed to timely discover changes in the operating state of the device, providing a basis for the maintenance of the airport visual berth guidance system, so as to timely handle potential problems and ensure the long-term stable operation of the airport visual berth guidance system. For example, automatically adjusting the radar parameters to maintain the high accuracy of the radar.

[0072] In this embodiment, by extracting the characteristic information of the reference object, more characteristic information can be obtained, and it is less affected by external factors. Compared with the method relying on limited reference points, the detection accuracy and reliability can be improved. In addition, since the extracted is the characteristic information of the reference object, this method has strong robustness to changes in external factors such as climate and site. This helps to reduce misjudgment and missed judgment, and improve the stability and reliability of the system.

[0073] In some embodiments, Figure 4 The method flow chart showing another abnormal detection method of the airport visual berth guidance system in the embodiments of the present disclosure is shown. Combining Figure 4 As shown, the abnormal detection method of the airport visual berth guidance system provided by the embodiments of the present disclosure may include the following steps:

[0074] S402, extract and create a first model of the reference object from the first detection data.

[0075] In this embodiment, from the acquired first detection data, reference objects such as berth lines and signs are identified, and then a model is constructed for these reference objects for subsequent comparative analysis. For example, a multi-object reference object template library can be constructed. The multi-object reference object template library integrates the precise geometric models of all fixed objects selected as references (such as the berth lines mentioned above) and their position information during calibration to form a template library. This library can provide a standard reference for subsequent detection. When the actual detection data does not match the models in the library, it may indicate an anomaly, thereby realizing the detection of anomalies in the airport visual berth guidance system.

[0076] S404, extract the second model of the reference object from the second detection data.

[0077] In this embodiment, the second detection data is the data obtained when detecting the apron environment. Extracting the second model of the reference object from it is to construct a model of the same reference object as in the first detection data, such as a model of reference objects such as berth lines. By comparing the models of the same reference object under two different detection data, work such as detecting and analyzing the apron environment and the airport visual berth guidance system can be carried out.

[0078] In some embodiments, this embodiment can use a deep learning model (such as PointNet++) or a feature matching-based algorithm (such as FPFH descriptor) to process the scan data of the current airport apron environment and identify the positions and shapes of fixed objects. Among them, PointNet++ is a deep learning model that identifies objects by hierarchically aggregating local features when processing point cloud data and is suitable for large-scale point clouds. The FPFH descriptor is a feature matching-based algorithm that calculates the differences between points and neighboring points to generate feature vectors for object matching. Both are used to identify the positions and shapes of fixed objects on the apron. It should be noted that the above models are only examples, and in actual application processes, other models can also be used, and this embodiment does not limit this.

[0079] S406, compare and register the first model and the second model of the reference object to obtain a spatial transformation matrix.

[0080] In this embodiment, model comparison and registration is an operation of matching and aligning the first model and the second model of the reference object. The spatial transformation matrix can describe the spatial transformation relationship between two point clouds. Through this matrix, one point cloud can be transformed into the coordinate system of another point cloud.

[0081] Specifically, the spatial transformation matrix T can be composed of a rotation matrix R and a translation vector t. It satisfies the formula P' = RP + t, where P and P' represent the point cloud coordinate sets in the calibration and current states respectively.

[0082] In some embodiments, the Iterative Closest Point (ICP) algorithm can be used for registration. The ICP algorithm continuously finds the pairs of points with the closest distances between two sets of point clouds, and then calculates a spatial transformation matrix based on these point pairs to minimize the distance error between the two sets of point clouds.

[0083] S408. Determine whether the airport visual berth guidance system is abnormal based on the spatial transformation matrix.

[0084] In this embodiment, the obtained spatial transformation matrix T is decomposed to determine the specific changes of the radar or image acquisition device, etc., relative to the original position. And a certain threshold is set, and when this threshold is exceeded, the alarm mechanism is triggered or the calibration program is automatically started.

[0085] Specifically, the rotation angles theta_x, theta_y, theta_z around the three coordinate axes can be extracted from the rotation matrix R, and the displacement amounts Delta x, Delta y, Delta z can be read from the translation vector t.

[0086] For example, if the rotation is represented by Euler angles, the angles in each direction can be calculated by the following formula:

[0087] theta_x = arctan2(R_{32}, R_{33})

[0088] theta_y = arctan2(R_{31}, sqrt{R_{32}^2 + R_{33}^2})

[0089] theta_z = arctan2(R_{21}, R_{11}).

[0090] Set thresholds such as theta > 1° and / or Delta x > 10 cm). When the rotation angle and / or the translation vector exceed the threshold, the alarm mechanism is triggered or the calibration program is automatically started.

[0091] In some embodiments, the alarm mechanism can be a visual or auditory signal to remind the maintenance personnel to check the system; the automatic calibration program adjusts the coordinate system of the radar or image acquisition device, etc., according to the calculated offset parameters to restore it to the correct relative position.

[0092] In this embodiment, compared with the related art that only realizes the qualitative judgment of the abnormality of the airport visual berth guidance system, this embodiment can quantitatively output the offset angle, direction, and displacement, with more accurate output, and can provide more detailed and accurate information, which is helpful for more in-depth analysis and decision-making. In addition, compared with the method that requires manual calibration of the reference point in the related art, this embodiment can dynamically update the reference object template library, with stronger dynamic adaptability.

[0093] In some embodiments, Figure 5 FIG. shows a flowchart of a method for determining an abnormality of an airport visual berth guidance system provided by an embodiment of the present disclosure. Combining Figure 5 as shown, determining whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information may include:

[0094] S502, extracting a first plane equation of the plane where the guiding line is located from the first detection data.

[0095] Figure 6 FIG. shows a schematic diagram of an apron plane provided by an embodiment of the present disclosure. Combining Figure 6 as shown, in this embodiment, the guiding line includes two intersecting sub-guiding lines. Extracting a first plane equation of the plane where the guiding line is located from the first detection data may include: respectively extracting equations of the two intersecting sub-guiding lines from the first detection data; determining a first plane equation of the plane where the guiding line is located based on the equations of the two intersecting sub-guiding lines.

[0096] In this embodiment, since the point cloud is a set of discrete point data in a three-dimensional space, by fitting the processed point cloud, a straight line that best fits these points can be obtained. For obtaining the straight line equation of the guiding line, mathematical methods such as the least squares method can be used. According to the coordinate relationship of the point cloud data points, the slope and intercept of the straight line are determined, so as to obtain the straight line equation. The Random Sample Consensus (RANSAC) algorithm can also be used to fit the point cloud data. By performing multiple random samplings and model validations, abnormal points are removed, and the robustness of the fitting is improved.

[0097] Combining Figure 6 with the schematic diagram of the apron plane shown, the spatial straight line L1 where the stop line is located is orthogonal to the spatial straight line L2 where the guiding line is located, which means that the product of the slopes of the two straight lines is -1. Given the guiding line equation, the slope of the stop line perpendicular (orthogonal) to it can be obtained through its slope, and then the intercept of the stop line is determined according to the point cloud data, and further the stop line equation is obtained. The guiding line and stop line equations obtained in this way can be used to determine a first plane equation of the plane where the guiding line is located.

[0098] To find the plane equation where two straight lines lie, specifically, according to the general form of the plane equation, i.e., Ax + By + Cz + D = 0, find the coordinates of three points on the plane, substitute them into the general form and then solve the equations (there are three equations and four unknowns, but A, B, C, and D are not unique. They can still represent the same equation when multiplied by the same multiple. Therefore, only the proportional relationship needs to be solved, or a particular solution of the system of equations can be obtained). In this embodiment, three points can be taken on the two obtained straight line guiding lines, but they should not all be on one of the straight lines, and a set of solutions for A, B, C, and D can be obtained. It can be understood that in this embodiment, the three parameters A, B, and C determine the normal vector of the plane. The normal vector is a vector perpendicular to the plane, and its coordinates are (A, B, C). D is related to the distance from the plane to the origin. When substituting the point (x0, y0, z0) into the equation and making it hold, this point is on the plane. From the perspective of distance, the distance from the plane Ax + By + Cz + D = 0 to the origin is

[0099] In this embodiment, the first plane equation of the plane where the guiding line lies is:

[0100] A1x + B1y + C1z + D1 = 0

[0101] where A1, B1, and C1 are the components of the normal vector of the first plane, and D1 is a constant term related to the distance from the first plane to the origin.

[0102] S504, Extract the second plane equation of the plane where the guiding line lies from the second detection data.

[0103] In this embodiment, extracting the second plane equation of the plane where the guiding line lies from the second detection data is similar to S502. The guiding line in the second detection data also includes two intersecting sub-guiding lines. Extracting the second plane equation of the plane where the guiding line lies from the second detection data may include: extracting the equations of the two intersecting sub-guiding lines from the second detection data respectively; determining the second plane equation of the plane where the guiding line lies based on the equations of the two intersecting sub-guiding lines.

[0104] The second plane equation of the plane where the guiding line lies is determined as:

[0105] A2x + B2y + C2z + D2 = 0

[0106] where A2, B2, and C2 are the components of the normal vector of the second plane, and D2 is a constant term related to the distance from the second plane to the origin.

[0107] S506, Determine whether the airport visual berth guiding system is abnormal based on the first plane equation and the second plane equation.

[0108] In this embodiment, to determine whether the airport visual berth guidance system is abnormal, the offset vector of one plane relative to another plane can be calculated. By accurately calculating the offset vector of the normal lines between two planes, the present disclosure can clearly determine the offset azimuth and offset degree of the radar, thus providing a solid scientific basis for subsequent calibration or repair work.

[0109] The offset vector includes a rotation vector and a translation vector. In three-dimensional space, the rotation vector represents the direction and angle of rotation of an object or coordinate system around a certain point, and this vector is usually determined by the rotation axis and the rotation angle. The rotation vector of one plane relative to another plane describes how the first plane rotates around a certain axis to reach the same direction as the second plane. This rotation axis is not necessarily the intersection line of the two planes, but is determined according to the relative positions and directions of the two planes. In short, the rotation vector of one plane relative to another plane reveals how the first plane needs to rotate to align with the second plane. The translation vector represents the parallel movement of an object in space, and it determines the distance and direction of the movement. For example, a point moves a certain distance in a certain direction from one position. It should be noted that in many cases, the change in the spatial position of an object may include both operations such as rotation that change the direction and operations such as translation that change the position. Therefore, the offset vector of one plane relative to another plane in the present disclosure can be either a rotation vector, a translation vector, or a combination of both a rotation vector and a translation vector, and the embodiments of the present disclosure do not limit this.

[0110] In some embodiments, the offset vector between two planes can be determined by the normal lines between the two planes. That is, to determine whether the airport visual berth guidance system is abnormal based on the first plane equation and the second plane equation, it can include: determining the normal vector of the first plane based on the first plane equation; determining the normal vector of the second plane based on the second plane equation; determining the offset vector based on the normal vector of the first plane and the normal vector of the second plane; and determining whether the airport visual berth guidance system is abnormal based on the offset vector. Further, determining whether the airport visual berth guidance system is abnormal based on the offset vector includes: determining the rotation vector based on the normal vector of the first plane and the normal vector of the second plane, and the rotation vector includes the included angle and the rotation axis.

[0111] In this embodiment, following the above example, the normal vector of the first plane The normal vector of the second plane The specific method for calculating the included angle between the normal vector of the first plane and the normal vector of the second plane is as follows:

[0112] The dot product formula can be used to calculate the included angle θ between the two normal vectors, and the dot product formula is Therefore, there is:

[0113]

[0114] where is the dot product of two normal vectors, and and are the magnitudes of the two normal vectors respectively, that is and A1, B1, C1 are the components of the normal vector of the first plane, and A2, B2, C2 are the components of the normal vector of the second plane.

[0115] The cross product of the normal vectors of two planes will give the direction of the rotation axis. This direction is perpendicular to the plane where the two normal lines are located, that is: calculate the rotation axis between the normal vector of the first plane and the normal vector of the second plane Specifically as follows:

[0116]

[0117] where A1, B1, C1 are the components of the normal vector of the first plane, and A2, B2, C2 are the components of the normal vector of the second plane.

[0118] In some embodiments, in order to make the rotation axis a unit vector, it is also necessary to normalize the rotation axes of the first plane and the second plane

[0119]

[0120] where is the length of the vector and can be obtained by calculating A1, B1, C1 are the components of the normal vector of the first plane, and A2, B2, C2 are the components of the normal vector of the second plane.

[0121] In some embodiments, the obtained rotation vector can be represented by a quaternion or an axis - angle.

[0122] Among them, the form represented by the axis - angle is indicating a rotation angle θ around the unit vector To further convert it into the form represented by a quaternion, the following formula can be used:

[0123] Here q represents the quaternion, its scalar part is

[0124]

[0125] and the vector part is and the vector part is

[0126] In this embodiment, not only can the airport visual berth guidance system be accurately and reliably judged qualitatively whether it has shifted, but also the offset vector can be further quantitatively calculated, providing rapid and intuitive support information for maintenance and correction work. It overcomes the problem that traditional detection methods can only qualitatively judge whether the radar has undergone a severe shift, but cannot provide detailed information such as the specific offset amount and direction. By precisely calculating the offset vector of the normal between two planes in this embodiment, the offset azimuth and offset degree of the airport visual berth guidance system can be clearly determined, thereby providing a scientific basis for subsequent correction or repair work.

[0127] In this embodiment, due to the particularity of the guiding line material, it has an obvious signal intensity performance in the detection data, such as in the lidar data, and the obvious signal intensity of the guiding line will avoid the influencing factors of the interference area in the surrounding environment, so as to calculate an accurate and reliable offset vector. That is to say, the present disclosure uses the difference between the guiding line and the ordinary area, and with the help of rich point cloud data, excludes interference and obtains an accurate offset result.

[0128] In some embodiments, Figure 7 The flowchart of a method for extracting the characteristics of the guiding line in the embodiment of the present disclosure is shown. Combining Figure 7 As shown, when the first detection data is the first point cloud data, the first characteristic information of the reference object is extracted from the first detection data, including:

[0129] S702, screening out the first point cloud data clusters with intensities greater than a preset threshold from the first point cloud data.

[0130] In this embodiment, the point cloud data is a massive set of points expressing the target space distribution and target surface characteristics in the same spatial reference system. These points contain information such as positions (such as x, y, z coordinates), which can be used to describe the characteristics of objects such as shapes. In the point cloud data, the intensity is usually related to devices such as lidar. For example, when a lidar emits a laser beam to the surface of an object and reflects back, characteristics such as the energy size of the reflected light can be represented by the intensity, and different intensity values reflect different materials, roughness, etc. of the object surface. The preset threshold is a standard value set artificially and is determined according to specific application requirements and scenarios. For example, when detecting a building, a certain intensity value may be set as the threshold according to experience. The point cloud data cluster is a set of some points in the point cloud data, and these points have a certain similarity, such as being close in space or having similar characteristics in terms of intensity, etc.

[0131] Since the signal intensity of the guiding line in the lidar data is stronger than that of the surrounding non-sprayed area. When screening from the first point cloud data, by finding out the point cloud data clusters with intensities greater than the preset threshold, the point cloud clusters representing the guiding line can be separately screened out, which is convenient for subsequent operations such as extracting, analyzing, or using information related to the guiding line.

[0132] When the first detection data is the first image data, filtering can also be performed based on color information. Specifically, the color information captured by a multispectral or color camera is used to distinguish the guiding line from the ordinary ground surface, and the guiding line area is extracted through color threshold filtering. Alternatively, texture feature extraction can be performed. Specifically, image processing techniques are used to extract the texture features of the guiding line area, such as edge detection, texture segmentation, etc., to further enhance the recognition accuracy of the guiding line.

[0133] S704, extracting the first feature information of the reference object from the first point cloud data cluster using filtering techniques.

[0134] In this embodiment, the filtering techniques include at least one of the following: morphological filtering, deep learning-based filtering, and clustering-based filtering. Among them, morphological filtering is an operation in digital image processing or point cloud processing. In point clouds, morphological filtering operates on point cloud data based on predefined structural elements (such as spheres, cubes, etc.). For example, removing noise points, smoothing data, and separating ground points. The reference object is a linear feature extracted from the first point cloud data cluster for specific purposes (such as positioning, recognition, etc.). The first feature information is various attributes related to this guiding line, such as the direction vector, starting point coordinates, length, etc. of the guiding line, and this information can be used to describe the characteristics of the point cloud cluster in this direction.

[0135] In some embodiments, in addition to using morphological filtering to extract the first feature information of the reference object from the first point cloud data cluster, filtering methods such as deep learning and clustering can also be used to extract the first feature information of the reference object from the first point cloud data cluster. Among them, deep learning-based filtering refers to using deep learning models such as convolutional neural networks to preprocess point cloud data to automatically remove noise and smooth data. This method can better adapt to complex environmental changes.

[0136] Clustering-based filtering classifies point cloud data using density-based spatial clustering of applications with noise (DBSCAN), K-means clustering algorithm, etc., removing outliers and noise points to improve data quality. Specifically, DBSCAN divides points into a cluster if the density of data points in a region exceeds a certain threshold according to the density of data points. Data points in low-density regions are regarded as noise points. K-means divides data into K clusters, minimizing the sum of the distances from data points in each cluster to the cluster center, and achieving the optimal clustering effect by continuously iterating and updating the cluster center. In this embodiment, DBSCAN does not require specifying the number of clusters in advance and has good processing ability for noise points; K-means requires specifying the number of clusters K and has a relatively fast calculation speed.

[0137] It should be noted that when performing filtering in this embodiment, one or more of morphological filtering, deep learning-based filtering, and clustering-based filtering can be selected. When multiple filtering techniques are selected, they can be weighted. Weighting means assigning a weight value to each filtering technique. For example, the weight of morphological filtering is 0.3, the weight of deep learning-based filtering is 0.5, and the weight of clustering-based filtering is 0.2. In this way, during the filtering process, different filtering techniques have different impacts on the results according to the weights, and combined to achieve a better filtering effect.

[0138] Similarly, when the second detection data is the second point cloud data, the second feature information of the reference object is extracted from the second detection data, including:

[0139] S706, screening out the second point cloud data clusters in the second point cloud data whose intensity is greater than a preset threshold.

[0140] In this embodiment, the processing method of the second point cloud data is similar to that of the first point cloud data, and will not be elaborated here.

[0141] S708, using filtering techniques to extract the second feature information of the reference object from the second point cloud data clusters.

[0142] Similarly, the filtering techniques in this embodiment include at least one of the following: morphological filtering, deep learning-based filtering, and clustering-based filtering, which will not be elaborated here.

[0143] In this embodiment, through an intelligent algorithm, the point cloud data most effective for calculation is automatically screened out, without the need for manual setting of parameters such as azimuth, which simplifies the operation process, makes the operation more simple and fast, reduces manual intervention, and improves work efficiency. Traditional detection methods rely on manual selection of reference points, which is not only time-consuming but also prone to introducing human errors, affecting the detection results. In contrast, this embodiment can avoid these problems through automatic screening, improving the accuracy and efficiency of detection. In addition, the detection methods of related technologies are affected by factors such as rain, snow, and specular effects, while this embodiment uses filtering technology to overcome the problems existing in related technologies. In short, the embodiments of the present disclosure optimize the data processing flow through automation and intelligent technologies, reduce the influence of human factors on the results, and thus improve the accuracy and reliability of the detection of the airport visual berth guidance system.

[0144] In some embodiments, different objects have different reflectivities under different spectra. The guiding line and the ordinary ground surface have different materials, and there are differences in color in multi-spectral or color images. The multi-spectral or color camera can also be used to distinguish the guiding line and the ordinary ground surface based on the captured color information. When the first detection data is image data, the first feature information of the reference object can be extracted from the first detection data, which may include: filtering by setting a color threshold to screen out the area that conforms to the color characteristics of the guiding line. For example, for a white guiding line and a gray asphalt road, the color contrast is obvious in a color image, and the guiding line area can be determined according to the color value range. After the guiding line area is determined, edge detection can find the edge contour of the guiding line, and texture segmentation can separate the guiding line texture from other messy textures. These texture features can make the guiding line features more unique, reduce misjudgment, and improve the recognition accuracy. When the second detection data is image data, the same processing is performed.

[0145] Based on the same inventive concept, an abnormal detection device for an airport visual berth guidance system is also provided in the embodiments of the present disclosure, as described in the following embodiments. Since the principle of solving problems in this device embodiment is similar to that of the above method embodiment, the implementation of this device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be described again.

[0146] Figure 8 The schematic diagram of an abnormal detection device for an airport visual berth guidance system in the embodiments of the present disclosure is shown, as Figure 8 shown, the device includes: an acquisition module 81, a second acquisition module 82, an extraction module 83, and a determination module 84;

[0147] A first acquisition module 81 is configured to acquire first detection data collected in a first time period; the first detection data includes a reference object, and the relative position of the reference object with respect to the ground remains unchanged; a second acquisition module 82 is configured to acquire second detection data collected in a second time period; the second detection data includes the reference object, and the second time period is a time period occurring after the first time period; an extraction module 83 is configured to extract first feature information of the reference object from the first detection data and extract second feature information of the reference object from the second detection data; a determination module 84 is configured to determine whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information.

[0148] In some embodiments, the extraction module 83 is specifically configured to: extract and create a first model of the reference object from the first detection data; extract a second model of the reference object from the second detection data; compare and register the first model and the second model of the reference object to obtain a spatial transformation matrix; the determination module 84 is specifically configured to: determine whether the airport visual berth guidance system is abnormal based on the spatial transformation matrix.

[0149] In some embodiments, when the reference object is a guiding line, the extraction module 83 is specifically configured to: extract a first plane equation of the plane where the guiding line is located from the first detection data; extract a second plane equation of the plane where the guiding line is located from the second detection data; the determination module 84 is specifically configured to: determine whether the airport visual berth guidance system is abnormal based on the first plane equation and the second plane equation.

[0150] In some embodiments, the guiding line includes two intersecting sub-guiding lines, and the extraction module 83 is specifically configured to: extract the equations of the two intersecting sub-guiding lines from the first detection data respectively; determine a first plane equation of the plane where the guiding line is located based on the equations of the two intersecting sub-guiding lines; extract the equations of the two intersecting sub-guiding lines from the second detection data respectively; determine a second plane equation of the plane where the guiding line is located based on the equations of the two intersecting sub-guiding lines.

[0151] In some embodiments, the determination module 84 is specifically configured to: determine a normal vector of the first plane based on the first plane equation; determine a normal vector of the second plane based on the second plane equation; determine an offset vector based on the normal vector of the first plane and the normal vector of the second plane; determine whether the airport visual berth guidance system is abnormal based on the offset vector.

[0152] In some embodiments, the determining module 84 is specifically configured to: determine a rotation vector based on the normal vector of the first plane and the normal vector of the second plane, where the rotation vector includes an included angle and a rotation axis.

[0153] In some embodiments, the first detection data includes first point cloud data and / or first image data. The first acquisition module 81 is specifically configured to: scan the apron including the reference object with a lidar during a first time period to obtain first point cloud data; and / or, capture the apron including the reference object with an image acquisition device during the first time period to obtain first image data. The second detection data includes second point cloud data and / or second image data. The second acquisition module 82 is specifically configured to: scan the apron including the reference object with a lidar during a second time period to obtain second point cloud data; and / or, capture the apron including the reference object with an image acquisition device during the second time period to obtain second image data.

[0154] In some embodiments, when the first detection data is first point cloud data, the extraction module 83 is specifically configured to: screen out a first point cloud data cluster with an intensity greater than a preset threshold from the first point cloud data; use a filtering technique to extract first feature information of the reference object from the first point cloud data cluster. When the second detection data is second point cloud data, screen out a second point cloud data cluster with an intensity greater than a preset threshold from the second point cloud data; use a filtering technique to extract second feature information of the reference object from the second point cloud data cluster. Wherein, the filtering technique includes at least one of the following: morphological filtering, deep learning-based filtering, and clustering-based filtering.

[0155] It should be noted here that the examples and application scenarios implemented by each module in the above device embodiments are the same as the corresponding steps in the method embodiments, but are not limited to the content disclosed in the above method embodiments. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0156] Those skilled in the art can understand that various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module" or "system" here.

[0157] Based on the same inventive concept, embodiments of the present disclosure also provide an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the abnormal detection method of the airport vision berth guidance system according to any one of the above by executing the executable instructions. Since the principle of solving problems in this embodiment of the electronic device is similar to that of the above method embodiment, the implementation of this embodiment of the electronic device can refer to the implementation of the above method embodiment, and the repeated parts will not be elaborated.

[0158] Next, refer to Figure 9 to describe the electronic device 900 according to this embodiment of the present disclosure. Figure 9 The shown electronic device 900 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

[0159] As Figure 9 shown, the electronic device 900 is presented in the form of a general-purpose computing device. The components of the electronic device 900 may include but are not limited to: at least one of the above processing units 910, at least one of the above storage units 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0160] Among them, the storage unit stores program codes, and the program codes can be executed by the processing unit 910, so that the processing unit 910 executes the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the processing unit 910 may execute the following steps of the above method embodiment: obtaining first detection data collected in a first time period; the first detection data includes a reference object, and the relative position of the reference object to the ground remains unchanged; obtaining second detection data collected in a second time period; the second detection data includes a reference object, and the second time period is a time period occurring after the first time period; extracting first feature information of the reference object from the first detection data, and extracting second feature information of the reference object from the second detection data; determining whether the airport vision berth guidance system is abnormal based on the first feature information and the second feature information.

[0161] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 9201 and / or a cache storage unit 9202, and may further include a read-only storage unit (ROM) 9203.

[0162] The storage unit 920 may further include a program / utility 9204 having a set (at least one) of program modules 9205. Such program modules 9205 include but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0163] The bus 930 can represent one or more of several types of bus architectures, including a memory unit bus or a memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of the various bus architectures.

[0164] The electronic device 900 can also communicate with one or more external devices 940 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 950. Moreover, the electronic device 900 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 960. As shown in the figure, the network adapter 960 communicates with other modules of the electronic device 900 through the bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0165] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or can be implemented by a combination of software and necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0166] Based on the same inventive concept, an embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the abnormal detection method of the airport vision berth guidance system in any one of the above. Since the principle of solving the problem of the embodiment of the computer-readable storage medium is similar to that of the above method embodiment, the implementation of the embodiment of the computer-readable storage medium can refer to the implementation of the above method embodiment, and the repeated parts will not be described again.

[0167] More specific examples of the computer-readable storage medium in this disclosure may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0168] In this disclosure, the computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0169] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above.

[0170] In specific implementation, the program code for performing the operations of this disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., connected through the Internet using an Internet service provider).

[0171] Based on the same inventive concept, an embodiment of this disclosure also provides a computer program product, including: a computer program or instruction, which when executed by a processor, implements the abnormal detection method of the airport vision berth guidance system in any one of the above method embodiments. Since the principle of solving problems in this embodiment of the computer program product is similar to that of the above method embodiments, the implementation of this embodiment of the computer program product may refer to the implementation of the above method embodiments, and the repeated parts will not be described again.

[0172] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0173] In addition, although the steps of the methods in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be performed in that specific order, or that all of the shown steps must be performed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0174] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the methods according to the embodiments of the present disclosure.

[0175] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

Claims

1. An abnormal detection method for an airport visual berth guidance system, characterized in that, Including: Obtaining first detection data collected in a first time period; The first detection data includes a reference object, and the relative position of the reference object with respect to the ground remains unchanged; Obtaining second detection data collected in a second time period; The second detection data includes the reference object, and the second time period is a time period occurring after the first time period; Extracting first feature information of the reference object from the first detection data, and extracting second feature information of the reference object from the second detection data; Determining whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information.

2. The anomaly detection method for the airport visual berth guidance system according to claim 1, wherein The extracting first feature information of the reference object from the first detection data and extracting second feature information of the reference object from the second detection data includes: Extracting and creating a first model of the reference object from the first detection data; Extracting a second model of the reference object from the second detection data; Comparing and registering the first model and the second model of the reference object to obtain a spatial transformation matrix; The determining whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information includes: Determining whether the airport visual berth guidance system is abnormal based on the spatial transformation matrix.

3. The anomaly detection method of the airport visual berth guidance system according to claim 1, characterized in that, When the reference object is a guiding line, the extracting first feature information of the reference object from the first detection data and extracting second feature information of the reference object from the second detection data; Extracting a first plane equation of the plane where the guiding line is located from the first detection data; Extracting a second plane equation of the plane where the guiding line is located from the second detection data; The determining whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information includes: Determining whether the airport visual berth guidance system is abnormal based on the first plane equation and the second plane equation.

4. The abnormal detection method of the airport visual berth guidance system according to claim 3, characterized in that The guiding line includes two intersecting sub-guiding lines, and the extracting a first plane equation of the plane where the guiding line is located from the first detection data includes: Respectively extracting the equations of two intersecting sub-guiding lines from the first detection data; Determining a first plane equation of the plane where the guiding line is located based on the equations of the two intersecting sub-guiding lines; The extracting a second plane equation of the plane where the guiding line is located from the second detection data includes: Respectively extracting the equations of two intersecting sub-guiding lines from the second detection data; Determining a second plane equation of the plane where the guiding line is located based on the equations of the two intersecting sub-guiding lines.

5. The abnormal detection method of the airport visual berth guidance system according to claim 3 or 4, characterized in that, The determining whether the airport visual berth guidance system is abnormal based on the first plane equation and the second plane equation includes: Determining a normal vector of the first plane based on the first plane equation; Determining a normal vector of the second plane based on the second plane equation; Determining an offset vector based on the normal vector of the first plane and the normal vector of the second plane; Determining whether the airport visual berth guidance system is abnormal based on the offset vector.

6. The anomaly detection method of the airport visual berth guidance system according to claim 5, characterized in that, The determining whether the airport visual berth guidance system is abnormal based on the offset vector includes: Determine a rotation vector based on the normal vector of the first plane and the normal vector of the second plane, where the rotation vector includes an included angle and a rotation axis.

7. The anomaly detection method for the airport visual berth guidance system according to claim 1, characterized in that The first detection data includes first point cloud data and / or first image data. Obtaining the first detection data collected in the first time period includes: Scanning the apron containing the reference object with a lidar during the first time period to obtain first point cloud data; and / or, Taking pictures of the apron containing the reference object with an image acquisition device during the first time period to obtain first image data; The second detection data includes second point cloud data and / or second image data. Obtaining the second detection data collected in the second time period includes: Scanning the apron containing the reference object with a lidar during the second time period to obtain second point cloud data; and / or, Taking pictures of the apron containing the reference object with an image acquisition device during the second time period to obtain second image data.

8. The anomaly detection method for the airport visual berth guidance system according to claim 7, characterized in that, When the first detection data is first point cloud data, extracting the first feature information of the reference object from the first detection data includes: Filtering out first point cloud data clusters with intensities greater than a preset threshold from the first point cloud data; Using a filtering technique to extract the first feature information of the reference object from the first point cloud data clusters; When the second detection data is second point cloud data, extracting the second feature information of the reference object from the second detection data includes: Filtering out second point cloud data clusters with intensities greater than a preset threshold from the second point cloud data; Using a filtering technique to extract the second feature information of the reference object from the second point cloud data clusters; Wherein, the filtering technique includes at least one of the following: morphological filtering, deep learning-based filtering, and clustering-based filtering.

9. An abnormal detection device for an airport visual berth guidance system, characterized in that, The device includes: A first acquisition module, configured to acquire first detection data collected in a first time period; the first detection data contains a reference object, and the relative position of the reference object with respect to the ground remains unchanged; A second acquisition module, configured to acquire second detection data collected in a second time period; the second detection data contains the reference object, and the second time period is a time period occurring after the first time period; An extraction module, configured to extract first feature information of the reference object from the first detection data and extract second feature information of the reference object from the second detection data; A determination module, configured to determine whether the airport visual berth guidance system is abnormal based on the first feature information and the second feature information.

10. An electronic device, characterized in that, Includes: A processor; And A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the abnormal detection method of the airport visual berth guidance system according to any one of claims 1 to 8 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the abnormal detection method of the airport visual berth guidance system according to any one of claims 1 to 8.

12. A computer program product, comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, it implements the abnormal detection method of the airport visual berth guidance system according to any one of claims 1 to 8.

Images