Airplane taxi distance detection method, device, equipment and medium

Abstract

The application provides an aircraft docking distance detection method, device, equipment and medium, relates to the technical field of aviation ground equipment, and can accurately detect the distance between the docking part of an aircraft and the docking part of a service vehicle. The method comprises the following steps: acquiring a docking image through an image acquisition device, wherein the docking image at least comprises the docking part of the aircraft and the docking part of the service vehicle; acquiring a first image corresponding to the docking part of the aircraft and a second image corresponding to the docking part of the service vehicle based on the docking part of the aircraft and the docking part of the service vehicle; inputting the first image into an aircraft docking part recognition model for recognition to obtain a first recognition result of the docking part of the aircraft; inputting the second image into a service vehicle docking part recognition model for recognition to obtain a second recognition result of the docking part of the service vehicle; and performing geometric ranging based on the first recognition result and the second recognition result to obtain the aircraft docking distance.

Description

Technical Field

[0001] This application relates to the field of aviation ground equipment technology, and in particular to a method, device, equipment and medium for detecting aircraft approach distance. Background Technology

[0002] During flight transit, the work vehicle needs to approach the parked aircraft to perform tasks such as cargo loading and unloading, passenger boarding and disembarking, refueling, supplying fresh water, discharging wastewater, cleaning up garbage, and towing. Real-time monitoring of the distance between the work vehicle and the aircraft is crucial as the work vehicle approaches the aircraft.

[0003] In related technologies, images of the operation scene of the work vehicle are captured by a single camera, and then the images are processed using monocular vision ranging technology (such as a general target detection model or a monocular depth estimation large model) to determine the distance between the work vehicle and the aircraft fuselage.

[0004] However, the above technical solution can only detect the distance between the aircraft and the work vehicle as a whole, and the detection accuracy is low. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, equipment and medium for detecting aircraft docking distance, which can accurately detect the distance between the aircraft docking part and the work vehicle docking part.

[0006] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application provides a method for detecting aircraft docking distance. The method includes: acquiring a docking image using an image acquisition device, the docking image including at least the docking part of the aircraft and the docking part of the work vehicle; acquiring a first image corresponding to the aircraft docking part and a second image corresponding to the work vehicle docking part based on the aircraft docking part and the work vehicle docking part, respectively; inputting the first image into an aircraft docking part recognition model for recognition, obtaining a first recognition result for the aircraft docking part; the aircraft docking part recognition model being a recognition model constructed based on the geometric characteristics of the aircraft docking part; inputting the second image into a work vehicle docking part recognition model for recognition, obtaining a second recognition result for the work vehicle docking part; and performing geometric distance measurement based on the first and second recognition results to obtain the aircraft docking distance. The aircraft docking distance is used to describe the distance between the aircraft docking part and the work vehicle docking part.

[0007] The technical solution provided in this application involves acquiring docking images of the aircraft docking point and the work vehicle docking point, then extracting images from these images to obtain a first image corresponding to the aircraft docking point and a second image corresponding to the work vehicle docking point. The first image includes only the aircraft portion, and the second image includes only the non-aircraft portion. Based on the first and second images, the aircraft docking point and the work vehicle docking point can be independently identified. Specifically, according to the geometric characteristics of the aircraft docking point, the first image is input into the aircraft docking point recognition model for recognition, resulting in a first recognition result for the aircraft docking point; the second image is input into the work vehicle docking point recognition model for recognition, resulting in a second recognition result for the work vehicle docking point. Recognizing the first and second images separately reduces the recognition search space and improves the recognition efficiency and accuracy of the aircraft docking point and the work vehicle docking point. Finally, geometric distance measurement is performed based on the first and second recognition results to obtain the aircraft approach distance between the aircraft docking point and the work vehicle docking point. Since the first and second recognition results accurately characterize the positions of the aircraft docking point and the work vehicle docking point. Therefore, the aircraft docking distance calculated based on the first and second identification results can accurately detect the distance between the aircraft docking point and the work vehicle docking point.

[0008] In some embodiments, the aircraft includes multiple aircraft docking parts with different geometric characteristics. The aircraft docking part recognition model includes multiple sub-recognition models corresponding to the geometric characteristics of each aircraft docking part. Inputting a first image into the aircraft docking part recognition model for recognition yields a first recognition result for the aircraft docking part. Specifically, this can be implemented by: determining the target geometric characteristics of the aircraft docking parts in the first image; inputting the first image into the sub-recognition model corresponding to the target geometric characteristics for recognition, thereby obtaining the first recognition result for the aircraft docking parts in the first image. By constructing corresponding sub-recognition models based on the geometric characteristics of the aircraft docking parts, compared to using the same recognition model to recognize aircraft docking parts with different geometric characteristics, each sub-recognition model can more specifically identify the aircraft docking parts, resulting in more accurate recognition results. By recognizing the geometric characteristics of the aircraft docking parts, and then matching the corresponding sub-recognition model based on those characteristics, inputting the first image into that sub-recognition model for recognition, and obtaining the first recognition result, accurate recognition of the first image can be achieved even when the geometric characteristics of the aircraft docking parts are significantly heterogeneous, thereby improving the detection accuracy of the aircraft approach distance.

[0009] In some embodiments, the geometric characteristics of the aircraft docking parts include at least rectangular, elliptical, curved surface, and anchor point types. Multiple sub-recognition models include a rectangular sub-recognition model corresponding to the rectangular type, an elliptical sub-recognition model corresponding to the elliptical type, a curved surface sub-recognition model corresponding to the curved surface type, and an anchor point sub-recognition model corresponding to the anchor point type. By classifying the aircraft docking parts into rectangular, elliptical, curved surface, and anchor point types according to their geometric characteristics, and assigning a unique recognition model to each type of aircraft docking part, this application improves the recognition accuracy of the first image corresponding to the aircraft docking parts compared to the prior art's use of the same recognition model for all aircraft docking parts.

[0010] In some embodiments, based on the aircraft docking area and the work vehicle docking area, a first image corresponding to the aircraft docking area and a second image corresponding to the work vehicle docking area are obtained, respectively. Specifically, this can be implemented as follows: Based on the aircraft docking area in the docking image, an aircraft mask is obtained. The resolution of the aircraft mask matches the resolution of the docking image. Based on the aircraft mask, the first image corresponding to the aircraft docking area and the second image corresponding to the work vehicle docking area are obtained.

[0011] In some embodiments, based on the first recognition result, the pixel size and physical size of the aircraft docking location are obtained; based on the second recognition result, the pixel size and physical size of the work vehicle docking location are obtained. Geometric ranging is performed based on the first and second recognition results to obtain the aircraft approach distance. Specifically, this can be implemented as follows: based on the first and second recognition results and the target focal length of the image acquisition device, the first absolute physical depth between the aircraft docking location and the optical center of the image acquisition device, and the second absolute physical depth between the work vehicle docking location and the optical center of the image acquisition device are obtained. Geometric ranging is then performed based on the first and second absolute physical depths to obtain the aircraft approach distance.

[0012] In some embodiments, based on the first recognition result, the second recognition result, and the target focal length of the image acquisition device, a first absolute physical depth between the aircraft docking part and the optical center of the image acquisition device, and a second absolute physical depth between the work vehicle docking part and the optical center of the image acquisition device are obtained. Specifically, this can be implemented as follows: the first absolute physical depth is determined using a pinhole equation based on the pixel size of the aircraft docking part, the physical size of the aircraft docking part, and the target focal length. The second absolute physical depth is determined using a pinhole equation based on the pixel size of the work vehicle docking part, the physical size of the work vehicle docking part, and the target focal length.

[0013] In some embodiments, the pinhole equation can be expressed as:

[0014] in, The first absolute physical depth, For the target focal length, These are the physical dimensions of the aircraft docking area. This refers to the pixel dimensions of the aircraft docking area. This refers to the physical pixel spacing of the sensor.

[0015] In some embodiments, geometric ranging based on a first absolute physical depth and a second absolute physical depth is used to obtain the aircraft approach distance. Specifically, this can be implemented as follows: Based on the first absolute physical depth, the three-dimensional coordinates of the pixel center of the aircraft docking point in the camera coordinate system are determined using a back-projection formula. These three-dimensional coordinates are then mapped to the world coordinate system to obtain the first coordinates. Based on the second absolute physical depth, the three-dimensional coordinates of the pixel center of the work vehicle docking point in the camera coordinate system are determined using a back-projection formula. These three-dimensional coordinates are then mapped to the world coordinate system to obtain the second coordinates. The Euclidean distance between the first and second coordinates is calculated. An offset correction is applied to the Euclidean distance to obtain the aircraft approach distance.

[0016] In some embodiments, the back projection formula can be expressed as:

[0017]

[0018]

[0019] in, For the camera coordinate system, This refers to the coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the X-axis in the camera coordinate system. This refers to the coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the Y-axis in the camera coordinate system. This refers to the coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the Z-axis in the camera coordinate system. The coordinates of the pixel center of the aircraft docking point or the work vehicle docking point on the first or second image along the x-axis. The coordinates of the pixel center along the y-axis of the aircraft docking point or the work vehicle docking point in the first or second image. The physical pixel pitch of the sensor. For the target focal length, The coordinates of the principal point along the x-axis in the first or second image. The coordinates of the principal point along the y-axis in the first or second image. It is either the first absolute physical depth or the second absolute physical depth.

[0020] In some embodiments, the first identification result includes the pixel size, category, and aircraft model of the aircraft docking part. The aircraft approach distance detection method provided in this application further includes: querying the physical size corresponding to the aircraft docking part from a physical size database based on the category and aircraft model of the aircraft docking part; performing focal length calibration based on the physical size and pixel size of the aircraft docking part to obtain the target focal length of the image acquisition device. By calibrating the focal length of the image acquisition device according to the physical size and pixel size of the aircraft docking part, the target focal length of the image acquisition device is obtained, and then the aircraft approach distance is determined based on the target focal length. Therefore, calibrating the focal length of the image acquisition device can improve the detection accuracy of the aircraft approach distance.

[0021] In some embodiments, focal length calibration is performed based on the physical dimensions and pixel dimensions corresponding to the aircraft docking area to obtain the target focal length of the image acquisition device. Specifically, this can be achieved by: performing depth inference processing on the docking image using a monocular depth estimation model to obtain the depth estimate corresponding to the aircraft docking area; obtaining the instantaneous focal length of the image acquisition device based on the pixel dimensions, physical dimensions, and depth estimate of the aircraft docking area; generating an instantaneous focal length sequence based on the instantaneous focal lengths of multiple docking images; and performing robust averaging processing on the instantaneous focal length sequence to determine the focal length of the image acquisition device.

[0022] In some embodiments, the aircraft docking distance detection method provided in this application further includes: performing post-filtering processing on the aircraft docking distance to obtain a smoothed value of the aircraft docking distance corresponding to the aircraft docking distance. When the steady-state locking condition is met, the average of the smoothed values ​​of the aircraft docking distances corresponding to a preset number of docking images closest to the current frame is taken as the target aircraft docking distance. When the steady-state locking condition is not met, the smoothed value of the aircraft docking distance is taken as the target aircraft docking distance. The steady-state locking condition includes: the operation time of the work vehicle reaches an operation time threshold, and the smoothed values ​​of the aircraft docking distances of multiple consecutive docking images are less than the aircraft docking distance threshold. By performing post-filtering processing on the aircraft docking distance and outputting a smooth and stable smoothed value of the aircraft docking distance, the ranging jitter and backtracking caused by anomalies such as instantaneous occlusion and sudden changes in light and shadow can be effectively suppressed.

[0023] In some embodiments, the post-filtering process includes at least one of the following: exponential decay weighted sliding window filtering, exponential moving average filtering with secondary low-pass processing, monotonically decreasing forced constraint, and abnormal jump suppression processing.

[0024] In some embodiments, the aircraft approach distance detection method provided in this application further includes: issuing a warning based on the target aircraft approach distance using at least one warning method. The at least one warning method includes: visual color warning, frequency conversion audio warning, tactile feedback, and network reporting.

[0025] In some embodiments, warnings are issued based on the target aircraft's approach distance using at least one warning method. Specifically, this can be implemented as follows: When the target aircraft's approach distance is greater than or equal to a first threshold, a first visual color warning is issued. The first visual color warning includes displaying the target aircraft's approach distance label background color in a first color on the display screen. When the target aircraft's approach distance is less than the first threshold and greater than or equal to a second threshold, a second visual color warning and a first frequency-modulated audio warning are issued. The second visual color warning includes displaying the target aircraft's approach distance label background color in a second color on the display screen, and the first frequency-modulated audio warning includes setting the buzzer frequency between a first frequency and a second frequency. The second threshold is less than the first threshold, and the first frequency is less than the second frequency. When the target aircraft's approach distance is less than the second threshold, a third visual color warning, a second frequency-modulated audio warning, and tactile feedback are issued, and the warning is reported to the network. The third visual color warning includes displaying the target aircraft's approach distance label background color in a third color on the display screen, and the second frequency-modulated audio warning includes setting the buzzer frequency to a second frequency. Tactile feedback includes issuing a warning through vibration. The first color indicates a lower urgency level for a shortened approach distance to the target aircraft compared to the second color. The second color indicates a lower urgency level for a shortened approach distance compared to the third color. By classifying aircraft approach distances and then implementing corresponding warning methods based on these classifications, the approach distances can be visually presented to frontline personnel, thereby assisting them in performing safe approach operations.

[0026] Secondly, this application provides an aircraft approach distance detection device, the device comprising: The acquisition module is used to acquire docking images through an image acquisition device. The docking images include at least the docking parts of the aircraft and the docking parts of the work vehicle.

[0027] The processing module is used to acquire a first image corresponding to the aircraft docking point and a second image corresponding to the work vehicle docking point, based on the aircraft docking point and the work vehicle docking point, respectively.

[0028] The recognition module is used to input the first image into the aircraft docking part recognition model for recognition, and obtain the first recognition result of the aircraft docking part. The aircraft docking part recognition model is a recognition model constructed based on the geometric characteristics of the aircraft docking part.

[0029] The recognition module is also used to input the second image into the recognition model of the docking part of the work vehicle for recognition, and obtain the second recognition result of the docking part of the work vehicle.

[0030] The processing module is also used to perform geometric ranging based on the first and second identification results to obtain the aircraft docking distance. The aircraft docking distance describes the distance between the aircraft docking point and the work vehicle docking point.

[0031] Thirdly, this application provides an electronic device, which includes: a memory and one or more processors; the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the above-mentioned aircraft approach distance detection method.

[0032] Fourthly, this application provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the above-mentioned aircraft approach distance detection method.

[0033] Fifthly, this application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, implements the above-mentioned aircraft approach distance detection method.

[0034] The solutions provided in aspects two through five above are used to implement the method provided in aspect one above, and their specific implementations will not be described in detail here. The technical effects corresponding to any implementation method of the solutions provided in aspects two through five above can be found in the technical effects corresponding to any implementation method in aspect one above, and will not be described in detail here.

[0035] It should be noted that any of the possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 A schematic diagram of an aircraft approach distance detection system provided in an embodiment of this application; Figure 2 A flowchart illustrating an aircraft approach distance detection method provided in an embodiment of this application; Figure 3 A schematic diagram of an aircraft docking point identification model provided in an embodiment of this application; Figure 4 A schematic diagram of pinhole camera imaging provided for an embodiment of this application; Figure 5 A flowchart illustrating another aircraft approach distance detection method provided in this application embodiment; Figure 6 A schematic diagram of a dual-channel ranging method provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an aircraft approach distance detection device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "front," "rear," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or relative positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and for simplification, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Unless otherwise specified, the above-mentioned orientational descriptions can be flexibly set in practical applications, provided that the relative positional relationships shown in the accompanying drawings are satisfied.

[0040] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0041] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "communication" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection. They can refer to a direct connection or an indirect connection through an intermediate medium, or a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0042] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0043] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0044] In the embodiments of this application, at least one can also be described as one or more, and multiple can be two, three, four or more, and this application does not impose any restrictions.

[0045] In the description of this specification, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0046] To facilitate understanding, the terms used in the embodiments of this application will be explained first.

[0047] Aircraft docking operations on civil aviation aprons refer to the entire process by which various ground support vehicles (hereinafter referred to as support vehicles or GSEs, i.e., Ground Support Equipment) perform tasks such as cargo loading and unloading, passenger boarding and disembarking, refueling, supplying clean water, discharging sewage, cleaning garbage, and towing / pushing close to the fuselage of a parked aircraft during the flight transit phase. According to the relevant standards of the International Air Transport Association (IATA) and the Civil Aviation Administration of China, when the support vehicle approaches the aircraft fuselage within a certain threshold (usually within five meters), it must perform a mandatory braking check and slowly approach at a very low speed. When the distance between the docking part of the support vehicle (such as the upper edge of the conveyor belt, the front edge of the loading and unloading platform, the tanker service port, etc.) and the docking part of the aircraft (such as the lower edge of the cargo door, the edge of the door, the lower edge of the service port, etc.) enters the permissible operating range, the commander issues a docking command and completes the final docking.

[0048] It should be noted that the information (including but not limited to device information, personal information of the subject, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the subject or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0049] The apron is a complex working environment characterized by high noise levels, limited space, immense time pressure, and the involvement of high-value aircraft assets. According to statistical models and industry research data from the International Air Transport Association (IATA), approximately 40% of apron damage accidents worldwide are caused by collisions between work vehicles and aircraft. Based on current flight density and the increasing number of new-generation composite material aircraft, it is projected that by 2035, global economic losses due to apron damage will approach $10 billion annually. The root cause of these accidents primarily stems from human factors: frontline workers are constrained by time pressure, physical fatigue, blind spots, and communication disruptions, making it difficult to accurately judge centimeter-level safety distances through visual estimation alone, and misjudgments are more likely to occur during the final contact phase.

[0050] Therefore, the introduction of intelligent technologies such as machine vision to automatically, continuously, and with high precision detect and warn of the relative distance between the work vehicle and the aircraft in real time has become an inevitable trend for the global civil aviation industry to improve the level of apron operation safety.

[0051] Existing distance measurement technologies for collision avoidance during apron operations mainly fall into the following four categories: (1) Commercial integrated collision avoidance system. This type of solution is provided by the original equipment manufacturer of ground equipment. It adopts a sensor deep fusion architecture, which is stable and plug-and-play. However, the purchase and maintenance cost of a single system is between RMB 150,000 and RMB 180,000. It is also highly dependent on mature overseas supply chains, making it difficult to meet the cost affordability and core technology self-control requirements for large-scale fleet deployment.

[0052] (2) LiDAR technology. By measuring the return time of high-frequency laser pulses, three-dimensional point clouds can be drawn, and the ranging accuracy can reach the centimeter level. However, the hardware cost per vehicle is about RMB 40,000 to RMB 50,000. At the same time, its performance is degraded under common harsh working conditions on the apron, such as rain, snow, fog and strong direct sunlight. Furthermore, the processing of point cloud data requires high edge computing power.

[0053] (3) Binocular vision technology. Based on the principle of bionic human eye binocular baseline triangulation, it has low-cost depth perception capability in close-range, textured scenes. However, binocular stereo matching is highly dependent on the feature point density of the object surface. Aircraft skin and cargo doors have large areas of low-texture surfaces, which can easily lead to parallax mismatch. At the same time, binocular cameras require accurate baseline calibration, which has poor calibration retention under tarmac vibration. Furthermore, real-time parallax calculation of dual-channel high-definition video streams consumes a lot of edge computing power.

[0054] (4) Monocular vision technology. This technology acquires two-dimensional images using only a single camera and relies entirely on backend computer vision algorithms to infer three-dimensional depth. The hardware cost of the monocular solution is extremely low (approximately 500 RMB for a single industrial-grade camera), and it is flexible in deployment. It can make full use of existing or newly added vehicle-mounted cameras on work vehicles, and it has strong economies of scale for large-scale fleets. Its core challenge lies in the fact that monocular images naturally lack absolute scale information, and the same two-dimensional observation can correspond to multiple real three-dimensional geometric structures. It is necessary to establish an accurate image-distance mapping relationship through algorithmic models.

[0055] In summary, in scenarios involving large airport fleets and highly diverse ground operations, monocular vision approaches offer significant advantages in terms of cost, deployment convenience, and economies of scale, making them the optimal technical approach for large-scale deployment across the entire fleet. However, directly applying existing general-purpose monocular vision ranging technologies (such as those relying solely on general target detection models or large monocular depth estimation models) to apron parking operations still has several inherent drawbacks and fails to meet the requirements for practical engineering applications.

[0056] Existing general monocular ranging solutions face the following technical bottlenecks in apron docking operations: The core measurement object for aircraft-to-aircraft distance measurement on the apron is not the outer boundary frame of the entire aircraft or the entire work vehicle in the image, but the "connection point" between the two that truly needs to maintain a safe distance in the sense of business procedures—such as the lower edge of the aircraft cargo door, the edge of the service port, the lowest point of the fuselage underside, the outer edge of the engine cowling, etc.; on the work vehicle side, it is the upper edge of the conveyor belt, the leading edge of the loading and unloading platform, the end of the tank truck service arm, the head of the tow bar, etc. Existing general object detection models (such as directly applying the general YOLO, Faster Region-based Convolutional Neural Network (Faster R-CNN) architecture, etc.) trained on public datasets such as the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) Dataset (ImageNet) and the Common Objects in Context (COCO) dataset can only provide coarse-grained overall bounding boxes for "aircraft" or "vehicles". They lack the ability to perform fine-grained semantic recognition of the docking parts of the aircraft and the docking parts of the work vehicle. Direct application of these models can lead to errors in the selection of distance measurement reference points and significant deviations of distance values ​​from the true safe distance.

[0057] More importantly, different docking parts of an aircraft (such as cargo doors being regular rectangles, service ports being small-sized circles or semi-ellipses, fairings being large-curvature surfaces, and the lower fuselage being complex three-dimensional surfaces) vary greatly in appearance, size, texture, and attitude variability. Using a single general network for "uniform" training would result in the model's capacity being taken up by a large number of easily identifiable categories, while the performance of difficult-to-identify categories would be severely insufficient. Furthermore, the recognition requirements for different parts (whether sub-pixel level corner points are needed, whether surface parameters are needed, and whether orientation angle estimation is needed) are also highly heterogeneous, and a simple shared network cannot simultaneously accommodate all of them.

[0058] Based on this, this application provides an aircraft docking distance detection method. It involves acquiring docking images including the aircraft docking point and the work vehicle docking point, then extracting images from these images to obtain a first image corresponding to the aircraft docking point and a second image corresponding to the work vehicle docking point. The first image includes only the aircraft portion, and the second image includes only the non-aircraft portion. Based on the first and second images, the aircraft docking point and the work vehicle docking point can be independently identified. Specifically, according to the geometric characteristics of the aircraft docking point, the first image is input into the aircraft docking point recognition model for identification, obtaining a first identification result for the aircraft docking point; the second image is input into the work vehicle docking point recognition model for identification, obtaining a second identification result for the work vehicle docking point. Identifying the first and second images separately reduces the search space and improves the efficiency and accuracy of identifying the aircraft docking point and the work vehicle docking point. Finally, geometric distance measurement is performed based on the first and second identification results to obtain the aircraft docking distance between the aircraft docking point and the work vehicle docking point. Since the first and second identification results accurately characterize the positions of the aircraft docking point and the work vehicle docking point... Therefore, the aircraft docking distance calculated based on the first and second identification results can accurately detect the distance between the aircraft docking point and the work vehicle docking point.

[0059] To facilitate understanding of the embodiments of this application, before introducing the aircraft approach distance detection method, we will first introduce the aircraft approach distance detection system to which the aircraft approach distance detection method is applicable.

[0060] Figure 1 This is a schematic diagram of an aircraft approach distance detection system provided in an embodiment of this application. Figure 1 As shown, the aircraft approach distance detection system 100 includes an image acquisition device 101, an inference calculation device 102, a data storage device 103, an information presentation device 104, and several communication and power auxiliary components 105. The components are interconnected by wired or wireless means and work together to complete the real-time approach distance detection function.

[0061] (1) Exemplarily, the image acquisition device 101 may be at least one monocular camera, with typical parameters including: a sensor width in the millimeter range (typically about 4.8 mm), a pixel physical pitch in the micrometer range (typically about 0.00375 mm per pixel), an image output resolution of 1280×720 or higher, and an output frame rate of 30 frames per second or higher. The camera may be installed in one or a combination of the following two methods: Method 1: Installed on the front roof, side wings, or vehicle body structure adjacent to the docking point of the work vehicle, with the viewing angle facing the aircraft fuselage, covering the two key targets of the work vehicle docking point and the aircraft docking point.

[0062] Method 2: Installed on a fixed position on the apron (such as a fixed structure such as a machine position column, a corridor bridge support, or a ground marker pole), with the viewing angle facing the machine operation area.

[0063] The image acquisition device 101 transmits the acquired video data to the subsequent inference computing device 102 via a wired harness or video streaming protocols such as Real Time Streaming Protocol (RTSP) / Real Time Messaging Protocol (RTMP).

[0064] (2) The inference computing device 102 may include an edge computing device and a back-end inference server.

[0065] The edge computing device includes an embedded AI computing module with at least 8 TOPS of neural network inference power, which can adopt an Advanced RISC Machines (ARM) architecture, a Reduced Instruction Set Computer – Version Five (RISC-V) architecture, or an equivalent embedded processor architecture. The edge computing device can be installed on the vehicle body or near the apron and directly connected to the image acquisition device 101 via a wiring harness to achieve low-latency localized, remote inference.

[0066] The backend inference server can be a graphics processing unit (GPU) server or an AI acceleration board cluster deployed in the airport data center. It can receive video streams from multiple cameras via 5G or wired networks and support centralized parallel inference for multiple work vehicles.

[0067] The two inference computing devices 102 described above can be deployed individually or in combination. Both of them are loaded with the aircraft docking distance detection model (such as the aircraft docking part identification model or the work vehicle docking part identification model) and various algorithm pipeline modules provided in the embodiments of this application.

[0068] (3) The data storage device 103 includes at least the following components: Model parameter storage unit: used to store the weight parameters of neural network models such as fuselage segmentation network, aircraft docking part recognition model, work vehicle docking part recognition model, and monocular depth estimation large model.

[0069] Physical Dimension Database: Organized by a two-dimensional index of aircraft type-dating part and work vehicle type-dating part, storing the actual physical dimensions of each docking part (such as the actual width of the aircraft cargo door, the actual width of the conveyor belt, the actual diameter of the service port, etc.). The data comes from legitimate sources such as civil aviation aircraft manuals (such as the Aircraft Flight Manual (AFM) / Aircraft Maintenance Manual (AMM)), aircraft maintenance manuals, and official technical parameters of work vehicle manufacturers.

[0070] Calibration parameter storage unit: Used to store calibration parameters such as camera focal length value, principal point coordinates, camera tilt angle, and system-level calibration offset compensation amount obtained after dynamic focal length self-calibration.

[0071] Timing buffer: Used to store ranging results from the most recent few frames, serving as input for timing filtering.

[0072] (4) The information presentation device 104 includes at least one or a combination of the following units: Visual display unit: such as a vehicle-mounted display screen installed in the driver's seat, or a large monitoring screen in the apron command console / dispatch center, used to display a real-time visual image with target marker boxes, key points, distance values, and graded warning colors.

[0073] Audio warning unit: such as a car speaker, used to output a sine wave beep warning signal with variable frequency according to the real-time distance.

[0074] Haptic feedback unit (optional), such as steering wheel or seat vibration device.

[0075] Network reporting unit: Used to report distance data and early warning events to the airport's central security control system via vehicle-to-everything (V2X) protocol.

[0076] (5) The communication and power auxiliary components 105 include vehicle power supply / airfield power supply module, wired / wireless network interface, video codec module, etc., to ensure the normal power supply and data connectivity of the above devices.

[0077] Image acquisition device 101 is connected to inference computing device 102 (edge ​​computing device / back-end inference server) via wiring harness or RTSP stream. Video frame data flows from image acquisition device 101 to inference computing device 102. Inference computing device 102 reads model parameters, real physical size database entries, and calibration parameters from data storage device 103 through storage interface. The ranging results and warning signals calculated by inference computing device 102 are pushed to information presentation device 104 via vehicle bus or airport intranet. At the same time, inference computing device 102 writes intermediate timing results into timing buffer for timing filtering of subsequent frames. Calibration parameters generated during the calibration stage are written back to calibration parameter storage unit by inference computing device 102 for subsequent use.

[0078] The operation of the above-mentioned aircraft approach distance detection system can be divided into two stages: the calibration stage and the normal distance measurement stage.

[0079] (1) The work process during the calibration phase: When the image acquisition device on the work vehicle is deployed for the first time, when the parameters of the image acquisition device change (such as lens replacement, post-maintenance reset, etc.), or when other preset trigger conditions are met, the system automatically enters the calibration phase. During this phase: S111: The image acquisition device sends the first N frames of docking images acquired to the inference computing device.

[0080] In some embodiments, N can be set according to the actual situation; for example, the value of N can range from 20 to 50.

[0081] S112: The inference computing device calls the deployed monocular depth estimation large model to process each docking image and output a dense relative depth map.

[0082] S113: The inference computing device calls the fuselage segmentation network to process the docking image to obtain an aircraft mask. Then, the aircraft mask generates an aircraft sub-image and a non-aircraft sub-image. The aircraft docking component recognition model and the work vehicle docking part recognition model are called on the aircraft sub-image and the non-aircraft sub-image respectively to obtain the bounding boxes and key points of the two docking parts, as well as the pixel size of the two docking parts.

[0083] S114: The inference computing device calculates the instantaneous focal length of each frame based on the relative depth, physical size, and pixel size according to the preset pinhole equation.

[0084] The pinhole equation can also be called the pinhole geometry inverse formula or the pinhole camera equation. The name of the pinhole equation is not limited in the embodiments of this application.

[0085] S115: Calculate the robust mean of the instantaneous focal lengths of N frames (e.g., remove the extreme values ​​at the beginning and end of each frame and then take the arithmetic mean) to obtain the calibrated target focal length. .

[0086] S116: Write the target focal length into the calibration parameter storage unit. The calibration phase ends, and the system enters the normal ranging phase.

[0087] (2) Working process during the normal distance measurement phase: The normal distance measurement phase is performed sequentially in the following order: S121: The image acquisition device continuously acquires docking images at a set frame rate and sends the acquired docking images to the inference computing device in real time.

[0088] S122: The inference computing device performs the following processing sequentially on each frame of docking image: fuselage segmentation, aircraft component-level dedicated recognition model integration inference, work vehicle docking part recognition, absolute depth calculation based on pinhole equation, world coordinate transformation, and relative distance calculation between the aircraft docking part and the work vehicle docking part.

[0089] S123: The inference computing device sends the frame-level ranging results to the multi-level time-series filtering module, which sequentially performs exponential decay weighted sliding window, exponential moving average (EMA) filtering, double low-pass filtering, monotonically decreasing forced constraint, lower bound protection, and steady-state locking determination, and outputs a smooth and stable distance, i.e., the smoothed value of the aircraft approach distance.

[0090] S124: The reasoning and calculation device outputs graded warning signals (visual color, audio frequency, tactile feedback, network reporting, etc.) based on the smoothed distance value.

[0091] S125: The warning signal is sent to the information display device via the communication link to assist front-line operators in completing safe machine operation.

[0092] Once the current docking operation is completed (e.g., after the work vehicle has moved a certain distance away from the aircraft), the system enters an idle state, waiting for the next docking operation to begin. During this process, the system can decide whether to re-execute the calibration phase based on preset adaptive triggering conditions (e.g., situations such as drastic changes in lighting or a continuous decrease in ranging confidence over multiple frames).

[0093] Before detailing the specific steps of the aircraft approach distance detection method provided in this application, the hierarchical architecture of the algorithm modules involved in the aircraft approach distance detection method is first clarified. The complete algorithm chain of recognition-ranging-filtering for the aircraft approach distance detection method provided in this application consists of the following three levels: First level (coarse segmentation layer): This level includes the fuselage segmentation module. The fuselage segmentation module is used to separate the docking image into a first image corresponding to the docking part of the aircraft and a second image corresponding to the docking part of the work vehicle.

[0094] The second level (component-level precision detection layer): This level consists of two parallel recognition branches (aircraft-side branch and vehicle-side branch). The aircraft-side branch is the aircraft docking part recognition model, which is internally composed of four recognition sub-models (rectangular, elliptical, curved surface, and anchor point) designed for four types of aircraft docking part geometric characteristics, integrated through a unified interface specification; the vehicle-side branch is the work vehicle docking part recognition model.

[0095] In some embodiments, the aircraft docking part identification model may also be referred to as an aircraft component-level dedicated identification model integration. The name of the aircraft docking part identification model is not limited in this application embodiment.

[0096] The third layer (ranging and filtering layer) consists of a dual-channel ranging module (a prior channel for the relative depth of the large model and a physical geometry channel for the pinhole camera, coupled through dynamic focal length self-calibration) and a multi-level temporal filtering module (exponential decay weighted sliding window, EMA secondary low-pass filter, monotonically decreasing forced constraint, lower bound protection, and steady-state locking) connected in series.

[0097] Based on the above three-tier architecture, this application provides a method for detecting aircraft approach distance.

[0098] Figure 2 This is a flowchart illustrating an aircraft approach distance detection method provided in an embodiment of this application. The method can be executed by an electronic device.

[0099] like Figure 2 As shown in the embodiments of this application, the aircraft approach distance detection method may include: S201: Acquire docking images through an image acquisition device.

[0100] The docking images include at least the docking points of the aircraft and the work vehicle.

[0101] For example, the image acquisition device can be a camera installed at a fixed location on a work vehicle or airport apron.

[0102] For example, a camera captures monocular RGB images or video frame sequences of the machine operation scene; that is, the camera's photoelectric sensor converts the optical signals within the field of view into digital RGB images within each frame exposure cycle. .

[0103] Where t is the frame number, and t=0 corresponds to the initial frame at the system startup time.

[0104] Digital RGB image The dimensions can be H×W×3.

[0105] Where H and W are the height and width of the image, respectively, with typical values ​​for H=720 and W=1280; 3 refers to the three color channels R, G, and B.

[0106] Then, for the above digital RGB image Perform at least one of the following preprocessing steps: (1) Distortion correction: If the camera is a fisheye or wide-angle lens, the original image is radially distorted using pre-stored distortion coefficients to obtain an image that approximates perspective projection. .

[0107] (2) Exposure normalization: Adaptive limited histogram equalization (CLAHE) or equivalent methods are used to alleviate dynamic range compression under strong light / weak light conditions.

[0108] (3) Size scaling (optional): Based on the subsequent model input requirements, unify the image size to the model's expected input size (e.g., 640×640), and record the scaling ratio. This is so that the pixel coordinates can be restored later.

[0109] In digital RGB images After preprocessing, the docking image is obtained. docking images It serves as the input for all subsequent modules.

[0110] Where t is the frame number, and t=0 corresponds to the initial frame at the system startup time.

[0111] S202: Based on the aircraft docking point and the work vehicle docking point, acquire the first image corresponding to the aircraft docking point and the second image corresponding to the work vehicle docking point, respectively.

[0112] After obtaining the docking images, the docking images are divided into aircraft part sub-images (only the aircraft area sub-images are retained) and non-aircraft part sub-images (only the non-aircraft area sub-images are retained).

[0113] The first image corresponding to the docking point with the aircraft is the aircraft sub-image; the second image corresponding to the docking point with the work vehicle is the non-aircraft sub-image.

[0114] For example, based on the a priori aviation fact that the aircraft is the largest and most geometrically complete target in the image, the image can be divided into an aircraft foreground region (i.e., the aforementioned aircraft sub-image) and a non-aircraft background region (i.e., the aforementioned non-aircraft sub-image) by the fuselage segmentation module.

[0115] The fuselage segmentation module can employ semantic segmentation network architectures such as U-shape Network for Biomedical Image Segmentation (U-Net), Simple and Efficient Design for Semantic Segmentation with Transformers (SegFormer), Masked-attention Mask Transformer for Universal Image Segmentation (Mask2Former), and Segment Anything Model (SAM).

[0116] In some embodiments, an aircraft mask is obtained based on the aircraft docking area in the docking image. Based on the aircraft mask, a first image corresponding to the aircraft docking area and a second image corresponding to the work vehicle docking area are obtained respectively.

[0117] The resolution of the aircraft mask is matched with the resolution of the docking image.

[0118] For example, after obtaining the docking image Then, the docking images will be... The image is sent to the fuselage segmentation module to obtain the docking image. Binary aircraft mask with equal resolution . It can also be called an aircraft foreground mask, which physically corresponds to the image area occupied by the aircraft.

[0119] .

[0120] in, For aircraft masking, It is a size of H The binary matrix of W, Represents pixels This area belongs to the foreground region of the aircraft. This indicates an area that is not in the background of an aircraft.

[0121] After obtaining the aircraft cover The first image can then be obtained using the following formula: .

[0122] in, For the first image, To dock images, For aircraft masking, This indicates pixel-wise masking.

[0123] For example, by aircraft mask Generate the first image: .

[0124] in, For the first image, To dock images, For aircraft masking, Used to indicate the image column index, Used to indicate the image row index Used to indicate the RGB channel index (0=R, 1=G, 2=B).

[0125] Furthermore, the second image can be obtained using the following formula: .

[0126] in, For the second image, To dock images, For aircraft masking, This indicates pixel-wise masking.

[0127] For example, by aircraft mask Generate a second image: .

[0128] in, For the second image, To dock images, For aircraft masking, Used to indicate the image column index, Used to indicate the image row index Used to indicate the RGB channel index (0=R, 1=G, 2=B).

[0129] The above technical solution enables the accurate identification and positioning of the true operational benchmark for ranging from a single RGB image in a tarmac scenario with large areas of similar metal skin, complex background interference, and dynamic occlusion of multiple targets. This benchmark is the docking points of the aircraft and the work vehicle, rather than the overall outer frame of the two.

[0130] Unlike existing general object detection methods that directly detect all targets on the entire image, this application leverages the inherent prior knowledge of aviation scenarios—that "in an apron docking operation scenario, the aircraft is the largest and most structurally rigid target in the image"—to propose a two-level hierarchical recognition architecture. First, a semantic segmentation model separates the input image into aircraft and non-aircraft sub-images. Then, two functionally and strategically independent recognition branches identify the aircraft docking area and the work vehicle docking area on their respective sub-images. This paradigm significantly outperforms general object detection methods in terms of search space, model capacity allocation, and false detection rate control.

[0131] S203: Input the first image into the aircraft docking part recognition model for recognition, and obtain the first recognition result of the aircraft docking part.

[0132] Among them, the aircraft docking part recognition model is a recognition model built based on the geometric characteristics of the aircraft docking part.

[0133] In some embodiments, the aircraft includes multiple aircraft docking parts with different geometric characteristics, and the aircraft docking part identification model includes multiple sub-identification models corresponding to the geometric characteristics of each aircraft docking part.

[0134] For example, the geometric characteristics of the aircraft docking area include at least rectangular, elliptical, curved surface, and anchor point types. Multiple sub-recognition models include a rectangular sub-recognition model corresponding to the rectangular type, an elliptical sub-recognition model corresponding to the elliptical type, a curved surface sub-recognition model corresponding to the curved surface type, and an anchor point sub-recognition model corresponding to the anchor point type.

[0135] Rectangular shapes can also be called regular rectangular shapes, such as the geometric characteristics of aircraft docking parts like cargo doors and emergency doors. Elliptical shapes can also be called small-sized circular / semi-elliptical shapes, such as the geometric characteristics of aircraft docking parts like service ports and water supply interfaces. Curved surfaces can also be called large-curvature curved surfaces, such as the geometric characteristics of aircraft docking parts like fairings and wing leading edges. Anchor point shapes refer to fuselage underside / low-altitude bonding shapes.

[0136] In some embodiments, target geometric characteristics of the aircraft docking location in the first image are determined. The first image is then input into a sub-recognition model corresponding to the target geometric characteristics for recognition, resulting in a first recognition result for the aircraft docking location in the first image.

[0137] Specifically, after obtaining the first image Then, the first image The aircraft docking area recognition model was input for recognition, and the first recognition result was obtained. ={Category Labels, Markup Boxes, Key Points or Parametric Profiles, Confidence Score}.

[0138] Each type of aircraft docking part corresponds to a "dedicated recognition model" (i.e., the aforementioned sub-recognition model) that is specifically optimized in terms of network structure, training data, loss function, and key point topology. These models are then integrated into a set of recognition models (i.e., the aforementioned aircraft docking part recognition model) through a unified output interface specification.

[0139] Figure 3 A schematic diagram of an aircraft docking point identification model provided in this application embodiment, as shown below. Figure 3 As shown, the aircraft docking part recognition model is divided into four sub-recognition models based on the geometric characteristics of the aircraft docking part (rectangular sub-recognition model, elliptical sub-recognition model, curved surface sub-recognition model, and anchor point sub-recognition model). Each sub-recognition model is trained and inferred independently, and its input is the first image. The output is organized according to the following standardized interface format: .

[0140] in, As the first identification result, For type tags, For the marker box, For key points or parametric contours, The confidence level is used for subsequent weighting or validity determination of the fusion.

[0141] Classification of aircraft docking parts Used to indicate rectangle class, Used to indicate ellipse class, It is a curved surface class. For anchor point class.

[0142] These are the pixel coordinates of the top left and bottom right corners of the marker box.

[0143] The specific definitions vary depending on the category: (1) Rectangle class .

[0144] in, It refers to the subpixel-level coordinates of the upper left corner of the docking point of a rectangular aircraft. It refers to the subpixel-level coordinates of the upper right corner of the docking point of a rectangular aircraft. It refers to the subpixel-level coordinates of the lower right corner of the docking point of a rectangular aircraft. It refers to the subpixel-level coordinates of the lower left corner of the docking point of a rectangular aircraft.

[0145] (2) Elliptic : .

[0146] in, The x-coordinate of the center pixel of the ellipse. Let y be the ordinate of the center pixel of the ellipse, a be the major semi-axis, and b be the minor semi-axis. It represents the rotation angle.

[0147] (3) Curved surfaces : .

[0148] in, The n+1 coefficients of the polynomial fitted to the edge of the corresponding curved surface projection.

[0149] (4) Anchor point type : .

[0150] in, The sub-pixel coordinates corresponding to m predefined security reference anchor points.

[0151] The key design considerations for the network structure of each sub-recognition model are as follows: (1) For the rectangular sub-identification model, a cascaded structure of “target detection network + lightweight four-corner point heat map regression network” is adopted, and Soft-Argmax with temperature parameters is introduced to obtain sub-pixel level corner points.

[0152] Specifically, an object detection network (such as YOLOv8, RT-DETR, etc.) is used to first output a coarse-grained bounding box (bbox). Then, a local region image is cropped from within the bbox and fed into a keypoint regression network. The keypoint regression network is a lightweight encoder-decoder structure: the encoder processes the local image (typically 256×256×3) through three dimensionality reduction stages (channel numbers are 3, 32, 64, and 256 respectively), and the decoder, after passing through a feature transition layer (channel numbers are 256, 128, and 64 respectively, with 3×3 convolutional kernels), outputs a 4-channel heatmap via a 1×1 convolution, with each channel corresponding to the probability distribution of a corner point. Soft-Argmax activation with a temperature parameter τ is applied to the heatmap. .

[0153] in, This refers to the probability heatmap after softmax normalization. It refers to an exponential normalization function. It refers to a heat map. This refers to the temperature coefficient. The value of τ can be set according to the actual situation. For example, τ can be 10. In this embodiment of the application, the value of τ is not limited.

[0154] .

[0155] Where x refers to the predicted x-coordinate. It refers to the probability value of the i-th row and j-th column, and satisfies that the sum of the probabilities of all positions is 1, where j refers to the pixel index in the x-direction.

[0156] .

[0157] Where y refers to the predicted y-coordinate. This refers to the probability value of the i-th row and j-th column, where i refers to the pixel index in the y-direction.

[0158] This yields sub-pixel level corner coordinates that break through the physical pixel resolution limit. Key points are numbered according to a conventional clockwise topology to facilitate subsequent input of physical distance measurement along the baseline (such as the line connecting point 2 and point 3).

[0159] (2) For the ellipse sub-identification model, the structure of "target detection network + ellipse parameter direct regression branch" is adopted, the five parameters of the ellipse are regressed and the reprojection intersection over union (IoU) loss is used.

[0160] Specifically, after the object detection network outputs the bounding box, a lightweight parametric regression branch (such as a 3-layer fully connected layer) is added to directly regress the five parameters of the ellipse. The loss function used during training is the reprojection IoU loss: .

[0161] in, This refers to elliptic loss. This is a binary elliptical mask rendered by five parameters. The cross-union ratio of the binary mask. To predict the parameter vector of the ellipse, is the parameter vector of the truth ellipse.

[0162] (3) For the surface sub-identification model, the structure of "target detection network + edge contour segmentation sub-network" is adopted to perform parametric curve fitting on the surface projection edge.

[0163] Specifically, after the object detection network outputs the bounding box (bbox), an edge segmentation subnet (U-Net structure) is added, which outputs a binary edge mask of equal resolution for the local image within the bbox. Then, a parameterized curve is obtained through skeleton thinning and least-squares polynomial fitting. The training loss is the sum of the IoU loss and the distance transformation loss. .

[0164] in, This refers to surface loss. , These are weighting coefficients. Calculated based on edge distance transformation map This is due to IoU loss.

[0165] (4) For the anchor point sub-identification model, the structure of "target detection network + safe benchmark anchor point regression" is adopted to regress the coordinates of the key anchor points in the pre-learning.

[0166] The predefined anchor point locations are obtained by performing K-means clustering on historical data of the aircraft model. The anchor point regression loss uses smoothed L1 loss.

[0167] Various sub-recognition models are uniformly invoked through a dispatcher. Based on the category of the target detection output, the local region image is routed to the corresponding sub-recognition model for fine recognition, and finally the first recognition result is output in the standardized interface format described above. .

[0168] The aforementioned aircraft docking part recognition model integration architecture has made targeted adaptations for each type of aircraft docking part in the model structure layer, training data layer, and training strategy layer. Compared with the existing technology that uses a large model to recognize all aircraft docking parts, this technology ensures that the recognition model takes into account the recognition accuracy of each part, even under the premise that the geometric and appearance characteristics of different aircraft docking parts are significantly heterogeneous. This avoids the unified model being dominated by easily recognizable categories while its performance degrades on difficult-to-recognize categories.

[0169] S204: Input the second image into the identification model of the docking part of the work vehicle for identification, and obtain the second identification result of the docking part of the work vehicle.

[0170] Specifically, upon obtaining the second image Then, the second image The identification model of the docking part of the work vehicle is fed in, and the second identification result is obtained. : .

[0171] in, This is the second identification result. For the type label of the docking part of the work vehicle, The marking box is for the docking area of ​​the work vehicle. This is a key point for the docking of the work vehicle. , where is the confidence level.

[0172] The identification model for the docking part of the work vehicle also adopts the "target detection + key point regression" structure. That is, the structure used by the identification model for the docking part of the work vehicle is the same as that used by the anchor point sub-identification model mentioned above, but the key point topology is set differently according to the type of work vehicle.

[0173] S205: Based on the first and second identification results, perform geometric ranging to obtain the aircraft's approach distance.

[0174] Among them, the aircraft docking distance is used to describe the distance between the docking point of the aircraft and the docking point of the work vehicle.

[0175] In some embodiments, the pixel size and physical size of the aircraft docking part are obtained based on the first recognition result, and the pixel size and physical size of the work vehicle docking part are obtained based on the second recognition result.

[0176] In some embodiments, the first identification result may further include the pixel size, category, and aircraft model of the aircraft docking part. Based on the category and aircraft model of the aircraft docking part, the physical size corresponding to the aircraft docking part is queried from a physical size database. Based on the physical size and pixel size of the aircraft docking part, focal length calibration is performed to obtain the target focal length of the image acquisition device.

[0177] In some embodiments, based on the first recognition result, the second recognition result, and the target focal length of the image acquisition device, a first absolute physical depth between the aircraft docking point and the optical center of the image acquisition device, and a second absolute physical depth between the work vehicle docking point and the optical center of the image acquisition device are obtained. Geometric ranging is then performed based on the first and second absolute physical depths to obtain the aircraft approach distance.

[0178] In addition, the following problems exist in the existing technology: (1) Physical scale distortion and timing jitter in pure data-driven models.

[0179] Large monocular depth estimation models, such as the Anything Depth Model, the ZoeDepth Zero-Shot Relative-Metric Fusion Depth Model, and the MiDaS Monocular Depth Estimation Model, can output dense relative depth maps with complete structure in natural scenes. However, their directly output absolute physical scale has the following inherent defects: First, the model relies on global scale priors during the training period. In unseen station scenes (strong light, weak light, metallic reflection, ground marking interference, etc.), its absolute scale has significant shifts. Second, there is a clear discontinuity in scale estimation at long distances (about 5 meters and above) and extremely short distances (10 centimeters), making it difficult to cover the cross-scale measurement needs of the entire machine operation process. Third, frame-by-frame independent inference inevitably produces high-frequency jitter and jumps in continuous video streams, making it unsuitable for direct use in millimeter-level precision safety control decisions.

[0180] (2) Traditional physical calibration schemes are not feasible for large-scale deployment of work vehicles.

[0181] Classical physical ranging methods, such as pinhole camera models, rely on precise camera intrinsic parameters (especially focal length). However, traditional methods for acquiring intrinsic parameters require the use of physical calibration boards, such as checkerboards, to acquire multi-pose images in a controlled environment and perform least-squares fitting. This approach has significant limitations in apron scenarios: First, airports have thousands of operation vehicles of various models and camera specifications, making offline calibration for each vehicle a massive undertaking; second, vibrations during vehicle use and maintenance / parts replacement can cause camera parameter drift, requiring periodic recalibration; and third, the camera installation positions of some operation vehicles are unique, making it difficult for physical calibration boards to reach the required field of view for calibration.

[0182] Based on this, this application provides a focal length calibration method to obtain the target focal length of the image acquisition device.

[0183] For example, the target focal length of the image acquisition device can be obtained in the following manner.

[0184] In some embodiments, a monocular depth estimation model is used to perform depth inference processing on the docking images to obtain depth estimates corresponding to the aircraft docking locations. Based on the pixel size, physical size, and depth estimates corresponding to the aircraft docking locations, the instantaneous focal length of the image acquisition device is obtained. Based on the instantaneous focal lengths of multiple docking images, an instantaneous focal length sequence is generated. Robust averaging is performed on the instantaneous focal length sequence to determine the target focal length of the image acquisition device.

[0185] Specifically, a target object can be selected from the first image or the second image. Based on the pixel size, physical size and depth estimate of the target object, the focal length of the image acquisition device can be reversed to obtain the target focal length of the image acquisition device.

[0186] In the process of determining the target focal length of the image acquisition device, the aircraft docking part, the work vehicle docking part, or other landmark objects in the first or second image can be selected as the target object.

[0187] The following description uses the aircraft docking area as an example: First, the pixel size and physical size of the aircraft docking area are acquired. Simultaneously, a monocular depth estimation model is used to perform depth inference processing on the first image to obtain the depth estimate corresponding to the aircraft docking area. Then, based on the pixel size, physical size, and depth estimate, the focal length of the aircraft docking area is inferred to obtain the instantaneous focal length. Finally, based on the instantaneous focal length, the target focal length of the image acquisition device is determined.

[0188] Specifically, the pixel dimensions of the aircraft docking area can be obtained from the first recognition result in the following manner: Obtain the pixel dimensions of the aircraft docking area For example, the rectangle class takes the distance between the two corner points of the bottom edge, the ellipse class takes the major axis, the surface class takes the projected length of the parametric curve, and the anchor point class takes the distance between the two anchor points.

[0189] The physical dimensions of the aircraft docking parts can be obtained by: based on the aircraft docking part category field. and aircraft type field The physical dimensions of the aircraft docking parts are obtained by querying the two-dimensional index in the physical dimension database. .

[0190] Specifically, if the current frame ( To set a frame count threshold for the calibration phase (typically 30), the docking images will be... The data is fed into a large monocular depth estimation model to obtain a dense relative depth map. Combined with the first identification result Second identification result By using the pixel dimensions and physical dimensions in the image, the focal length is calculated by performing a depth-size relationship on the aircraft docking area to obtain the instantaneous focal length. .

[0191] when At that time, for the sequence Perform robust averaging to obtain the calibrated target focal length. and target focal length Write to the calibration parameter storage unit.

[0192] Figure 4 This is a schematic diagram of pinhole camera imaging provided in an embodiment of this application, as shown below. Figure 4 As shown, based on the imaging principle of a pinhole camera, a point in three-dimensional physical space... The core geometric equation for projecting onto a two-dimensional digital image plane is: .

[0193] .

[0194] Where f is the camera focal length (in millimeters). Let P be the pixel coordinates of point P on the image plane. Principal point coordinates (image center). Z represents the three-dimensional coordinates of point P in the camera coordinate system (Z is the depth to the camera optical center, in meters).

[0195] Based on the principle of similar triangles, if the pixel size of the target object in a certain direction of the image is... (pixels), the actual physical size of the target object in physical space is (Unit: meters), then its depth Z satisfies the following formula: .

[0196] Where Z is the depth and f is the focal length. P represents the physical size of the target object, and P represents the physical pixel pitch of the sensor (millimeters per pixel, typically about 0.00375 mm / px). The pixel size of the target object.

[0197] In situations where the focal length f is unknown but we want to deduce it from the unknown, we need to introduce an independent depth prior. As an estimate of Z. Depth prior. A large model for monocular depth estimation can be used to analyze docking images. The reasoning leads to the conclusion that, specifically, robust statistical values ​​(such as the median or the mean after outlier filtering) are taken from the large model depth output of the region where the target docking site is located.

[0198] In some embodiments, the depth prior may also be referred to as the depth estimate. This application does not limit the name of the depth prior.

[0199] In obtaining deep priors Afterwards, Substituting into the above equation to solve for f, we can obtain the instantaneous focal length in that frame: .

[0200] in, The instantaneous focal length in the current frame t. This represents the relative depth estimate from the target object to the camera optical center, output by the large monocular depth estimation model. P is the physical pixel spacing of the sensor. The pixel size of the target object in the current frame t. The physical dimensions of the target object.

[0201] Continuously execute the above reverse calculation frame( (Typically 30), to obtain the instantaneous focal length sequence. .

[0202] Then, robust averaging is performed on the instantaneous focal length sequence to obtain the target focal length. : .

[0203] in, The target focal length is used, the mean function is used to calculate the arithmetic mean of the array elements, and trim means to remove the extreme values ​​at the beginning and end by 10%.

[0204] Furthermore, the final target focal length is obtained. Write the calibration parameters into the storage unit as the fixed focal length used during the normal ranging phase.

[0205] Understandably, this is based on the pinhole model. Multiply both sides Divide by f can then be solved inversely, and this algebraic derivation is rigorous; robust mean removal of extrema is a routine noise reduction operation in statistics, and its rationality is supported by the central limit theorem and outlier detection theory.

[0206] The technical solution provided in this application can achieve automatic dynamic calibration of camera intrinsic parameters (especially focal length) under engineering constraints such as variable station lighting, a wide variety of work vehicles, different camera models, and the inability to use physical calibration boards, and further restore the absolute physical scale required for distance measurement.

[0207] This application proposes a dual-channel ranging architecture that nests the relative depth output of a large monocular depth estimation model (such as Depth Anything V3, ZoeDepth, etc.) with a classic pinhole camera physical imaging model. The large model's relative depth channel provides virtual calibration priors, while the pinhole physical channel calculates the absolute distance based on real physical dimensions and geometric imaging laws. The two channels are coupled via focal length back-inference. Only during the system startup calibration phase (and when recalibration is triggered) does the large model's depth channel provide one-time prior information to the physical channel to complete focal length calibration. Once the normal ranging phase begins, the large model's depth channel is no longer invoked, and all frame-level ranging is performed independently by the pinhole physical channel, significantly reducing inference latency and computational cost during normal ranging. This dual-channel architecture overcomes the scale drift problem of pure deep learning models and avoids the engineering limitations of traditional physical calibration boards in station environments, enabling industrial-grade absolute distance measurement without a calibration board.

[0208] Specifically, if the current frame The first identification result Second identification result Including the target focal length The data is fed into the physical geometry ranging module, which calculates the absolute physical depths of the aircraft docking point and the work vehicle docking point according to the pinhole equation. Then, through a rotation transformation from the camera coordinate system to the world coordinate system, the three-dimensional coordinates of both are obtained in the world coordinate system. Finally, the relative distance in the current frame is obtained by subtracting the system-level calibration offset compensation from the Euclidean distance. , That is, the distance between the aircraft and the aircraft.

[0209] The pinhole equation can be expressed as: .

[0210] in, The physical meaning is "the actual physical length (in millimeters) occupied by the docking part of the aircraft / the docking part of the work vehicle on the camera sensor plane", that is, "the number of pixels × the sensor pixel pitch = the length of the sensor plane in millimeters"; The actual physical length (in meters) of the aircraft docking point / operation vehicle docking point in three-dimensional physical space. Z represents the target focal length of the image acquisition device (in millimeters); Z represents the absolute physical depth (in meters) from the aircraft docking point / operation vehicle docking point to the optical center of the image acquisition device.

[0211] As one possible implementation, the first absolute physical depth is determined using the pinhole equation based on the pixel size of the aircraft docking area, the physical size of the aircraft docking area, and the target focal length: .

[0212] in, The first absolute physical depth, For the target focal length, These are the physical dimensions of the aircraft docking area. This refers to the pixel dimensions of the aircraft docking area. This refers to the physical pixel spacing of the sensor.

[0213] As one possible implementation, the second absolute physical depth is determined using the pinhole equation based on the pixel size of the docking point of the work vehicle, the physical size of the docking point, and the target focal length. .

[0214] in, The second absolute physical depth, For the target focal length, The physical dimensions of the docking point between the work vehicle and the work platform. The pixel dimensions are for the docking area of ​​the work vehicle. This refers to the physical pixel spacing of the sensor.

[0215] After obtaining the first absolute physical depth Second absolute physical depth Then, based on the first absolute physical depth Second absolute physical depth Perform geometric ranging to obtain the aircraft's approach distance.

[0216] In some embodiments, based on a first absolute physical depth, the three-dimensional coordinates of the pixel center of the aircraft docking location in the camera coordinate system are determined using a back-projection formula. These three-dimensional coordinates are then mapped to the world coordinate system to obtain the first coordinates. Based on a second absolute physical depth, the three-dimensional coordinates of the pixel center of the work vehicle docking location in the camera coordinate system are determined using a back-projection formula. These three-dimensional coordinates are then mapped to the world coordinate system to obtain the second coordinates. The Euclidean distance between the first and second coordinates is calculated. An offset correction is applied to the Euclidean distance to obtain the aircraft approach distance.

[0217] After obtaining the first absolute physical depth Then, for the pixel center of the aircraft docking point in the first image. The three-dimensional coordinates of the pixel center of the aircraft docking area in the camera coordinate system are obtained by using the back projection formula. .

[0218] The back projection formula can be expressed as: .

[0219] .

[0220] .

[0221] in, For the camera coordinate system, This refers to the coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the X-axis in the camera coordinate system. This refers to the coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the Y-axis in the camera coordinate system. This refers to the coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the Z-axis in the camera coordinate system. The coordinates of the pixel center of the aircraft docking point or the work vehicle docking point on the first or second image along the x-axis. The coordinates of the pixel center along the y-axis of the aircraft docking point or the work vehicle docking point in the first or second image. The physical pixel pitch of the sensor. For the target focal length, The coordinates of the principal point along the x-axis in the first or second image. The coordinates of the principal point along the y-axis in the first or second image. It is either the first absolute physical depth or the second absolute physical depth.

[0222] The following is an introduction to the camera coordinate system C and the world coordinate system W: The camera coordinate system, with the optical center of the image acquisition device as the origin and the optical axis as the Z-axis, is denoted as C; the world coordinate system is denoted as W, and its origin is usually taken as the projection point on the ground below the image acquisition device, with the Z-axis pointing horizontally forward, the X-axis pointing horizontally to the right, and the Y-axis pointing vertically upward.

[0223] Let the elevation angle of the image acquisition device be... (Typically about 15°, indicating that the optical axis of the image acquisition device is tilted downwards relative to the horizontal ground), yaw angle is... The roll angle is The rotation matrix from the C system to the W system is: .

[0224] in, , , These are the basic rotation matrices about the X, Y, and Z axes, respectively.

[0225] For the pixel center of the docking point in the image, the three-dimensional coordinates in the camera system are obtained by pinhole back projection: .

[0226] .

[0227] .

[0228] In the back projection formula, and The physical meaning is also "the distance in millimeters that the pixel is offset from the principal point on the sensor plane" multiplied by Z / The three-dimensional coordinates of the camera system are obtained by scaling up the similar triangles.

[0229] Then, the three-dimensional coordinates of the pixel center of the aircraft docking area in the camera coordinate system are obtained using the following formula. Mapped to world coordinate system : .

[0230] Similarly, upon obtaining the second absolute physical depth... Then, for the pixel center of the docking part of the work vehicle on the second image. The three-dimensional coordinates of the pixel center of the docking part of the work vehicle in the camera coordinate system are obtained by using the back projection formula. Then, the three-dimensional coordinates of the pixel center of the docking point of the work vehicle in the camera coordinate system are obtained using the following formula. Mapped to world coordinate system .

[0231] Then, the Euclidean distance between the aircraft docking point and the work vehicle docking point in the world coordinate system is calculated using the following formula, i.e., the calculation... and The Euclidean distance between them.

[0232] .

[0233] in, The Euclidean distance between the aircraft docking point and the work vehicle docking point in the world coordinate system, in meters; Used to calculate the square root of a given value; The three-dimensional coordinates of the pixel center of the aircraft docking area in the world coordinate system; The pixel center of the docking part of the work vehicle is given in three-dimensional coordinates in the world coordinate system.

[0234] This represents the spatial distance between the two identification points from the camera's perspective. However, since the image acquisition device is not installed at the docking point of the work vehicle itself (such as the upper edge of the conveyor belt, the end of the service arm, etc.), the "work vehicle side reference point" observed within the field of view of the image acquisition device is usually a landmark point on the docking part of the work vehicle. This landmark point has a fixed physical offset along the working direction from the actual "docking end" in the business sense, denoted as . This offset was measured on-site with a tape measure when the work vehicle was first deployed (typically about 37 centimeters). Its physical meaning is the distance from the "work vehicle reference point pointed to by the camera" along the machine approach direction (approximately the forward direction of the Z-axis in the world system) to the "actual docking end".

[0235] Therefore, the relative distance between the actual docking end of the work vehicle and the docking point of the aircraft. for: .

[0236] in, This represents the relative distance (in meters or centimeters) between the docking points of the work vehicle and the aircraft in the current frame output. This represents the Euclidean distance between the aircraft docking point and the work vehicle docking point in the world coordinate system, expressed in meters. The fixed physical offset (in meters) from the reference point of the work vehicle to the docking end along the machine direction.

[0237] The formula assumes The direction approximates with The main directions should coincide (i.e., the docking direction of the work vehicle should be aligned with the horizontal forward direction of the camera's Z-axis). This applies to the vast majority of front-approaching work vehicles (conveyor belt vehicles, loading and unloading platform vehicles, tank work vehicles, etc.). If the docking direction of individual models is misaligned, it can be adjusted accordingly. Change to a three-dimensional offset vector in the coordinate system of the work vehicle and subtract the vectors after coordinate transformation.

[0238] Furthermore, there may be a physical offset between the actual docking end of the aircraft and the "aircraft side reference point" observed within the field of view of the image acquisition device. If there is a physical offset between the actual docking end of the aircraft and the aircraft side reference point, then... minus Finally, the physical offset between the actual docking end of the aircraft and the aircraft side reference point needs to be subtracted.

[0239] Furthermore, after obtaining the aircraft approach distance, the aircraft approach distance can be post-filtered to effectively suppress ranging jitter and backtracking caused by anomalies such as instantaneous occlusion and sudden changes in light and shadow.

[0240] The existing technology still has the following problems: it lacks timing constraints on the physical laws of the machine operation process.

[0241] The approaching operation is a physical process that proceeds from far to near with a monotonically decreasing distance. The reverse process (the operation vehicle moving away from the aircraft) usually occurs during the departure phase after the operation is completed. Logically, this can be distinguished from the approaching phase, where the ranging system requires highly reliable early warning. Existing general ranging schemes calculate each frame independently, lacking the utilization of the deterministic physical laws of the operation process. This makes it impossible to logically suppress anomalies such as ranging value regression and sudden jumps caused by instantaneous occlusion, sudden changes in light and shadow, and target detection failure.

[0242] Based on this, the embodiments of this application provide the following technical solution: by performing post-filtering processing on the aircraft approach distance, the ranging jitter and backtracking caused by anomalies such as instantaneous occlusion and sudden changes in light and shadow can be effectively suppressed.

[0243] Specifically, after obtaining the aircraft's approach distance Afterwards, The data is fed into a multi-stage time-series filtering module, which sequentially performs exponential decay weighted sliding window filtering, EMA double low-pass filtering, monotonically decreasing forced constraint, and lower bound protection to obtain the smoothed value of the aircraft approach distance. If the steady-state locking condition is triggered, the system enters the locked state and outputs the aircraft approach distance lock value. .

[0244] In some embodiments, the aircraft approach distance is post-filtered to obtain a smoothed value corresponding to the aircraft approach distance. If the steady-state locking condition is met, the average of the smoothed aircraft approach distance values ​​corresponding to the nearest preset number of docking images in the current frame is taken as the target aircraft approach distance. If the steady-state locking condition is not met, the smoothed aircraft approach distance value is taken as the target aircraft approach distance.

[0245] Herein, the steady-state locking condition includes: the operation duration of the operation vehicle reaches the operation duration threshold, and the smooth value of the aircraft approaching distance of a plurality of consecutive docking images is less than the aircraft approaching distance threshold.

[0246] Exemplarily, the post-filtering processing includes at least one of exponential decay weighted sliding window filtering processing, exponential moving average filtering secondary low-pass processing, forced monotonic decrease constraint, and abnormal jump suppression processing.

[0247] (1) The exponential decay weighted sliding window filtering processing may be implemented in the following manner.

[0248] Let W be the upper limit of the sliding window length (a typical value is 15 frames), be the actual available number of frames at the current moment (that is, at the initial start-up stage when t<W, only the existing frames are weighted, and the calculation is performed according to the actual number of frames if the window length is not reached). For the last N frames The formed window is assigned an exponentially decaying weight: .

[0249] Herein, k is the attenuation coefficient (a typical value of k is 2.0), i=0 corresponds to the current frame, and i=N-1 corresponds to the earliest frame, so that recent frames have larger weights and earlier frames have smaller weights.

[0250] Perform weighted averaging on the frames in the window to obtain a weighted mean value: .

[0251] (2) The EMA secondary low-pass processing may be implemented in the following manner.

[0252] On the basis of weighted averaging, exponential moving average is used to generate a candidate smooth value : .

[0253] Herein, is the smoothing coefficient (typically 0.3), is the smooth distance output by the previous frame (the initial frame takes ).

[0254] (3) The forced monotonic decrease constraint and abnormal jump suppression processing may be implemented in the following manner.

[0255] Let be the historical minimum smooth distance. This sub-process handles two types of anomalies at the same time: upward jump (violating the monotonic decrease rule) and downward cliff (the ranging value drops abnormally at the moment of occlusion), and updates the smooth value of the aircraft approaching distance according to the following rules : Case 1 (normal approaching): If and satisfy ( If the maximum allowable inter-frame drop is 5cm, then normal update will proceed. .

[0256] in, Candidate smoothed values ​​for aircraft approach distance. This is the smoothed value for the aircraft approach distance.

[0257] Case 2 (Upward Jump Inhibition): If (If an upward jump occurs, violating the monotonically decreasing law), then a forced downward slow approach to the historical minimum value is initiated: .

[0258] in, This is the smoothed value of the aircraft approach distance in the current frame t. This is the smoothed value of the aircraft approach distance from the previous frame. It is a small decreasing coefficient. It can be set according to the actual situation, for example, It can be 0.005, the embodiments of this application are for The value of is not limited, ensuring smooth output and no reverse flow.

[0259] Scenario 3 (Downward cliff suppression): If (i.e., the drop within one frame exceeds) The prompt indicates that momentary occlusion / misidentification caused the issue. If the distance is too large (which implies an excessively small distance), then the descent range is limited. .

[0260] in, This is the smoothed value of the aircraft approach distance in the current frame t. This represents the candidate smoothed values ​​for the aircraft approach distance in the current frame t. This is the smoothed value of the aircraft approach distance from the previous frame. This is the lower bound protection coefficient. It can be set according to the actual situation, for example, It can be 0.8, the embodiments of this application are for The value of is not limited. The meaning is: within a frame The distance is not allowed to fall below (1-β) = 20% of the historical observed distance, thereby suppressing the precipitous drop in the distance measurement value to an unreasonable minimum value caused by instantaneous occlusion.

[0261] When the working time of the work vehicle reaches the working time threshold And the smoothed value of the aircraft approach distance in multiple consecutive docking images Less than the aircraft approach distance threshold In the case that the steady-state locking condition is met, the nearest M-frame will be selected. The average value is used as the aircraft approach distance lock value. : .

[0262] in, The mean function is used to calculate the arithmetic mean of the array elements, representing the aircraft's approach distance lock value. This refers to the smoothed aircraft approach distance value output in the previous M frames. This refers to the smoothed aircraft approach distance value output in the current frame t. The value of M can be set according to the actual situation. For example, M can be 10 frames. This application embodiment does not limit the value of M.

[0263] In some embodiments, The value can be set according to the actual situation, for example, It can be 10 centimeters. This application embodiment... The value of is not limited. The value can be set according to the actual situation, for example, It can be 18.5 seconds, according to the embodiments of this application. The value of is not limited. The system outputs after locking. This will remain unchanged until the current docking operation is completed.

[0264] Understandably, the weight design of the exponential decay weighting is based on the prior that "the recent value of the signal is more valuable than the long-term value"; the EMA second-order low-pass filtering is a widely validated low-pass method in linear time-invariant systems; the monotonically decreasing forced constraint strictly conforms to the physical reality of docking operations (the distance will not increase on its own during docking); the downward cliff suppression utilizes the physical fact that "the distance change between frames in real mechanical motion is constrained by speed and will not have instantaneous large jumps" to effectively filter out outliers caused by instantaneous occlusion; steady-state locking avoids ranging fluctuations caused by drastic deformation in the pixel domain during the extreme bonding stage.

[0265] The technical solution provided in this application embodiment utilizes the physical law of "monotonically decreasing distance" in continuous video streams to effectively suppress instantaneous ranging noise, sudden jumps, and backwards, and maintains steady-state locked output during the extreme bonding stage, providing reliable underlying control signals for graded safety early warning.

[0266] Based on the established physical law that "the distance between the work vehicle and the aircraft inevitably decreases monotonically during the approach operation," this application constructs a multi-level temporal filtering mechanism with five constraints in the temporal dimension: exponential decay weighted sliding window filtering, EMA secondary low-pass filtering, monotonically decreasing forced constraint, lower bound protection, and limit-fit steady-state locking. Compared to general filtering methods in general autonomous driving scenarios (where the distance can increase or decrease and there is no monotonicity), the temporal constraint design of this application is a specialized utilization of the specific physical laws of the approach operation scenario, which can effectively suppress ranging jitter and backtracking caused by anomalies such as instantaneous occlusion and sudden changes in light and shadow.

[0267] In addition, after obtaining the target aircraft's approach distance, it is also possible to provide graded early warnings and visualization of the target aircraft's approach distance.

[0268] In some embodiments, warnings are issued based on the distance to the target aircraft using at least one warning method. The at least one warning method includes: visual color warning, frequency-modulated audio warning, haptic feedback, and network reporting.

[0269] Specifically, the target aircraft's approach distance is sent to the graded early warning and visualization module. Based on the preset distance threshold strategy, graded visual color warnings, frequency conversion audio warnings, tactile feedback, etc. are output, and multimodal visualization information is superimposed on the original image, and finally presented to front-line operators or superior control systems.

[0270] As one possible implementation, when the distance to the target aircraft is greater than or equal to a first threshold, a warning is issued using a first visual color warning.

[0271] The first visual color warning includes displaying the background color of a label indicating the aircraft's approach distance on the display screen in a first color.

[0272] As one possible implementation, when the target aircraft's approach distance is less than a first threshold and greater than or equal to a second threshold, a warning is issued using a second visual color warning and a first frequency-modulated audio warning.

[0273] The second visual color warning includes displaying the aircraft approach distance label background color in a second color on the display screen, and the first frequency-modulated audio warning includes setting the beep frequency between a first frequency and a second frequency (the smaller the aircraft approach distance, the louder the beep frequency). Simultaneously, visual information such as target marker boxes, key points, baselines, and distance measurement values ​​can be overlaid on the display screen.

[0274] The second threshold is less than the first threshold, and the first frequency is less than the second frequency.

[0275] For example, the first threshold can be 50cm, the second threshold can be 20cm, the first frequency can be 1Hz, and the second frequency can be 10Hz.

[0276] The audio signal is generated using a sine wave with a sampling rate of 44100 Hz and a reference frequency of 500 Hz. The sine wave is enclosed in a sin² envelope to avoid popping sounds. .

[0277] Where A is the amplitude, such as A=0.3, and T is the buzzer period.

[0278] As one possible implementation, when the target aircraft is less than the second threshold, a warning is issued through visual color warning, second frequency conversion audio warning, and tactile feedback, and the warning is reported to the network.

[0279] Third-visual color warnings include displaying the aircraft approach distance label in a third color on the display screen. Second-frequency audio warnings include setting the beep frequency to a second frequency. Haptic feedback includes warnings via vibration.

[0280] Specifically, when the target aircraft approaches within the red warning zone or triggers steady-state lock, a tactile feedback signal is simultaneously sent to the steering wheel / seat vibration device, and the event record is reported to the airport's central safety control system via the vehicle network.

[0281] Among them, the first color represents the urgency of the shortening of the target aircraft's approach distance as lower than the second color, and the second color represents the urgency of the shortening of the target aircraft's approach distance as lower than the third color.

[0282] For example, the first color can be green, the second color can be yellow, and the third color can be red.

[0283] In summary, the technical solution provided in this application acquires docking images including the aircraft docking point and the work vehicle docking point, and then extracts images based on these docking images to obtain a first image corresponding to the aircraft docking point and a second image corresponding to the work vehicle docking point. The first image includes only the aircraft portion, and the second image includes only the non-aircraft portion. Based on the first and second images, the aircraft docking point and the work vehicle docking point can be independently identified. Specifically, according to the geometric characteristics of the aircraft docking point, the first image is input into the aircraft docking point recognition model for recognition, obtaining a first recognition result for the aircraft docking point; the second image is input into the work vehicle docking point recognition model for recognition, obtaining a second recognition result for the work vehicle docking point. Recognizing the first and second images separately reduces the recognition search space and improves the recognition efficiency and accuracy of the aircraft docking point and the work vehicle docking point. Finally, geometric distance measurement is performed based on the first and second recognition results to obtain the aircraft approach distance between the aircraft docking point and the work vehicle docking point. Since the first and second recognition results can accurately characterize the positions of the aircraft docking point and the work vehicle docking point. Therefore, the aircraft docking distance calculated based on the first and second identification results can accurately detect the distance between the aircraft docking point and the work vehicle docking point.

[0284] This application integrates component-level target detection, large-model depth prior, and pinhole physical geometry to achieve continuous, smooth, and physically interpretable distance detection throughout the entire approach operation process (from a distance of 5 meters to a close contact of 10 centimeters) in complex apron scenarios. In internal verification based on typical approach operation videos, the technical solution provided by this application significantly reduces ranging errors in both the long-distance (approximately 5 meters) and close contact (approximately 10 centimeters) segments compared to single large-model and purely physical modeling schemes. The temporal trajectory remains smooth and monotonous throughout the approach operation process, without significant jitter or abrupt jumps, fully meeting the requirements of the underlying control signals for graded safety early warning.

[0285] Furthermore, this application adopts a monocular vision solution, which can control the hardware cost per vehicle to within 1,000 yuan, reducing it by more than two orders of magnitude compared to commercial integrated collision avoidance systems (150,000-180,000 yuan / unit) and by more than one order of magnitude compared to lidar solutions (40,000-50,000 yuan / unit). For large airlines or airports with a fleet of tens of thousands of vehicles, this is expected to bring hardware investment savings of hundreds of millions of yuan. At the same time, the data collection reach of the entire fleet established by the monocular vision deployment can be further accumulated into data assets with continuous algorithm iteration capabilities, building a smart apron vision platform with long-term commercial value.

[0286] According to IATA research data, if 75% of ground handling vehicles worldwide transition to enhanced collision avoidance systems, the industry's ground damage costs are expected to be reduced by more than 40%, corresponding to saving the global civil aviation industry approximately US$4 billion annually. This application provides a low-cost, scalable core algorithm foundation that helps drive the achievement of this industry goal, while reducing the mental burden on frontline ground staff and promoting a culture of safety in civil aviation ground operations.

[0287] The overall architecture of this application (two-level hierarchical recognition + component-level dedicated recognition model + dual-channel ranging + multi-level temporal filtering + dynamic focus self-calibration) has the ability to uniformly adapt to various types of vehicles (including but not limited to conveyor belt vehicles, loading and unloading platform vehicles, clean water / sewage vehicles, garbage trucks, refueling trucks, food trucks, tractor vehicles, etc.). By configuring the type of docking part with the aircraft, the vehicle docking part recognition model, the real physical size data entries, the system compensation offset, and the warning / locking threshold, the vehicle type can be expanded, providing a consistent technical foundation for the large-scale deployment of the entire airport vehicle fleet.

[0288] Figure 5 This is a schematic diagram of another aircraft approach distance detection process provided in an embodiment of this application, as shown below. Figure 5 As shown, the method may include the following steps: S501: Acquire docking images through an image acquisition device and preprocess the docking images.

[0289] For a description of this step, please refer to S201. It will not be elaborated further here.

[0290] S502: The docking image is split into a first image corresponding to the docking part of the aircraft and a second image corresponding to the docking part of the work vehicle.

[0291] For a description of this step, please refer to S202; it will not be elaborated upon here.

[0292] S503: Input the first image into the aircraft docking part recognition model for recognition to obtain the first recognition result; input the second image into the work vehicle docking part recognition model for recognition to obtain the second recognition result.

[0293] For a description of this step, please refer to S203 and S204. It will not be elaborated further here.

[0294] S504: Determine whether the current frame t is less than or equal to the preset frame number.

[0295] If the current frame t is less than or equal to the preset frame number, then execute S505; otherwise, execute S507.

[0296] S505: The focal length of the image acquisition device is calibrated to obtain the target focal length.

[0297] S506: Write the target focal length to the calibration parameter storage unit.

[0298] S507: Determine the aircraft approach distance based on the first identification result, the second identification result, and the target focal length.

[0299] S508: Perform post-filtering on the aircraft approach distance to obtain a smoothed value of the aircraft approach distance.

[0300] S509: Determine whether the steady-state locking condition is met.

[0301] If the steady-state locking condition is met, execute S511; otherwise, execute S510.

[0302] S510: Use the smoothed value of the aircraft approach distance as the target aircraft approach distance.

[0303] S511: Use the average of the smoothed aircraft docking distance values ​​corresponding to the preset number of docking images closest to the current frame as the target aircraft docking distance.

[0304] S512: Provides graded early warning and visualization based on the distance to the target aircraft.

[0305] Figure 6 This is a schematic diagram of a dual-channel ranging method provided in an embodiment of this application, as shown below. Figure 6 As shown, the first image / second image is input into the large model relative depth channel, which outputs a dense relative depth map. Simultaneously, the first image / second image is input into the aircraft docking part recognition model / operation vehicle docking part recognition model for recognition, obtaining a first recognition result / second recognition result. Then, the first recognition result / second recognition result is input into the pinhole physical geometry channel, which outputs the physical dimensions of the aircraft docking part / operation vehicle docking part. Finally, based on the dense relative depth map and the physical dimensions of the aircraft docking part / operation vehicle docking part, the focal length is calculated to obtain the calibrated target focal length.

[0306] The large model relative depth channel contains a monocular depth estimation large model; the pinhole physical geometry channel contains a physical dimension database (such as aircraft manuals and manufacturer parameters). Based on the first recognition result / second recognition result, the physical dimensions of the aircraft docking part / operation vehicle docking part are obtained by querying the physical dimension database.

[0307] After obtaining the dense relative depth map, the depth estimate of the aircraft docking point / operation vehicle docking point can be determined based on the dense relative depth map. After obtaining the depth estimate of the aircraft docking point / operation vehicle docking point, the instantaneous focal length is determined by the pinhole equation based on the pixel size of the aircraft docking point / operation vehicle docking point (the pixel size of the aircraft docking point is included in the first recognition result, and the pixel size of the operation vehicle docking point is included in the second recognition result), physical size, and depth estimate. Then, the target focal length is determined based on the instantaneous focal length.

[0308] For example, the aircraft approach distance detection method provided in this application embodiment can be applied to the following two embodiments.

[0309] Example 1: Distance detection between conveyor belt vehicle and machine.

[0310] Application Scenario: Conveyor belt vehicles are used for loading and unloading cargo on flights. During operation, the upper edge of the conveyor belt at the rear of the vehicle needs to be close to the lower edge of the aircraft cargo door, and then the loading and unloading of cargo is completed by handling personnel or loading robots. Operation Characteristics: The target is clear (aircraft cargo door + upper edge of conveyor belt), the space is compact, and the final stage involves two-stage actions: "vehicle moving closer + conveyor belt lifting".

[0311] System configuration: The image acquisition device is mounted on the roof in front of the conveyor belt vehicle's cab, covering the cargo door and the upper edge of the conveyor belt; the aircraft docking point is selected as the "cargo door" (using a dedicated recognition sub-model of a regular rectangle), and the actual width of the cargo door is queried from the database according to the aircraft type (typical value of 181.5 cm for narrow-body aircraft); the work vehicle docking point is selected as the "upper edge of the conveyor belt" (using a topology of line segments at both ends), and the typical value of the actual width of the conveyor belt is 80.0 cm; the camera pitch angle is approximately 15°; system-level calibration offset compensation amount. Typical value is 37 cm.

[0312] The specific processing flow of this embodiment may include the following steps: S301: Acquire the docking image of the current frame, input the docking image into the fuselage segmentation module to obtain the aircraft mask, and generate the aircraft sub-image and non-aircraft sub-image.

[0313] For a description of this step, please refer to S201 and S202. It will not be elaborated further here.

[0314] S302: Input the aircraft sub-image into the rectangular sub-recognition model and output the four corner points of the cargo door (numbered clockwise from 0 to 3); input the non-aircraft sub-image into the conveyor belt recognition model and output the two ends of the upper edge of the conveyor belt and the four corner points of the conveyor belt.

[0315] For a description of this step, please refer to S203 and S204. It will not be elaborated further here.

[0316] S303: For the first 30 frames, call Depth Anything V3 to output a relative depth map. Substitute the relative depths of the cargo door and conveyor belt areas into the inverse calculation formula and obtain the target focal length by performing a robust mean value calculation. (Typically about 2.05 mm).

[0317] For a description of this step, please refer to section S205 regarding the determination of the target focal length of the image acquisition device; further details will not be provided here.

[0318] S304: The length of the line connecting point 2 and point 3 is the pixel width of the lower edge of the cargo door. Calculate the first absolute physical depth The distance between the two ends of the conveyor belt is... Calculate the second absolute physical depth After world coordinate transformation and Compensation for aircraft landing distance .

[0319] From the pinhole equation It can be seen that when the work vehicle is extremely close to the aircraft (e.g., less than 30 centimeters), the number of pixels occupied by the cargo door width in the image is... This would exceed the image frame width limit (1280 pixels), meaning the cargo door could not be fully imaged in a single frame. Therefore, the "full width of the cargo door's lower edge" could not continue to be used as the ranging reference. For this reason, this embodiment... Automatic distance measurement reference switching: The distance is changed to the pixel distance between the midpoint of the line connecting key point 2 (lower right corner) and key point 3 (lower left corner) along the lower edge of the cargo door, and the midpoint of the line connecting the two endpoints along the upper edge of the conveyor belt. Alternative observations and adjustments accordingly. The actual width of the conveyor belt is minus the docking offset. This reference switching strategy ensures the continuity of ranging during the extreme bonding phase.

[0320] S305: Aircraft approach distance Multi-stage time-series filtering is performed to obtain the smoothed aircraft approach distance value in the current frame t. When the cumulative working time of the work vehicle reaches 18.5 seconds and the aircraft approach distance smoothing value is reached... Steady-state locking is triggered when the temperature remains below 10 cm.

[0321] For an introduction to this step, please refer to the description of post-filtering processing of aircraft approach distance in S205; it will not be elaborated further here.

[0322] S306: Outputs tiered early warnings and multimodal visualizations.

[0323] For a detailed explanation of this step, please refer to the section on graded early warning in S205; further details will not be provided here.

[0324] In a typical A320+ conveyor belt vehicle approach operation video (approximately 20-30 seconds), the system continuously outputs a smooth, monotonous distance curve that decreases from an initial distance of approximately 500 cm to a maximum contact distance of approximately 10 cm; graded color and variable frequency audio automatically switch according to thresholds; the final locked value is stable and unaffected by momentary occlusion interference.

[0325] Example 2: Distance detection of tank trucks (clean water trucks / sewage trucks) approaching the machine.

[0326] Application scenarios: Water trucks and sewage trucks complete resupply operations via flexible or rigid pipes extending to the aircraft service port. Features: The aircraft docking point is a small-sized circular / semi-elliptical service port (10-20 cm in diameter); the work vehicle docking point is the end of the service arm at the rear of the tanker truck; high precision is required for docking space (usually less than 5 cm).

[0327] The specific processing flow of this embodiment can be executed according to the steps of Embodiment 1. However, there are the following differences between this embodiment and Embodiment 1: (1) A small-sized circular / semi-elliptical sub-model is used for the aircraft docking area. After the target detection network outputs a coarse bounding box, the ellipse parameter regression branch directly outputs the center of the ellipse. Major and minor half-shafts Rotation angle θ; with the center of the ellipse as the reference point for the target pixel, the actual physical size. Determine the actual diameter of the service port (refer to the model manual; typically 10-20 cm).

[0328] (2) The docking part of the work vehicle adopts a single endpoint and direction vector topology. The end of the service arm is a relatively sharp point structure. The regression model outputs the coordinates of the end center point and the unit direction vector along the service arm direction, which makes it easy to determine the spatial relative attitude between the end and the service port in the world coordinate system.

[0329] (3) Fisheye distortion compensation. Tanker trucks often use fisheye wide-angle cameras to cover a wider field of view. In this embodiment, a fisheye distortion correction sub-step is added to the preprocessing of the docking images acquired by the image acquisition device, and an improved equidistant projection model is used for distortion correction. The fisheye distortion parameters are also obtained by back-calculation through the prior depth of the large model, without the need for an additional physical calibration plate.

[0330] (4) More stringent steady-state locking threshold. Due to the high accuracy requirements of the service port docking space, the distance threshold for steady-state locking is... When the size is reduced to the 2-5 cm level, the mean window M before locking is correspondingly increased (e.g., 15-20 frames).

[0331] This embodiment can obtain stable service port center positioning and centimeter-level real-time distance signals in scenarios with small service port size and high docking accuracy requirements, assisting operators in completing precise service port docking.

[0332] In summary, the technical solution provided in this application acquires docking images including the aircraft docking point and the work vehicle docking point, and then extracts images based on these docking images to obtain a first image corresponding to the aircraft docking point and a second image corresponding to the work vehicle docking point. The first image includes only the aircraft portion, and the second image includes only the non-aircraft portion. Based on the first and second images, the aircraft docking point and the work vehicle docking point can be independently identified. Specifically, according to the geometric characteristics of the aircraft docking point, the first image is input into the aircraft docking point recognition model for recognition, obtaining a first recognition result for the aircraft docking point; the second image is input into the work vehicle docking point recognition model for recognition, obtaining a second recognition result for the work vehicle docking point. Recognizing the first and second images separately reduces the recognition search space and improves the recognition efficiency and accuracy of the aircraft docking point and the work vehicle docking point. Finally, geometric distance measurement is performed based on the first and second recognition results to obtain the aircraft approach distance between the aircraft docking point and the work vehicle docking point. Since the first and second recognition results can accurately characterize the positions of the aircraft docking point and the work vehicle docking point. Therefore, the aircraft docking distance calculated based on the first and second identification results can accurately detect the distance between the aircraft docking point and the work vehicle docking point.

[0333] Figure 7 This is a schematic diagram of the structure of an aircraft approach distance detection device provided in an embodiment of this application, as shown below. Figure 7 As shown, the aircraft approach distance detection device 700 may include an acquisition module 701, a processing module 702, and an identification module 703.

[0334] The acquisition module 701 is used to perform... Figure 2 The illustrated method involves the operation of S201; the processing module 702 is used to execute... Figure 2 The illustrated method includes operations S202 and S205; the identification module 703 is used to execute... Figure 2 The illustrated method includes operations S203 and S204.

[0335] In some embodiments, the aircraft approach distance detection device 700 includes hardware structures and / or software modules corresponding to the execution of each function in order to achieve the above-described functions. Those skilled in the art will readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0336] This application embodiment can divide the aircraft approach distance detection device 700 into functional modules according to the above method embodiment. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one aircraft approach distance detection module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0337] like Figure 8 As shown, the electronic device provided in this application embodiment may include a processor 801, a bus 802, a communication interface 803, and a memory 804. The processor 801, the memory 804, and the communication interface 803 communicate with each other via the bus 802. It should be understood that this application does not limit the number of processors and memories in the network device.

[0338] The 802 bus can be a PCI bus, an Extended Industry Standard Architecture (EISA) bus, or a UB bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus 802 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 802 may include a path for transmitting information between various components of the network device (e.g., memory 804, processor 801, communication interface 803).

[0339] Processor 801 may include any one or more processors such as CPU, graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0340] Memory 804 may include volatile memory, such as random access memory (RAM). Memory 804 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0341] The communication interface 803 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between network devices and other devices or communication networks.

[0342] The memory 804 stores executable program code, and the processor 801 executes the executable program code to implement the functions of the aforementioned method embodiments. That is, the memory 804 stores instructions for executing the above-described aircraft approach distance detection method.

[0343] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the aircraft approach distance detection method provided in the above-described method embodiments.

[0344] This application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the aircraft approach distance detection method provided in the above-described method embodiments.

[0345] Through the above description of the implementation methods, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the module can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, modules, and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0346] Since the aircraft approach distance detection module, computer-readable storage medium, and computer program product in the embodiments of the present invention can be applied to the above method, the technical effects obtained can also be referred to the above method embodiments, and the embodiments of the present invention will not be repeated here.

[0347] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a network device. Of course, the processor and storage medium can also exist as discrete components in the network device.

[0348] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer programs or instructions. When a computer program or instruction is loaded and executed on a computer, the processes or functions of the embodiments of this application are performed, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable module. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, a computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0349] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting aircraft approach distance, characterized in that, The method includes: The docking image is acquired by an image acquisition device, and the docking image includes at least the docking part of the aircraft and the docking part of the work vehicle. Based on the aircraft docking point and the work vehicle docking point, a first image corresponding to the aircraft docking point and a second image corresponding to the work vehicle docking point are obtained respectively. The first image is input into the aircraft docking part recognition model for recognition, and the first recognition result of the aircraft docking part is obtained; the aircraft docking part recognition model is a recognition model constructed based on the geometric characteristics of the aircraft docking part; The second image is input into the identification model of the docking part of the work vehicle for identification, and a second identification result of the docking part of the work vehicle is obtained. Based on the first and second identification results, geometric distance measurement is performed to obtain the aircraft docking distance; the aircraft docking distance is used to describe the distance between the aircraft docking part and the work vehicle docking part.

2. The method according to claim 1, characterized in that, The aircraft includes multiple aircraft docking parts with different geometric characteristics, and the aircraft docking part identification model includes multiple sub-identification models corresponding to the geometric characteristics of each aircraft docking part. The step of inputting the first image into the aircraft docking part recognition model for recognition, and obtaining the first recognition result of the aircraft docking part, includes: Determine the target geometric characteristics of the aircraft docking area in the first image; The first image is input into a sub-recognition model corresponding to the geometric characteristics of the target for recognition, and a first recognition result of the aircraft docking part of the first image is obtained.

3. The method according to claim 2, characterized in that, The geometric characteristics of the aircraft docking parts include at least rectangular, elliptical, curved surface, and anchor point types; the multiple sub-recognition models include a rectangular sub-recognition model corresponding to the rectangular type, an elliptical sub-recognition model corresponding to the elliptical type, a curved surface sub-recognition model corresponding to the curved surface type, and an anchor point sub-recognition model corresponding to the anchor point type.

4. The method according to any one of claims 1 to 3, characterized in that, The step of acquiring a first image corresponding to the aircraft docking point and a second image corresponding to the work vehicle docking point, based on the aircraft docking point and the work vehicle docking point, respectively, includes: Based on the docking area of ​​the aircraft in the docking image, an aircraft mask is obtained; the resolution of the aircraft mask matches the resolution of the docking image. Based on the aircraft mask, a first image corresponding to the docking part of the aircraft and a second image corresponding to the docking part of the work vehicle are acquired.

5. The method according to claim 1, characterized in that, Based on the first recognition result, the pixel size and physical size of the aircraft docking part are obtained; based on the second recognition result, the pixel size and physical size of the work vehicle docking part are obtained. The step of performing geometric ranging based on the first identification result and the second identification result to obtain the aircraft approach distance includes: Based on the first recognition result, the second recognition result, and the target focal length of the image acquisition device, the first absolute physical depth between the aircraft docking part and the optical center of the image acquisition device and the second absolute physical depth between the work vehicle docking part and the optical center of the image acquisition device are obtained respectively. Geometric ranging is performed based on the first absolute physical depth and the second absolute physical depth to obtain the aircraft's approach distance.

6. The method according to claim 5, characterized in that, The step of obtaining the first absolute physical depth between the aircraft docking point and the optical center of the image acquisition device and the second absolute physical depth between the work vehicle docking point and the optical center of the image acquisition device based on the first recognition result, the second recognition result, and the target focal length of the image acquisition device includes: Based on the pixel size of the aircraft docking part, the physical size of the aircraft docking part, and the target focal length, the first absolute physical depth is determined by the pinhole equation. Based on the pixel size of the docking part of the work vehicle, the physical size of the docking part of the work vehicle, and the target focal length, the second absolute physical depth is determined by the pinhole equation.

7. The method according to claim 6, characterized in that, The pinhole equation is expressed as: in, The first absolute physical depth, The target focal length, The physical dimensions of the aircraft docking area are given. The pixel size of the aircraft docking area. This refers to the physical pixel spacing of the sensor.

8. The method according to claim 5, characterized in that, The step of obtaining the aircraft approach distance by performing geometric ranging based on the first absolute physical depth and the second absolute physical depth includes: Based on the first absolute physical depth, the three-dimensional coordinates of the pixel center of the aircraft docking part in the camera coordinate system are determined by the back projection formula. The three-dimensional coordinates of the pixel center of the aircraft docking part in the camera coordinate system are mapped to the world coordinate system to obtain the first coordinates; Based on the second absolute physical depth, the three-dimensional coordinates of the pixel center of the docking part of the work vehicle in the camera coordinate system are determined by the back projection formula; The three-dimensional coordinates of the pixel center at the docking point of the work vehicle in the camera coordinate system are mapped to the world coordinate system to obtain the second coordinates; Calculate the Euclidean distance between the first coordinate and the second coordinate; The Euclidean distance is corrected for offset to obtain the aircraft approach distance.

9. The method according to claim 8, characterized in that, The back projection formula is expressed as follows: in, For the camera coordinate system, The coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the X-axis in the camera coordinate system. The coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the Y-axis in the camera coordinate system are given. The coordinates of the pixel center of the aircraft docking point or the work vehicle docking point along the Z-axis in the camera coordinate system are given. The coordinates of the pixel center of the aircraft docking point or the work vehicle docking point on the first image or the second image along the x-axis. The coordinates of the pixel center of the aircraft docking point or the work vehicle docking point on the first image or the second image along the y-axis. The physical pixel pitch of the sensor. The target focal length, The coordinates of the principal point along the x-axis in the first or second image. The coordinates of the principal point along the y-axis on the first or second image. It is either the first absolute physical depth or the second absolute physical depth.

10. The method according to claim 5, characterized in that, The first identification result includes the pixel size, category, and aircraft model of the docking part of the aircraft; The method further includes: Based on the category of the aircraft docking part and the aircraft model to which it belongs, the physical dimensions corresponding to the aircraft docking part are queried from the physical dimension database; Based on the physical dimensions and pixel dimensions of the aircraft docking area, focal length calibration is performed to obtain the target focal length of the image acquisition device.

11. The method according to claim 10, characterized in that, The step of performing focal length calibration based on the physical dimensions and pixel dimensions corresponding to the aircraft docking area to obtain the target focal length of the image acquisition device includes: The docking image is processed by a monocular depth estimation model to obtain the depth estimate of the docking part of the aircraft. Based on the pixel size, physical size, and depth estimate of the aircraft docking area, the instantaneous focal length of the image acquisition device is obtained. Based on the instantaneous focal lengths of multiple docking images, an instantaneous focal length sequence is generated; Robust averaging is performed on the instantaneous focal length sequence to determine the focal length of the image acquisition device.

12. The method according to claim 1, characterized in that, The method further includes: The aircraft approach distance is post-filtered to obtain a smoothed value of the aircraft approach distance corresponding to the aircraft approach distance. Under the condition of steady-state locking, the average of the smoothed aircraft approach distance values ​​corresponding to the preset number of docking images closest to the current frame is taken as the target aircraft approach distance. If the steady-state locking condition is not met, the smoothed value of the aircraft approach distance is taken as the target aircraft approach distance; the steady-state locking condition includes: the operation time of the work vehicle reaches the operation time threshold, and the smoothed value of the aircraft approach distance of multiple consecutive docking images is less than the aircraft approach distance threshold.

13. The method according to claim 12, characterized in that, The post-filtering process includes at least one of the following: exponential decay weighted sliding window filtering, exponential moving average filtering with secondary low-pass processing, monotonically decreasing forced constraint, and abnormal jump suppression processing.

14. The method according to claim 12, characterized in that, The method further includes: Based on the distance to the target aircraft, an early warning is issued through at least one warning method; the at least one warning method includes: visual color warning, frequency conversion audio warning, tactile feedback, and network reporting.

15. The method according to claim 14, characterized in that, The provision of early warning based on the target aircraft's approach distance, using at least one warning method, includes: When the distance to the target aircraft is greater than or equal to a first threshold, a first visual color warning is issued; the first visual color warning includes displaying the background color of a label indicating the distance to the target aircraft in a first color on the display screen; When the target aircraft's approach distance is less than the first threshold and greater than or equal to the second threshold, a warning is issued using a second visual color warning and a first frequency-modulated audio warning. The second visual color warning includes displaying the background color of the target aircraft's approach distance label in the display screen in a second color, and the first frequency-modulated audio warning includes setting the buzzer frequency to between a first frequency and a second frequency. The second threshold is less than the first threshold, and the first frequency is less than the second frequency. When the smoothed value of the target aircraft approach distance is less than the second threshold, a warning is issued through a third visual color warning, a second variable frequency audio warning, and the tactile feedback, and the warning is reported to the network. The third visual color warning includes displaying the background color of the target aircraft approach distance label in a third color on the display screen. The second variable frequency audio warning includes setting the beep frequency to the second frequency. The tactile feedback includes issuing a warning through vibration. The first color indicates a lower degree of urgency in the shortening of the target aircraft's approach distance than the second color indicates a lower degree of urgency in the shortening of the target aircraft's approach distance; the second color indicates a lower degree of urgency in the shortening of the target aircraft's approach distance than the third color indicates a lower degree of urgency in the shortening of the target aircraft's approach distance.

16. An aircraft approach distance detection device, characterized in that, The device includes: The acquisition module is used to acquire docking images through an image acquisition device, wherein the docking images include at least the docking part of the aircraft and the docking part of the work vehicle. The processing module is used to acquire, based on the aircraft docking part and the work vehicle docking part, a first image corresponding to the aircraft docking part and a second image corresponding to the work vehicle docking part, respectively. The recognition module is used to input the first image into the aircraft docking part recognition model for recognition, and obtain the first recognition result of the aircraft docking part; the aircraft docking part recognition model is a recognition model constructed based on the geometric characteristics of the aircraft docking part. The recognition module is also used to input the second image into the recognition model of the docking part of the work vehicle for recognition, and obtain the second recognition result of the docking part of the work vehicle; The processing module is also used to perform geometric ranging based on the first identification result and the second identification result to obtain the aircraft docking distance; the aircraft docking distance is used to describe the distance between the aircraft docking part and the work vehicle docking part.

17. An electronic device, characterized in that, The electronic device includes: a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the aircraft approach distance detection method as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the aircraft approach distance detection method as described in any one of claims 1 to 15.

19. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed by a processor, implement the aircraft approach distance detection method as described in any one of claims 1 to 15.

Images