Recognition and tracking camera platform, recognition and tracking method and device for aircraft landing gear

Abstract

The invention provides a recognition and tracking camera platform, a recognition and tracking method and recognition and tracking equipment for an aircraft landing gear, and is applied to the technical field of image processing. Comprising the steps that an undercarriage visual probe drives a visible light sensor or a night vision sensor to collect a first video picture according to luminous flux in a to-be-recognized airport area; the undercarriage video gateway performs format conversion on the first video picture to obtain a second video picture; the undercarriage identification tracker generates a switching instruction or switching information when identifying the undercarriage in the second video picture, wherein the switching information is used for prompting a user to manually switch the high-frame-rate sensor; the undercarriage visual probe responds to the switching instruction and is automatically switched to a high-frame-rate sensor so as to acquire a third video picture; the undercarriage video gateway performs format conversion on the third video picture to obtain a fourth video picture; and the undercarriage identification tracker tracks and marks the aircraft undercarriage in the fourth video picture to obtain a target video picture. And the complexity and the hardware cost of a camera platform are effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an aircraft landing gear recognition and tracking camera platform, a recognition and tracking method and equipment. BACKGROUND

[0002] In the flight test process, obtaining the dynamic images of the landing gear in the take-off and landing stages, the landing gear extension and retraction state, the locking condition and the tire in the high-speed sliding process is a key link to verify the reliability of the camera platform design and analyze the dynamic characteristics. These video pictures provide indispensable visual data for designers, and are also an important basis for ground command personnel to judge the aircraft state in real time and assist flight personnel to complete high-risk subject decision-making.

[0003] In order to capture these short and high-speed dynamic processes, conventional cameras are often used to shoot targets with slow motion, high-frame-rate cameras are relied on to analyze fast actions, or infrared thermal imagers are used to image under complex light conditions in flight test engineering. At present, according to different flight test requirements, there are various specifications of cameras to choose from, such as visible light cameras, high-resolution cameras, high-frame-rate cameras, infrared cameras, etc.

[0004] However, each type of camera needs to be matched with a special on-board acquisition and transmission device (hereinafter referred to as a backend device) on the backend. Once the camera model is changed, the backend device also needs to be replaced. This "one camera-one backend" adaptation method significantly increases the complexity and hardware cost of the camera platform, bringing great inconvenience to the operation and maintenance of researchers. SUMMARY

[0005] The embodiments of the present application provide an aircraft landing gear recognition and tracking camera platform, a recognition and tracking method and equipment, which solve the defects that the prior art significantly increases the complexity and hardware cost of the camera platform, brings great inconvenience to the operation and maintenance of researchers, and realize that no matter how the sensor in the landing gear visual probe 101 is switched, the backend device also does not need to be replaced. This "multi-camera-one-backend" adaptation method makes multiple sensors have higher adaptability and compatibility to the backend device, improves the universality of the equipment, effectively reduces the complexity and hardware cost of the recognition and tracking camera platform, and brings convenience to the operation and maintenance of researchers.

[0006] The embodiment of the application provides an aircraft landing gear identification and tracking camera platform, comprising: a landing gear visual probe, a landing gear video gateway and a landing gear identification and tracker, the landing gear visual probe is connected with the landing gear video gateway, the landing gear video gateway is connected with the landing gear identification and tracker; the landing gear visual probe comprises a visible light sensor, a night vision sensor and a high frame rate sensor, the frame rate of the high frame rate sensor is higher than that of the visible light sensor and the night vision sensor; The landing gear visual probe is used for driving the visible light sensor or the night vision sensor to collect a first video picture in a first format according to the light flux in the airport area to be identified, and sending the first video picture to the landing gear video gateway through a first interface. The landing gear video gateway is used for performing format conversion on the first video picture to obtain a second video picture in a second format, and sending the second video picture to the landing gear identification and tracker through a second interface. The landing gear identification and tracker is used for identifying the second video picture, generating a first switching instruction or first switching information in the case of identifying the aircraft landing gear, the first switching information is used for prompting a user to manually switch the visible light sensor or the night vision sensor to the high frame rate sensor, and sending the first switching instruction to the landing gear visual probe through the landing gear video gateway. The landing gear visual probe is further used for automatically switching the visible light sensor or the night vision sensor to the high frame rate sensor in response to the first switching instruction, controlling the high frame rate sensor to collect a third video picture in the first format, and sending the third video picture to the landing gear video gateway through the first interface, the third video picture being a next frame video picture of the first video picture. The landing gear video gateway is further used for performing format conversion on the third video picture to obtain a fourth video picture in the second format, and sending the fourth video picture to the landing gear identification and tracker through the second interface. The landing gear identification and tracker is further used for tracking and marking the aircraft landing gear in the fourth video picture to obtain a target video picture.

[0007] The application further provides an aircraft landing gear identification and tracking method, applied to the aircraft landing gear identification and tracking camera platform, the aircraft landing gear identification and tracking camera platform comprising: a landing gear visual probe, a landing gear video gateway and a landing gear identification and tracker, the landing gear visual probe being connected with the landing gear video gateway, the landing gear video gateway being connected with the landing gear identification and tracker; the landing gear visual probe comprising a visible light sensor, a night vision sensor and a high frame rate sensor, the frame rate of the high frame rate sensor being higher than that of the visible light sensor and the night vision sensor; the method comprising: driving the visible light sensor or the night vision sensor to collect a first video picture in a first format through the landing gear visual probe according to the light flux in the airport region to be identified, and sending the first video picture to the landing gear video gateway through a first interface; format-converting the first video picture through the landing gear video gateway to obtain a second video picture in a second format, and sending the second video picture to the landing gear identification and tracker through a second interface; identifying the second video picture through the landing gear identification and tracker, generating a first switching instruction or first switching information in the case of identifying the aircraft landing gear, the first switching information being used to prompt a user to manually switch the visible light sensor or the night vision sensor to the high frame rate sensor, and sending the first switching instruction to the landing gear visual probe through the landing gear video gateway; automatically switching the visible light sensor or the night vision sensor to the high frame rate sensor through the landing gear visual probe in response to the first switching instruction, controlling the high frame rate sensor to collect a third video picture in the first format, and sending the third video picture to the landing gear video gateway through the first interface, the third video picture being a next frame video picture of the first video picture; format-converting the third video picture through the landing gear video gateway to obtain a fourth video picture in the second format, and sending the fourth video picture to the landing gear identification and tracker through the second interface; tracking and labeling the aircraft landing gear in the fourth video picture through the landing gear identification and tracker to obtain a target video picture.

[0008] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, the processor implementing the aircraft landing gear identification and tracking method as described above when executing the computer program.

[0009] The embodiment of the present application further provides a non-transitory computer readable storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to implement the aircraft landing gear identification and tracking method.

[0010] The embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the aircraft landing gear identification and tracking method.

[0011] The aircraft landing gear identification and tracking camera platform, the identification and tracking method and the equipment provided by the embodiment of the present application can drive the visible light sensor or the night vision sensor to collect the first video picture in the first format according to the light flux in the airport area to be identified through the landing gear visual probe, and send the first video picture to the landing gear video gateway through the first interface. The whole process can automatically select the optimal sensor according to the environmental light condition (i.e. the light flux), and ensure that the first video picture is available at any time and under any weather condition, thereby ensuring the full-time working ability of the identification and tracking camera platform.

[0012] After receiving the first video picture sent by the first interface, the landing gear video gateway can perform format conversion on the first video picture to obtain the second video picture in the second format, and send the second video picture to the landing gear identification and tracker through the second interface. The whole process provides a unified and standardized data format for the landing gear identification and tracker, simplifies the design of the identification and tracking algorithm, and makes the identification and tracking algorithm not need to contain complex multi-format analysis code, but only need to process one fixed format.

[0013] After the landing gear identification tracker receives the second video picture sent by the second interface, the second video picture can be identified. If the landing gear of the airplane is identified, it means that the airplane is in the take-off or landing stage. At this time, the sensor switching decision can be executed. Specifically, the first switching instruction or the first switching information is generated. The first switching information is used to prompt the user to manually switch the visible light sensor or the night vision sensor to the high frame rate sensor to realize manual switching. The first switching instruction is sent to the landing gear visual probe through the landing gear video gateway. After the landing gear visual probe receives the first switching instruction sent by the landing gear video gateway, the landing gear visual probe can respond to the first switching instruction and automatically switch the visible light sensor or the night vision sensor to the high frame rate sensor to realize automatic triggering of the device. Then, the high frame rate sensor is controlled to collect a third video picture of a first format, and the third video picture is sent to the landing gear video gateway through the first interface. The third video picture is the next frame of video picture of the first video picture. The whole process realizes the intelligent process of "reconnaissance-confirmation-fine tracking". In a wide scene, the conventional sensor (such as the visible light sensor or the night vision sensor) is used for "reconnaissance". Once the target (i.e. the landing gear of the airplane) is found, the special high-performance resource (such as the high frame rate sensor) is called to perform "fine tracking", so as to realize the optimal allocation of computing and energy consumption resources. In addition, the whole process also realizes the sensor switching decision, provides two modes of "automatic instruction" and "manual information", and in most cases, the camera platform automatically runs, but in maintenance, debugging or special scenes, the staff can manually intervene according to the prompt information, so as to balance the efficiency and flexibility. In addition, in the process of executing the sensor switching decision, the high frame rate sensor is controlled to shoot when the landing gear of the airplane is down, and the ordinary sensor (i.e. the visible light sensor or the night vision sensor) is controlled to shoot when the landing gear of the airplane is retracted, so as to effectively reduce the invalid data record and improve the use efficiency.

[0014] After the landing gear video gateway receives the third video picture sent by the first interface, the landing gear video gateway can perform format conversion on the third video picture to obtain a fourth video picture of a second format. The fourth video picture is sent to the landing gear identification tracker through the second interface. In the whole process, no matter what kind of sensor and frame rate data is returned from the front end, the landing gear video gateway converts the data into a unified second format, so as to ensure that the data format received by the rear-end identification tracker is consistent and predictable.

[0015] After receiving the fourth video picture sent by the second interface, the landing gear identification tracker can identify the fourth video picture, and track and mark the airplane landing gear in the fourth video picture to obtain a target video picture when the airplane landing gear is identified. Since the landing gear identification tracker is compatible with the backend device (also referred to as an upper computer), the backend device does not need to be replaced accordingly regardless of how the sensor in the landing gear visual probe is switched. This "one backend for multiple machines" adaptation mode makes multiple sensors have higher adaptability and compatibility for the backend device, improves the device versatility, effectively reduces the complexity and hardware cost of the identification and tracking camera platform, and brings convenience to the operation and maintenance of researchers. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0017] Figure 1 is a structural schematic diagram of an airplane landing gear identification and tracking camera platform provided by the embodiments of the present application; Figure 2 is a structural schematic diagram of a landing gear identification tracker provided by the embodiments of the present application; Figure 3 is a structural schematic diagram of an airplane landing gear identification and tracking camera platform provided by the embodiments of the present application; Figure 4 is a connection relationship schematic diagram of a landing gear video gateway, a landing gear identification tracker, a landing gear safety lock and a wheel load power distribution unit provided by the embodiments of the present application; Figure 5 is a structural schematic diagram of a wheel load power distribution unit provided by the embodiments of the present application; Figure 6 is a connection relationship schematic diagram of a landing gear visual probe, a landing gear video gateway, a landing gear identification tracker, a landing gear safety lock and a wheel load power distribution unit provided by the embodiments of the present application; Figure 7 is a process schematic diagram of identifying and tracking a landing gear provided by the embodiments of the present application; Figure 8 is a process schematic diagram of an airplane landing gear identification and tracking method provided by the embodiments of the present application; Figure 9 is a structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0019] In order to better understand the embodiments of the present application, the reasons for identifying the aircraft landing gear or the aircraft tire in the embodiments of the present application are first explained in detail; The aircraft landing gear or the aircraft tire has particularity in flight test. The aircraft landing gear and the aircraft tire are high-speed moving targets in the process of take-off and landing of the aircraft, and are collected into the cabin in the normal flight stage. Therefore, an identification and tracking camera platform is needed to realize simultaneous consideration of normal monitoring and high-speed movement of the landing gear, complete the shooting task in the flight process, and in addition, the identification and tracking camera platform can also be applied to identification, tracking and other functions in different scenes.

[0020] The application scenarios of the identification and tracking camera platform of the aircraft landing gear are described in detail below: In the field of aviation test, especially in the process of new model development, existing model modification or periodic deep repair, it is crucial to verify the function of landing gear system. The design of the identification and tracking camera platform provided in the embodiments of the present application provides a high-precision, automatic, visualized and all-around monitoring and data acquisition solution for this key test scene. The design can be specifically applied to monitoring, identification, tracking and other scenes in the landing gear collection and release stage.

[0021] The identification and tracking camera platform of the aircraft landing gear provided in the embodiments of the present application is described in detail below: Figure 1 is a structural schematic diagram of the identification and tracking camera platform of the aircraft landing gear provided in the embodiments of the present application. As Figure 1 shown, the identification and tracking camera platform comprises a landing gear visual probe 101, a landing gear video gateway 102 and a landing gear identification and tracking device 103, the landing gear visual probe 101 is connected with the landing gear video gateway 102, and the landing gear video gateway 102 is connected with the landing gear identification and tracking device 103; the landing gear visual probe 101 comprises a visible light sensor 1011, a night vision sensor 1012 and a high frame rate sensor 1013, the frame rate of the high frame rate sensor 1013 is higher than that of the visible light sensor 1011 and the night vision sensor 1012; The landing gear visual probe 101 is configured to drive the visible light sensor 1011 or the night vision sensor 1012 to collect a first video picture in a first format according to a light flux in an airport area to be identified, and send the first video picture to the landing gear video gateway 102 through a first interface; The landing gear video gateway 102 is configured to perform format conversion on the first video picture to obtain a second video picture in a second format, and send the second video picture to the landing gear identification tracker 103 through a second interface. The landing gear identification tracker 103 is configured to identify the second video picture, and generate a first switching instruction or first switching information in a case where the landing gear of the airplane is identified, the first switching information being used to prompt a user to manually switch the visible light sensor 1011 or the night vision sensor 1012 to the high-frame-rate sensor 1013, and send the first switching instruction to the landing gear visual probe 101 through the landing gear video gateway 102. The landing gear visual probe 101 is further configured to automatically switch the visible light sensor 1011 or the night vision sensor 1012 to the high-frame-rate sensor 1013 in response to the first switching instruction, control the high-frame-rate sensor 1013 to collect a third video picture in the first format, and send the third video picture to the landing gear video gateway 102 through the first interface, the third video picture being a next frame of the first video picture. The landing gear video gateway 102 is further configured to perform format conversion on the third video picture to obtain a fourth video picture in the second format, and send the fourth video picture to the landing gear identification tracker 103 through the second interface. The landing gear identification tracker 103 is further configured to track and mark the landing gear of the airplane in the fourth video picture to obtain a target video picture.

[0022] The visible light sensor 1011 is an imaging device capable of sensing light in the visible light spectrum range (usually about 380-780 nm wavelength) of the human eye and converting the light into an electronic signal. The visible light sensor 1011 usually works in daylight or well-lit environments to provide images with rich colors and clear details. Exemplarily, the visible light sensor 1011 is an IMX265 sensor, which has the characteristics of high imaging quality and low noise, and is very suitable for machine vision applications that require accurate color and detail capture.

[0023] The night vision sensor 1012 is a special sensor highly sensitive to near-infrared light or short-wave infrared light, which can generate a clear monochrome image in a dim light or almost no light environment by receiving infrared light invisible to the human eye (usually needs to be used with an infrared fill light). Exemplarily, the night vision sensor 1012 is an SC5336P sensor, which has excellent performance in night environment, can effectively penetrate fog and haze, and provides a longer monitoring distance than the visible light sensor 1011. It can be understood that the night vision sensor 1012 is different from the existing night shooting sensor, which needs to be matched with a fill light or airport light to realize shooting of the to-be-identified airport area, thereby increasing the equipment cost, while the night vision sensor 1012 itself can realize shooting of the to-be-identified airport area, thereby effectively reducing the equipment cost.

[0024] The high frame rate sensor 1013 is a sensor capable of capturing continuous images at a very high speed (for example, 100 frames, 200 frames or even higher per second), and the frame rate (i.e., time resolution) of the high frame rate sensor 1013 is higher than that of the visible light sensor 1011 and the night vision sensor 1012, which indicates that the high frame rate sensor 1013 focuses on capturing details of high-speed moving objects and avoiding motion blur. Exemplarily, the high frame rate sensor 1013 is an OV9281 sensor, which can output an image with a resolution of one million pixels at a maximum speed of 120 fps, and is very suitable for machine vision applications with extremely high speed requirements.

[0025] The to-be-identified airport area refers to the airspace and ground area where the aircraft landing gear may appear during the take-off and landing process of the aircraft.

[0026] The light flux refers to the total amount of light energy passing through the area of the to-be-identified airport area per unit time, which is commonly referred to as "light flow" or "light intensity".

[0027] The video frame refers to a single static image (frame) captured by the landing gear visual probe 101.

[0028] The aircraft landing gear refers to a key component system for take-off, landing, taxiing and parking on an aircraft, which usually includes a buffer strut, a wheel, a brake system and a retracting actuator, etc. Optionally, the number of the aircraft landing gear is not limited, that is, the identification and tracking camera platform can simultaneously identify and track one or more aircraft landing gears.

[0029] In the embodiment of the present application, after the landing gear visual probe 101 is powered on and works, according to the light flux in the airport area to be identified, the visible light sensor 1011 or the night vision sensor 1012 is driven to collect a first video picture in a first format; and the first video picture is sent to the landing gear video gateway 102 through the first interface. The whole process can automatically select the optimal sensor according to the environmental light condition (i.e. the light flux), and ensure that the "available" first video picture can be obtained at any time and under any weather condition, thereby ensuring the full-time domain working ability of the identification and tracking camera platform.

[0030] After the landing gear video gateway 102 receives the first video picture sent by the first interface, the first video picture can be format-converted to obtain a second video picture in a second format; and the second video picture is sent to the landing gear identification and tracker 103 through the second interface. The whole process provides a unified and standardized data format for the landing gear identification and tracker 103, simplifies the design of the identification and tracking algorithm, and makes the identification and tracking algorithm not need to contain complex multi-format analysis code, but only need to process one fixed format.

[0031] The landing gear identification tracker 103 can identify the second video picture sent by the second interface after receiving the second video picture, and if the landing gear of the airplane is identified, it indicates that the airplane is in the take-off or landing stage, at which time the sensor switching decision can be executed, specifically, the first switching instruction or the first switching information is generated, which is used to prompt the user to manually switch the visible light sensor 1011 or the night vision sensor 1012 to the high frame rate sensor 1013 to realize manual switching; the first switching instruction is sent to the landing gear visual probe 101 through the landing gear video gateway 102. The landing gear visual probe 101 can automatically switch the visible light sensor 1011 or the night vision sensor 1012 to the high frame rate sensor 1013 in response to the first switching instruction after receiving the first switching instruction sent by the landing gear video gateway 102, to realize automatic triggering of the device; and then control the high frame rate sensor 1013 to collect a third video picture of a first format, and send the third video picture to the landing gear video gateway 102 through the first interface, the third video picture being the next frame of video picture of the first video picture. The whole process realizes the intelligent process of “reconnaissance-confirmation-fine tracking”, uses conventional sensors (such as visible light sensor 1011 or night vision sensor 1012) to “reconnaissance” in a wide scene, and as soon as the target (i.e. the landing gear of the airplane) is found, special high-performance resources (such as high frame rate sensor 1013) are called to “fine tracking”, realizing the optimal allocation of computing and energy consumption resources. In addition, the whole process also realizes the sensor switching decision, providing two modes of “automatic instruction” and “manual information”, in most cases the camera platform automatically runs, but in maintenance, debugging or special scenes, the staff can manually intervene according to the prompt information, balancing efficiency and flexibility. In addition, in the process of executing the sensor switching decision, the high frame rate sensor 1013 is controlled to shoot when the airplane landing gear is down, and the ordinary sensor (i.e. visible light sensor 1011 or night vision sensor 1012) is controlled to shoot when the airplane landing gear is retracted, which can effectively reduce the invalid data record and improve the use efficiency.

[0032] The landing gear video gateway 102 can perform format conversion on the third video picture sent by the first interface after receiving the third video picture, to obtain a fourth video picture of a second format; and send the fourth video picture to the landing gear identification tracker 103 through the second interface. In the whole process, no matter what kind of sensor and frame rate data is returned from the front end, the landing gear video gateway 102 will convert the data into a unified second format, ensuring that the data format received by the rear-end identification tracker 103 is consistent and predictable.

[0033] After receiving the fourth video picture sent by the second interface, the landing gear recognition tracker 103 can recognize the fourth video picture, and in the case of recognizing the landing gear, track and mark the landing gear in the fourth video picture to obtain a target video picture. Since the landing gear recognition tracker 103 is compatible with the backend device, regardless of how the sensor in the landing gear visual probe 101 switches, the backend device also does not need to be replaced accordingly. This "one backend for multiple machines" adaptation mode makes multiple sensors have higher adaptability and compatibility for the backend device, improves the universality of the device, effectively reduces the complexity and hardware cost of the recognition and tracking camera platform, and brings convenience to the operation and maintenance of researchers.

[0034] Optionally, the first format is a Low-Voltage Differential Signaling (LVDS) format.

[0035] Optionally, the first interface is an LVDS interface.

[0036] Optionally, the second format is a YUV format, where Y represents Luma, which contains the gray scale information (details, light and dark contrast) of the video picture; U and V represent chroma, which together describe the color information of the video picture, including hue (what color) and saturation (the degree of color intensity), where U represents the Blue-Luma component, that is, the difference between blue and brightness (B-Y), and V represents the Red-Luma component, that is, the difference between red and brightness (R-Y).

[0037] Optionally, the second interface is a Mobile Industry Processor Interface (MIPI) interface.

[0038] It should be noted that the above visible light sensor 1011, night vision sensor 1012 and high frame rate sensor 1013 each contain an LVDS interface and a serial port, and each serial port is used to adjust related parameters such as exposure, hue, contrast and white balance of the corresponding sensor. That is, any sensor contained in the landing gear visual probe 101 uses the LVDS protocol, that is, transmits the video picture in the LVDS format, and transmits data (i.e. video picture) of the corresponding sensor through the serial port driver.

[0039] Optionally, the landing gear visual probe 101 is powered by the landing gear video gateway 102 through an inter-board connector.

[0040] Optionally, the landing gear visual probe 101 is connected to the landing gear video gateway 102 using a high-speed connector.

[0041] Optionally, the landing gear visual probe 101 has a target tracking function, and can track a target of interest (such as an aircraft landing gear or an aircraft tire) in real time.

[0042] Optionally, the landing gear video gateway 102 is composed of a Field-Programmable Gate Array (FPGA) JFMK50T4 of Fudan Microelectronics and peripheral circuits thereof. Specifically, the FPGA is externally connected with two Double Data Rate 4 (DDR4) memories, each of which has a capacity of 4 GB and can provide sufficient buffer for program running and other functions; the FPGA is provided with a Quad Serial Peripheral Interface Flash (QSPI FLASH) for storing a logic program. It should be noted that the FPGA receives a first format video picture sent by the landing gear visual probe 101, analyzes the first format video picture into a second format video picture, and sends the second format video picture to the landing gear recognition tracker 103 through a MIPI interface, that is, the FPGA serves as a passage for information transmission between the landing gear visual probe 101 and the landing gear recognition tracker 103, and realizes communication with the two devices through a serial port.

[0043] Optionally, the landing gear video gateway 102 is configured to perform format conversion on the first video picture to obtain a second video picture in a second format, and the landing gear video gateway 102 is configured to perform format conversion on the first video picture by using a Real Time Streaming Protocol (RTSP) to obtain the second video picture in the second format.

[0044] In the embodiment of the application, after receiving the first video picture sent by the landing gear visual probe 101, the landing gear video gateway 102 can perform real-time format conversion on the first video picture by using the RTSP to obtain a second video picture in a second format, so that the subsequent streaming media session has very low delay (which can be optimized to the order of milliseconds in general) and meets the high real-time requirement.

[0045] In the embodiment of the application, after receiving the first video picture sent by the landing gear visual probe 101, the landing gear video gateway 102 can perform real-time format conversion on the first video picture by using the RTSP to obtain a second video picture in a second format, so that the subsequent streaming media session has very low delay (which can be optimized to the order of milliseconds in general) and meets the high real-time requirement.

[0046] Optionally, the landing gear identification tracker 103 is composed of a Rockchip RK3588 and its peripheral circuit. Specifically, the landing gear identification tracker 103 is externally connected with one DDR4, and the memory is 4 GB, which can provide sufficient buffer for program running and the like; and the landing gear identification tracker 103 is externally connected with one Embedded MultiMediaCard (EMMC), and the memory is 32 GB, which can be used to store programs, temporary images, Log files and the like. In addition, the landing gear identification tracker 103 is expanded with one RS422 interface by an HT3490 chip, and is expanded with two Gigabit network interfaces by two YT8521 chips, one of which is used as a debugging interface, and the other of which is used as a video output interface.

[0047] Optionally, the landing gear identification tracker 103 has functions of realizing image input of the landing gear video gateway 102, image signal processor (ISP) imaging processing, On-Screen Display (OSD) superposition, video encoding and decoding, and peripheral driving and the like.

[0048] Optionally, the landing gear identification tracker 103 provides a debugging serial port and a network interface, which are used to realize at least one of the following functions: data interaction, message communication, firmware upgrade, Log file reading and the like between the landing gear identification tracker 103 and a back-end device.

[0049] Optionally, the landing gear visual probe 101 is also used to control the high-frame-rate sensor 1013 to collect the third video picture in the first format, which can include that the landing gear visual probe 101 is specifically used to control the high-frame-rate sensor 1013 to collect the third video picture in the first format in cooperation with a light supplementing lamp on the aircraft landing gear.

[0050] The light supplementing lamp is a kind of high-intensity, actively controllable lighting device installed on the aircraft landing gear. The purpose of the light supplementing lamp is not to light the way for the aircraft at night, but to provide additional controllable light source illumination for the ground or the identification tracking camera platform of the aircraft body at a specific time (such as when the aircraft lands under low light conditions), so as to ensure that the high-frame-rate sensor 1013 can obtain clear and reliable video pictures.

[0051] Optionally, the landing gear identification tracker 103 is used to identify the second video picture, which can include that the landing gear identification tracker 103 is specifically used to identify the second video picture according to the trained landing gear feature.

[0052] The landing gear feature is obtained by feature extraction on historical landing gear images based on a feature extraction model. The feature extraction model is trained based on historical landing gear images and labeled landing gear features.

[0053] In the embodiment of the present application, after the landing gear identification tracker 103 receives the second video picture sent by the landing gear video gateway 102, the trained landing gear feature can be obtained, and the second video picture is identified according to the landing gear feature. Specifically, if there is a feature in the second video picture that has a high similarity (also referred to as matching degree) with the landing gear feature, it is determined that the aircraft landing gear is identified, otherwise, it is determined that the aircraft landing gear is not identified, which provides data support for the processing of subsequent video pictures. Wherein, the high similarity refers to that the similarity between the landing gear feature and part of the features in the second video picture is greater than a preset similarity threshold. In addition, through sample training, the identification and tracking of other targets (such as aircraft tires) can also be realized, and multiple different feature targets can be tracked at a time.

[0054] In the embodiment of the present application, the identification and tracking camera platform is no longer passively dependent on the changeable environmental light such as sunlight, moonlight or airport runway light. By actively controlling the fill light, stable and sufficient light source can be provided for shooting the aircraft landing gear under any lighting conditions (including completely dark night, thick fog, smoke and other harsh environments), which ensures that the high frame rate sensor 1013 can always collect video pictures with correct exposure and clear details, greatly improves the reliability and robustness of the identification and tracking camera platform, and truly realizes the "all-weather" working ability of the identification and tracking camera platform.

[0055] In some embodiments, Figure 2 is a structural schematic diagram of the landing gear identification tracker provided in the embodiment of the present application. As Figure 2 shown, the landing gear identification tracker 103 includes an illuminance sensor 1031; The landing gear identification tracker 103 is further configured to generate a second switching instruction or second switching information in a case where the light flux collected by the illuminance sensor 1031 reaches a preset threshold, the second switching information being used to prompt a user to manually switch the night vision sensor 1012 to the visible light sensor 1011; and send the second switching instruction to the landing gear visual probe 101 through the landing gear video gateway 102. The landing gear visual probe 101 is specifically configured to automatically switch the night vision sensor 1012 to the visible light sensor 1011 in response to the second switching instruction; and drive the visible light sensor 1011 to collect the first video picture in the first format. The landing gear identification tracker 103 is further configured to generate a third switching instruction or third switching information in a case where the light flux collected by the illuminance sensor 1031 does not reach the preset threshold, the third switching information being used to prompt a user to manually switch the visible light sensor 1011 to the night vision sensor 1012; and send the third switching instruction to the landing gear visual probe 101 through the landing gear video gateway 102. The landing gear visual probe 101 is specifically configured to automatically switch the visible light sensor 1011 to the night vision sensor 1012 in response to a third switching instruction; and drive the night vision sensor 1012 to collect a first video picture in a first format.

[0056] The light intensity sensor 1031 is an electronic device for measuring the intensity of ambient light, which can continuously or intermittently monitor the brightness of the airport area to be identified, and provide the most direct data basis for the sensor switching decision. For example, the light intensity sensor 1031 is a MAX44009 sensor, which outputs the detected data (i.e., light flux) in real time through an RS485 interface. The RS485 interface is a differential signal standard, which has the advantages of strong anti-interference ability, long transmission distance (up to kilometers), and support for multi-point bus, and is perfectly suitable for the industrial scene of the airport, which has a complex electromagnetic environment and equipment that may be widely distributed, thereby ensuring that the light intensity data can be stably, reliably and remotely transmitted to the landing gear identification tracker 103, and effectively avoiding misjudgment caused by signal interference.

[0057] The preset threshold value is a pre-set illumination value, which is a critical point for determining whether the current environment should be defined as "day" or "night", and provides a judgment basis for the sensor switching decision.

[0058] In the embodiment of the present application, in the case that the light flux collected by the light intensity sensor 1031 does not reach the preset threshold value, the night vision sensor 1012 is driven to monitor and shoot the airport area to be identified. During the process of continuously collecting the light flux by the light intensity sensor 1031, if the current light flux reaches the preset threshold value, it is determined that the light intensity at the current time is sufficient. At this time, the night vision sensor 1012 can be manually switched to the visible light sensor 1011, and the visible light sensor 1011 is driven to collect the first video picture. In the case that the light flux collected by the light intensity sensor 1031 reaches the preset threshold value, the visible light sensor 1011 is driven to monitor and shoot the airport area to be identified. During the process of continuously collecting the light flux by the light intensity sensor 1031, if the current light flux does not reach the preset threshold value, it is determined that the light intensity at the current time is insufficient. At this time, the visible light sensor 1011 can be manually switched to the night vision sensor 1012, and the night vision sensor 1012 is driven to collect the first video picture. Through the judgment of the light flux and the preset threshold value, the whole process can respond to the change of the environmental light intensity of the airport area to be identified in real time, ensure that the landing gear visual probe 101 can be quickly switched to the appropriate sensor under any light intensity mutation, continuously provide high-quality and identifiable video pictures, and avoid the situation that the monitoring is "blind" due to the darkening of light or the night vision image is overexposed due to the brightening of light.

[0059] In addition, different light fluxes correspond to different sensors, so that each sensor can operate under the best working condition, not only the optimal video picture quality is obtained, but also the overload work of the sensor under the inappropriate condition is avoided, which helps to prolong the service life of the landing gear visual probe 101.

[0060] In some embodiments, the landing gear identification tracker 103 is also used for identifying the fourth video picture, generating a fourth switching instruction or fourth switching information in the case that the aircraft landing gear is not identified, the fourth switching information being used for prompting the user to manually switch the high frame rate sensor 1013 to the visible light sensor 1011 or the night vision sensor 1012; and the fourth switching instruction is sent to the landing gear visual probe 101 through the landing gear visual gateway 102. The landing gear visual probe 101 is also used for automatically switching the high frame rate sensor 1013 to the visible light sensor 1011 or the night vision sensor 1012 in response to the fourth switching instruction; and the visible light sensor 1011 or the night vision sensor 1012 continues to monitor and shoot the to-be-identified airport area.

[0061] In the embodiment of the present application, after the landing gear identification tracker 103 receives the fourth video picture sent by the second interface, the fourth video picture can be identified, and if the landing gear of the airplane is not identified, it indicates that the landing gear of the airplane has been retracted into the cabin, at this time, the sensor switching decision can be executed, specifically, the fourth switching instruction or the fourth switching information is generated, the fourth switching information is used to prompt the user to manually switch the high frame rate sensor 1013 to the visible light sensor 1011 or the night vision sensor 1012, to realize manual switching; the fourth switching instruction is sent to the landing gear visual probe 101 through the landing gear video gateway 102; after the landing gear visual probe 101 receives the first switching instruction sent by the landing gear video gateway 102, the landing gear visual probe 101 can respond to the fourth switching instruction, and automatically switch the high frame rate sensor 1013 to the visible light sensor 1011 or the night vision sensor 1012, to realize automatic triggering switching of the equipment; and then the visible light sensor 1011 or the night vision sensor 1012 continues to monitor and shoot the to-be-identified airport area. That is, the high frame rate sensor 1013 is the "high-performance mode" of the identification and tracking camera platform, which is used to process high-priority tasks (i.e., tracking the landing gear), and after the high-priority task is completed, the identification and tracking camera platform can intelligently exit the high-performance mode and return to normal monitoring. If the high-priority task is completed, the identification and tracking camera platform still uses the high frame rate sensor 1013, which will only produce a large amount of non-target ultra-high-definition video stream, and actively switches back to the normal monitoring mode, which means that the landing gear identification tracker 103 does not need to perform meaningless identification operation on the high frame rate and high data volume video stream, which can greatly reduce the calculation load of the landing gear identification tracker 103, so that the landing gear identification tracker 103 can be more "focused" on waiting for the appearance of the next real target, thereby improving the response speed of the identification and tracking camera platform.

[0062] In some embodiments, the landing gear visual probe 101 is further configured to automatically switch the high frame rate sensor 1013 to the visible light sensor 1011 or the night vision sensor 1012 in response to the fourth switching instruction, including: the landing gear visual probe 101 is specifically configured to, in a case where the light flux reaches a preset threshold, automatically switch the high frame rate sensor 1013 to the visible light sensor 1011 in response to the fourth switching instruction; and in a case where the light flux does not reach the preset threshold, automatically switch the high frame rate sensor 1013 to the night vision sensor 1012 in response to the fourth switching instruction.

[0063] In the embodiment of the present application, in the process of actively switching from the high-performance mode back to the regular monitoring mode, the tracking camera platform can accurately perform sensor switching decision on the high-frame-rate sensor 1013 according to the judgment of the luminous flux and the preset threshold. Specifically, if the luminous flux reaches the preset threshold, it means that the illumination is sufficient, at which time the high-frame-rate sensor 1013 can be automatically switched to the visible light sensor 1011 to monitor and shoot the airport area to be identified in the daytime. If the luminous flux does not reach the preset threshold, it means that the illumination is insufficient, at which time the high-frame-rate sensor 1013 can be automatically switched to the night vision sensor 1012 to monitor and shoot the airport area to be identified at night.

[0064] Optionally, the landing gear identification tracker 103 is further configured to track and mark the airplane landing gear in the fourth video picture to obtain a target video picture, which can include that the landing gear identification tracker 103 is specifically configured to track and mark the pixel coordinates of the airplane landing gear in the fourth video picture to obtain the target video picture.

[0065] In the embodiment of the present application, the landing gear identification tracker 103 tracks and marks the pixel coordinates of the airplane landing gear in the fourth video picture to obtain the target video picture, which can realize accurate and quantitative perception of the movement trajectory of the airplane landing gear, and provide visual and quantifiable core data basis for subsequent landing attitude analysis, safety risk assessment and abnormal alarm.

[0066] Optionally, the landing gear identification tracker 103 is further configured to output the target video picture after compression.

[0067] In the embodiment of the present application, the landing gear identification tracker 103 compresses the target video picture into a unified encapsulated data stream format, so that the backend device does not need to pay attention to the data source and only needs to store or transmit according to the standard format. In this way, no matter how the sensor in the landing gear visual probe 101 switches or how different the data source is, the backend device also does not need to be replaced accordingly. This "one backend for multiple machines" adaptation mode makes multiple sensors have higher adaptability to the backend device, improves the device versatility, effectively reduces the complexity and hardware cost of the identification tracking camera platform, and brings convenience to the operation and maintenance of researchers.

[0068] In the embodiment of the present application, the landing gear identification tracker 103 compresses the target video picture into a unified encapsulated data stream format, so that the backend device does not need to pay attention to the data source and only needs to store or transmit according to the standard format. In this way, no matter how the sensor in the landing gear visual probe 101 switches or how different the data source is, the backend device also does not need to be replaced accordingly. This "one backend for multiple machines" adaptation mode makes multiple sensors have higher adaptability to the backend device, improves the device versatility, effectively reduces the complexity and hardware cost of the identification tracking camera platform, and brings convenience to the operation and maintenance of researchers.

[0069] In the embodiment of the present application, the landing gear identification tracker 103 compresses the target video picture into a unified encapsulated data stream format, so that the backend device does not need to pay attention to the data source and only needs to store or transmit according to the standard format. In this way, no matter how the sensor in the landing gear visual probe 101 switches or how different the data source is, the backend device also does not need to be replaced accordingly. This "one backend for multiple machines" adaptation mode makes multiple sensors have higher adaptability to the backend device, improves the device versatility, effectively reduces the complexity and hardware cost of the identification tracking camera platform, and brings convenience to the operation and maintenance of researchers.

[0070] That is, the landing gear recognition tracker 103 searches for the aircraft landing gear in the second video frame, and after capturing the aircraft landing gear, continuously tracks the aircraft landing gear from the next frame of video (i.e., the fourth video frame) and marks and compresses the output of the aircraft landing gear in the fourth video frame and each frame of video thereafter. Each frame of video after the fourth video frame is in the second format described above.

[0071] Optionally, the recognition tracking camera platform is connected with a backend device, and the landing gear recognition tracker 103 is further configured to send the pixel coordinates of the aircraft landing gear in the target video frame to the backend device through a network interface for use by the backend device.

[0072] In the embodiments of the present application, the landing gear recognition tracker 103 converts the visual recognition result from "in-frame" labeling to "data-based" coordinate information and outputs it through a network interface, so that the backend device does not need to repeatedly perform image analysis and can directly use the most core position data for subsequent control and judgment, greatly improving the integration efficiency and response speed of the recognition tracking camera platform.

[0073] Optionally, the target video frame contains time information, pixel coordinates of the center of the aircraft landing gear, size of the aircraft landing gear in the frame, and similarity between the aircraft landing gear and the trained landing gear feature.

[0074] In some embodiments, the landing gear recognition tracker 103 is further configured to receive time information sent by the aircraft and superimpose the time information on the fourth video frame.

[0075] Optionally, the time information is Network Time Protocol (NTP) or Institute of Electrical and Electronics Engineers (IEEE) 1588 Precision Time Protocol (referred to as: 1588 time). The NTP is a network protocol for time synchronization between devices included in the recognition tracking camera platform, and the main purpose is to ensure that all devices in the recognition tracking camera platform use uniform and accurate time. The 1588 time is based on the high-precision time synchronization protocol of the IEEE 1588 standard and can achieve sub-microsecond level precision clock synchronization.

[0076] In the embodiment of the present application, after receiving the time information sent by the aircraft, the landing gear identification tracker 103 can parse the time information and output the parsed time information as a time reference superimposed on the fourth video picture. The whole process realizes the accurate binding of the video picture and the time sent by the aircraft, provides a unique, reliable and tamper-proof time reference for post-audit, fault backtracking and multi-source data fusion, and greatly improves the reliability and usability of the video picture.

[0077] In some embodiments, the landing gear identification tracker 103 is further configured to generate a parameter adjustment instruction according to the image information of the second video picture, and send the parameter adjustment instruction to the landing gear visual probe 101 through the landing gear video gateway 102. The landing gear visual probe 101 is further configured to adjust the parameters of the high-frame-rate sensor 1013 in response to the parameter adjustment instruction, and control the high-frame-rate sensor 1013 to collect the third video picture in the first format according to the adjusted parameters.

[0078] The parameters include at least exposure, color tone, contrast and white balance.

[0079] In the embodiment of the present application, remote closed-loop intelligent optimization of imaging parameters is realized, which ensures that the high-frame-rate sensor 1013 can automatically output high-quality images most suitable for algorithm identification and tracking in any complex lighting environment, effectively improving the adaptive ability and perception reliability of the identification and tracking camera platform. That is, the parameter adjustment instruction of the landing gear identification tracker 103 is fed back to the landing gear visual probe 101 to form an intelligent closed loop of "perception-optimization-re-perception", dynamically adjusting key parameters such as exposure and color tone, thereby overcoming challenging scenes such as backlight, shadow and strong light, ensuring clear presentation of landing gear features and providing stable data guarantee for accurate tracking.

[0080] In some embodiments, Figure 3 is a structural schematic diagram of an identification and tracking camera platform of an aircraft landing gear provided by the embodiment of the present application. As Figure 3 shown, the identification and tracking camera platform further comprises a landing gear safety lock 104 and a wheel load power distribution unit 105. Figure 4 is a connection relationship schematic diagram of the landing gear video gateway, the landing gear identification tracker, the landing gear safety lock and the wheel load power distribution unit provided by the embodiment of the present application. As Figure 4 shown, the landing gear safety lock 104 is connected with the wheel load power distribution unit 105 and the landing gear identification tracker 103 respectively, the wheel load power distribution unit 105 is connected with the landing gear video gateway 102 and the landing gear identification tracker 103 respectively, and power supply is realized for the identification and tracking camera platform of the aircraft landing gear. The landing gear safety lock 104 is configured to reset power supply of the landing gear identification tracker 103 by the landing gear power distribution unit 105 when it is detected that the landing gear identification tracker 103 is in an abnormal state. The landing gear power distribution unit 105 is configured to supply power to the landing gear video gateway 102, the landing gear identification tracker 103 and the landing gear safety lock 104 respectively.

[0081] In the embodiment, the landing gear safety lock 104 communicates with the landing gear identification tracker 103 in real time. When the communication is abnormal, the landing gear safety lock 104 determines that the landing gear identification tracker 103 is in an abnormal state. At this time, a reset instruction is generated and sent to the landing gear power distribution unit 105. After receiving the reset instruction sent by the landing gear safety lock 104, the landing gear power distribution unit 105 can respond to the reset instruction to complete power-off restart of the entire identification tracking camera platform, so as to ensure that the identification tracking camera platform can operate normally.

[0082] In addition, the landing gear power distribution unit 105 is configured to supply power to the landing gear video gateway 102, the landing gear identification tracker 103 and the landing gear safety lock 104 respectively, so as to effectively ensure normal operation of these devices.

[0083] Optionally, the landing gear safety lock 104 is composed of an SGM706 chip, a relay and peripheral circuits thereof. That is, the landing gear safety lock 104 communicates with the landing gear identification tracker 103. When the identification tracking camera platform works abnormally, the communication is interrupted. The SGM706 chip controls the relay to power off the identification tracking camera platform for a preset time length (such as 500 ms) and then power on, so as to complete restart of the identification tracking camera platform and realize power-off reset of the identification tracking camera platform.

[0084] In some embodiments, Figure 5 is a structural schematic diagram of the landing gear power distribution unit provided in the embodiment. As shown in Figure 5 The landing gear power distribution unit 105 is a two-stage direct current (DC)-DC converter. The two-stage DC-DC converter includes a first-stage DC-DC circuit and a second-stage DC-DC circuit. The first-stage DC-DC circuit is configured to output a DC 5V voltage by using a first chip after the DC 28V voltage passes through an overcurrent protection circuit, an overvoltage protection circuit, a transient suppression protection circuit and an ElectroMagnetic Interference (EMI) filter circuit. The second-stage DC-DC circuit is configured to convert the DC 5V voltage into a target device required power supply voltage by using a second chip and provide an output voltage tracking through a target pin in external programming.

[0085] The target device is any one of the landing gear video gateway 102, the landing gear identification tracker 103, and the landing gear safety lock 104.

[0086] The overcurrent protection, overvoltage protection, and transient suppression protection circuit is a combination of power protection circuits, which aims to protect the equipment in the identification tracking camera platform from various abnormal conditions on the power line. The three work together to deal with the three main power threats of "too much current", "too high voltage", and "too strong pulse", providing comprehensive survival protection for the equipment in the identification tracking camera platform.

[0087] The EMI filter circuit is a passive network, and its main function is to prevent high-frequency noise generated by the power circuit from interfering with the outside world, and also to prevent external electromagnetic noise from interfering with the normal operation of the power itself.

[0088] Optionally, the first chip is a CFDA30-24S05 chip.

[0089] Optionally, the second chip is an IS6607A chip or an IS6605A chip.

[0090] Optionally, the target pin is a TRACK / SS pin.

[0091] In the embodiment of the present application, the wheel load power distribution unit 105 adopts a two-stage DC-DC converter to convert the power supply. Specifically, the first-stage DC-DC circuit converts the 28V DC, i.e. DC28V voltage, through the overcurrent protection, overvoltage protection, and transient suppression protection circuit, and the EMI filter circuit, and then outputs a DC5V voltage using the CFDA30-24S05 chip. The second-stage DC-DC circuit converts the DC5V voltage into a power supply voltage required by the target device, such as one of 3.3V, 1.5V, 1.2V, 1V, 0.9V, and 0.75V, and then provides output voltage tracking through the TRACK / SS pin for external programming, to realize the up and down power sequence control.

[0092] Optionally, the identification tracking camera platform further includes a tracking turntable. The target coordinate information (i.e. pixel coordinates of the aircraft landing gear) output by the tracking turntable can be used as the rotation parameter of the tracking turntable.

[0093] Exemplarily, Figure 6 is a schematic diagram of the connection relationship of the devices in the identification tracking camera platform provided by the embodiment of the present application. As shown in Figure 6As shown, the identification tracking camera platform comprises a landing gear visual probe 101, a landing gear video gateway 102, a landing gear identification tracker 103, a landing gear safety lock 104 and a wheel load power distribution unit 105; the landing gear visual probe 101 comprises a visible light sensor 1011, a night vision sensor 1012 and a high frame rate sensor 1013; the landing gear identification tracker 103 comprises an illumination sensor 1031.

[0094] Specifically, the visible light sensor 1011, the night vision sensor 1012 and the high frame rate sensor 1013 are connected with the landing gear video gateway 102 respectively, and the landing gear video gateway 102 supplies power for the three sensors; the landing gear video gateway 102 is connected with the landing gear identification tracker 103 and the illumination sensor 1031; the landing gear identification tracker 103 is connected with the landing gear safety lock 104; the landing gear safety lock 104 is connected with the wheel load power distribution unit 105, and the landing gear safety lock 104 detects the working conditions of the landing gear identification tracker 103 and the wheel load power distribution unit 105 in real time; the wheel load power distribution unit 105 is connected with the landing gear video gateway 102, the landing gear identification tracker 103 and the landing gear safety lock 104, and supplies power for the three devices.

[0095] In combination with the above Figures 1-6 , Figure 7 is a flowchart of the process of identifying and tracking the landing gear provided by the embodiment of the present application. As shown in Figure 7 S1, after obtaining the first frame of video picture (i.e. the second video picture described above), first, the first frame of video picture is preprocessed, such as removing image random noise, enhancing details, processing image according to the principle of normalization, etc., to obtain a first image; then, the first image is scaled in size to obtain a second image with the same image sample size as the trained landing gear feature; then, according to the landing gear feature, the second image is detected and identified (such as classification, front and back background classification, bounding box), and when the situation of the aircraft landing gear is captured in the second image, the identification result is output; S2, according to the identification result, the current state of the aircraft landing gear is obtained, and a real-time model of the aircraft landing gear feature is established; S3, the positive samples and negative samples of the aircraft landing gear (such as the scene near the aircraft landing gear) are continuously trained and learned by using the machine learning algorithm, and at the same time, the parameters of the aircraft landing gear model and the landing gear identification tracker are updated in real time in combination with the motion trajectory of the aircraft landing gear; S4, the main features of the aircraft landing gear, such as position, size, color, etc., are recorded; S5, the next frame of preprocessed video picture (i.e. the second frame of video picture, i.e. the fourth video picture described above) is obtained, and from this video picture, the aircraft landing gear identification detection is not necessary, but the tracking program is directly performed.

[0096] Optionally, the landing gear feature is obtained based on the following steps: obtaining historical landing gear images and inputting the historical landing gear images into a feature extraction model; performing channel attention extraction on the historical landing gear images through a channel attention branch of the feature extraction model to obtain a first landing gear image; performing spatial attention extraction on the historical landing gear images through a spatial attention branch of the feature extraction model to obtain a second landing gear image; performing weighted summation on the first landing gear image and the second landing gear image to obtain a third landing gear image; performing feature extraction on the third landing gear image according to different scales to obtain a plurality of fourth landing gear images; performing Gaussian filtering on the plurality of fourth landing gear images to obtain a plurality of Gaussian filtering images; fusing the plurality of Gaussian filtering images and the plurality of fourth landing gear images one by one to obtain a plurality of fusion images; performing loss calculation on the plurality of fusion images and labeled landing gear features respectively to adjust the feature extraction model to obtain a trained feature extraction model; and finally inputting the historical landing gear images into the trained feature extraction model again to output the trained landing gear feature.

[0097] The aircraft landing gear recognition and tracking method provided by the embodiments of the present application is described below. The aircraft landing gear recognition and tracking camera platform described below can be referred to in conjunction with the aircraft landing gear recognition and tracking method described above.

[0098] Figure 8 is a flowchart of the aircraft landing gear recognition and tracking method provided by the embodiments of the present application. As shown in Figure 8 The recognition and tracking camera method is applied to any of the aircraft landing gear recognition and tracking camera platforms described above, and the aircraft landing gear recognition and tracking camera platform includes a landing gear visual probe, a landing gear video gateway, and a landing gear recognition and tracker. The landing gear visual probe is connected to the landing gear video gateway, and the landing gear video gateway is connected to the landing gear recognition and tracker. The landing gear visual probe includes a visible light sensor, a night vision sensor, and a high frame rate sensor. The frame rate of the high frame rate sensor is higher than that of the visible light sensor and the night vision sensor. The recognition and tracking camera method includes the following steps 801-806.

[0099] Step 801: driving the visible light sensor or the night vision sensor to collect a first video picture of a first format according to the luminous flux in the airport area to be recognized through the landing gear visual probe; and sending the first video picture to the landing gear video gateway through a first interface.

[0100] Optionally, before step 801, the landing gear identification tracker comprises an illuminance sensor; the method can further comprise: generating, by the landing gear identification tracker, a second switching instruction or second switching information in a case where the light flux collected by the illuminance sensor reaches a preset threshold, the second switching information being used to prompt the user to manually switch the night vision sensor to the visible light sensor; sending, by the landing gear video gateway, the second switching instruction to the landing gear visual probe; automatically switching, by the landing gear visual probe in response to the second switching instruction, the night vision sensor to the visible light sensor; and driving the visible light sensor to collect a first video picture in a first format; generating, by the landing gear identification tracker, a third switching instruction or third switching information in a case where the light flux collected by the illuminance sensor does not reach the preset threshold, the third switching information being used to prompt the user to manually switch the visible light sensor to the night vision sensor; sending, by the landing gear video gateway, the third switching instruction to the landing gear visual probe; and automatically switching, by the landing gear visual probe in response to the third switching instruction, the visible light sensor to the night vision sensor; and driving the night vision sensor to collect the first video picture in the first format.

[0101] Step 802: performing format conversion on the first video picture by the landing gear video gateway to obtain a second video picture in a second format; and sending the second video picture to the landing gear identification tracker through a second interface.

[0102] Step 803: identifying, by the landing gear identification tracker, the second video picture, and generating a first switching instruction or first switching information in a case where the landing gear of the aircraft is identified, the first switching information being used to prompt the user to manually switch the visible light sensor or the night vision sensor to the high-frame-rate sensor; and sending, by the landing gear video gateway, the first switching instruction to the landing gear visual probe.

[0103] Optionally, after step 803, the method can further comprise: generating, by the landing gear identification tracker, a parameter adjustment instruction according to the image information of the second video picture, and sending, by the landing gear video gateway, the parameter adjustment instruction to the landing gear visual probe; adjusting, by the landing gear visual probe in response to the parameter adjustment instruction, the parameters of the high-frame-rate sensor; controlling, according to the adjusted parameters, the high-frame-rate sensor to collect a third video picture in the first format; and wherein the parameters at least include exposure, tone, contrast, and white balance.

[0104] Step 804: automatically switching, by the landing gear visual probe in response to the first switching instruction, the visible light sensor or the night vision sensor to the high-frame-rate sensor; controlling the high-frame-rate sensor to collect a third video picture in the first format, and sending the third video picture to the landing gear video gateway through a first interface, the third video picture being a next frame of video picture of the first video picture.

[0105] Step 805, format conversion is performed on the third video picture by the landing gear video gateway to obtain a fourth video picture in a second format; and the fourth video picture is sent to the landing gear recognition tracker through the second interface.

[0106] Step 806, the aircraft landing gear in the fourth video picture is tracked and labeled by the landing gear recognition tracker to obtain a target video picture.

[0107] Optionally, after step 806, the method can further include: identifying the fourth video picture by the landing gear recognition tracker, and generating a fourth switching instruction or fourth switching information in the case that the aircraft landing gear is not identified, the fourth switching information being used to prompt a user to manually switch the high-frame-rate sensor to the visible light sensor or the night vision sensor; sending the fourth switching instruction to the landing gear visual probe by the landing gear video gateway; and automatically switching the high-frame-rate sensor to the visible light sensor or the night vision sensor in response to the fourth switching instruction by the landing gear visual probe; and controlling the visible light sensor or the night vision sensor to continue monitoring and photographing the to-be-identified airport area.

[0108] Optionally, automatically switching the high-frame-rate sensor to the visible light sensor or the night vision sensor in response to the fourth switching instruction by the landing gear visual probe can include: automatically switching the high-frame-rate sensor to the visible light sensor in response to the fourth switching instruction by the landing gear visual probe in the case that the light flux reaches a preset threshold; and automatically switching the high-frame-rate sensor to the night vision sensor in response to the fourth switching instruction in the case that the light flux does not reach the preset threshold.

[0109] Optionally, after step 806, the method can further include: receiving time information sent by the aircraft by the landing gear recognition tracker, and superimposing the time information on the fourth video picture.

[0110] Optionally, the recognition and tracking camera platform further includes a landing gear safety lock and a wheel load power distribution unit, the landing gear safety lock being connected with the wheel load power distribution unit and the landing gear recognition tracker respectively, and the wheel load power distribution unit being connected with the landing gear video gateway and the landing gear recognition tracker respectively; after step 806, the method can further include: performing power-off reset by the wheel load power distribution unit in the case that the landing gear recognition tracker is detected to be in an abnormal state by the landing gear safety lock; and supplying power to the landing gear video gateway, the landing gear recognition tracker and the landing gear safety lock by the wheel load power distribution unit respectively.

[0111] Optionally, the wheel load power distribution unit is a two-stage direct current (DC-DC) converter, the two-stage DC-DC converter comprising: a first-stage DC-DC circuit and a second-stage DC-DC circuit; the method can further comprise: through the first-stage DC-DC circuit, passing the DC 28V voltage through an overcurrent protection circuit, an overvoltage protection circuit, and a transient suppression protection circuit, and an electromagnetic interference filter circuit, and then outputting a DC 5V voltage by using a first chip; through the second-stage DC-DC circuit, converting the DC 5V voltage into a power supply voltage required by a target device by using a second chip, and providing an output voltage tracking through a target pin for external programming; wherein the target device is any one of a landing gear video gateway, a landing gear identification tracker, and a landing gear safety lock.

[0112] In the technical solutions of steps 801-806, the landing gear visual probe drives the visible light sensor or the night vision sensor to collect a first video picture in a first format according to the light flux in the airport region to be identified, and sends the first video picture to the landing gear video gateway through a first interface. The whole process can automatically select the optimal sensor according to the environmental lighting conditions (i.e., the light flux), ensuring that the "available" first video picture can be obtained at any time and under any weather conditions, and ensuring the full-time domain working capability of the identification and tracking camera platform. After receiving the first video picture sent by the first interface, the landing gear video gateway can perform format conversion on the first video picture to obtain a second video picture in a second format, and send the second video picture to the landing gear identification and tracker through a second interface. The whole process provides a unified and standardized data format for the landing gear identification and tracker, simplifies the design of the identification and tracking algorithm, and enables the identification and tracking algorithm to not include complex multi-format analysis code and only need to process a fixed format. After receiving the second video picture sent by the second interface, the landing gear identification and tracker can identify the second video picture. If the aircraft landing gear is identified, it indicates that the aircraft is in the take-off or landing stage. At this time, a sensor switching decision can be executed. Specifically, a first switching instruction or first switching information is generated. The first switching information is used to prompt a user to manually switch the visible light sensor or the night vision sensor to a high-frame-rate sensor to realize manual switching. The landing gear video gateway sends the first switching instruction to the landing gear visual probe. After receiving the first switching instruction sent by the landing gear video gateway, the landing gear visual probe can respond to the first switching instruction, automatically switch the visible light sensor or the night vision sensor to the high-frame-rate sensor to realize automatic triggering of the equipment, and then control the high-frame-rate sensor to collect a third video picture in the first format and send the third video picture to the landing gear video gateway through the first interface. The third video picture is the next frame of video picture of the first video picture. The whole process realizes an intelligent process of "reconnaissance-confirmation-fine tracking". In a broad scene, a conventional sensor (such as a visible light sensor or a night vision sensor) is used for "reconnaissance". Once the target (i.e., the aircraft landing gear) is found, a special high-performance resource (such as a high-frame-rate sensor) is called to perform "fine tracking", thereby realizing optimal allocation of computing and energy consumption resources. In addition, the whole process also realizes sensor switching decision, provides two modes of "automatic instruction" and "manual information", and automatically runs the camera platform in most cases, but in maintenance, debugging or special scenes, the staff can manually intervene according to the prompt information, thereby balancing efficiency and flexibility. In addition, in the process of executing the sensor switching decision, the high-frame-rate sensor is controlled to shoot when the aircraft landing gear is down, and the ordinary sensor (i.e., the visible light sensor or the night vision sensor) is controlled to shoot when the aircraft landing gear is retracted, thereby effectively reducing invalid data recording and improving use efficiency.The landing gear video gateway can perform format conversion on the third video picture sent by the first interface to obtain a fourth video picture in a second format after receiving the third video picture, and send the fourth video picture to the landing gear recognition tracker through the second interface. In the whole process, no matter what kind of sensor and frame rate data is transmitted back by the front end, the landing gear video gateway will convert the data into a unified second format, ensuring that the data format received by the rear-end recognition tracker is consistent and predictable. The landing gear recognition tracker can identify the fourth video picture sent by the second interface, and track and mark the aircraft landing gear in the fourth video picture to obtain a target video picture when the aircraft landing gear is identified. Since the landing gear recognition tracker is compatible with the rear-end device (also referred to as the host computer), the rear-end device does not need to be replaced accordingly no matter how the sensor in the landing gear visual probe is switched. This "one rear-end for multiple machines" adaptation mode makes multiple sensors have higher adaptability and compatibility for the rear-end device, improves the device versatility, effectively reduces the complexity and hardware cost of the recognition tracking camera platform, and brings convenience to the operation and maintenance of researchers.

[0113] Figure 9 is a structural schematic diagram of an electronic device provided by the embodiment of the application. As shown in Figure 9 the electronic device can include a processor 910, a communications interface 920, a memory 930, and a communications bus 940, wherein the processor 910, the communications interface 920, and the memory 930 communicate with each other through the communications bus 940. The processor 910 can invoke the logical instructions in the memory 930 to execute the identification and tracking method of the aircraft landing gear.

[0114] In addition, the logical instructions in the memory 930 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0115] In another aspect, the embodiments of the present application further provide a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer readable storage medium, and the computer program being executed by a processor, so that the computer can execute the aircraft landing gear identification and tracking method provided by the above method.

[0116] In yet another aspect, the embodiments of the present application further provide a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the aircraft landing gear identification and tracking method provided by the above method.

[0117] The apparatus embodiments described above are merely illustrative, and the units described as separate units can or can not be physically separated, and the units displayed as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0118] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0119] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An identification and tracking camera platform for an aircraft landing gear, characterized by, The landing gear visual probe, the landing gear video gateway and the landing gear identification tracker are connected; the landing gear visual probe comprises a visible light sensor, a night vision sensor and a high frame rate sensor, and the frame rate of the high frame rate sensor is higher than that of the visible light sensor and the night vision sensor; The landing gear visual probe is used for driving the visible light sensor or the night vision sensor to collect a first video picture in a first format according to the light flux in the airport area to be identified; The landing gear video gateway is used for performing format conversion on the first video picture to obtain a second video picture in a second format; The landing gear identification tracker is used for identifying the second video picture, and generating a first switching instruction or first switching information in the case of identifying the aircraft landing gear, wherein the first switching information is used for prompting a user to manually switch the visible light sensor or the night vision sensor to the high frame rate sensor; and the first switching instruction is sent to the landing gear visual probe through the landing gear video gateway; The landing gear visual probe is further used for automatically switching the visible light sensor or the night vision sensor to the high frame rate sensor in response to the first switching instruction; controlling the high frame rate sensor to collect a third video picture in the first format, and sending the third video picture to the landing gear video gateway through the first interface, wherein the third video picture is a next frame of video picture of the first video picture; The landing gear video gateway is further used for performing format conversion on the third video picture to obtain a fourth video picture in the second format; and sending the fourth video picture to the landing gear identification tracker through the second interface; The landing gear identification tracker is further used for tracking and labeling the aircraft landing gear in the fourth video picture to obtain a target video picture. The landing gear identification tracker comprises an illumination sensor; The landing gear identification tracker is further used for generating a second switching instruction or second switching information in the case that the light flux collected by the illumination sensor reaches a preset threshold, wherein the second switching information is used for prompting the user to manually switch the night vision sensor to the visible light sensor; and the second switching instruction is sent to the landing gear visual probe through the landing gear video gateway; The landing gear visual probe is specifically used for automatically switching the night vision sensor to the visible light sensor in response to the second switching instruction; and driving the visible light sensor to collect the first video picture in the first format.

2. An identification tracking camera platform for an aircraft landing gear as claimed in claim 1, characterised in that, ​ ​ ​ The landing gear identification tracker is further configured to generate a third switching instruction or third switching information in a case where the light flux collected by the light intensity sensor does not reach the preset threshold, the third switching information being used to prompt the user to manually switch the visible light sensor to the night vision sensor; and send the third switching instruction to the landing gear visual probe through the landing gear video gateway. The landing gear visual probe is specifically configured to automatically switch the visible light sensor to the night vision sensor in response to the third switching instruction; and drive the night vision sensor to collect the first video picture in the first format.

3. The identification tracking camera platform of the aircraft landing gear according to claim 1, wherein The landing gear identification tracker is further configured to identify the fourth video picture, and generate a fourth switching instruction or fourth switching information in a case where the aircraft landing gear is not identified, the fourth switching information being used to prompt the user to manually switch the high frame rate sensor to the visible light sensor or the night vision sensor; and send the fourth switching instruction to the landing gear visual probe through the landing gear video gateway. The landing gear visual probe is further configured to automatically switch the high frame rate sensor to the visible light sensor or the night vision sensor in response to the fourth switching instruction; and control the visible light sensor or the night vision sensor to continue monitoring and shooting the to-be-identified airport area.

4. An identification and tracking camera platform for an aircraft landing gear as claimed in claim 3, characterised in that, The landing gear visual probe is further configured to automatically switch the high frame rate sensor to the visible light sensor or the night vision sensor in response to the fourth switching instruction, including: The landing gear visual probe is specifically configured to automatically switch the high frame rate sensor to the visible light sensor in response to the fourth switching instruction in a case where the light flux reaches the preset threshold; and automatically switch the high frame rate sensor to the night vision sensor in response to the fourth switching instruction in a case where the light flux does not reach the preset threshold.

5. The identification tracking camera platform of the aircraft landing gear according to any one of claims 1-4, wherein The landing gear identification tracker is further configured to receive time information sent by an aircraft, and superimpose the time information on the fourth video picture.

6. The identification tracking camera platform of the aircraft landing gear according to any one of claims 1-4, wherein The landing gear identification tracker is further configured to generate a parameter adjustment instruction according to the image information of the second video picture, and send the parameter adjustment instruction to the landing gear visual probe through the landing gear video gateway; The landing gear visual probe is further configured to adjust the parameters of the high frame rate sensor in response to the parameter adjustment instruction; and control the high frame rate sensor to collect a third video picture in the first format according to the adjusted parameters. The parameters at least include exposure, tone, contrast, and white balance.

7. An identification and tracking camera platform for an aircraft landing gear according to any one of claims 1 to 4, characterised in that, Further comprising: The landing gear safety lock is connected with the landing gear identification tracker and the landing gear video gateway, and the landing gear video gateway is connected with the landing gear identification tracker. The landing gear safety lock is used for power-off reset through the landing gear power distribution unit when the landing gear identification tracker is in an abnormal state. The landing gear power distribution unit is used for supplying power to the landing gear video gateway, the landing gear identification tracker and the landing gear safety lock.

8. An identification tracking camera platform for an aircraft landing gear as claimed in claim 7, characterised in that, The landing gear power distribution unit is a two-stage DC-DC converter, which comprises a first-stage DC-DC circuit and a second-stage DC-DC circuit. The first-stage DC-DC circuit is used for outputting a DC 5V voltage through a first chip after a DC 28V voltage passes through an overcurrent protection circuit, an overvoltage protection circuit, a transient suppression protection circuit and an electromagnetic interference filter circuit. The second-stage DC-DC circuit is used for converting the DC 5V voltage into a target device required power supply voltage through a second chip and providing an output voltage tracking through a target pin in external programming. The target device is any one of the landing gear video gateway, the landing gear identification tracker and the landing gear safety lock.

9. A method of identification tracking of an aircraft landing gear, characterized in that, The identification tracking camera platform of the aircraft landing gear comprises a landing gear visual probe, a landing gear video gateway and a landing gear identification tracker, the landing gear visual probe is connected with the landing gear video gateway, and the landing gear video gateway is connected with the landing gear identification tracker; the landing gear visual probe comprises a visible light sensor, a night vision sensor and a high frame rate sensor, the frame rate of the high frame rate sensor is higher than that of the visible light sensor and the night vision sensor; the method comprises: The landing gear visual probe drives the visible light sensor or the night vision sensor to collect a first video picture in a first format according to the light flux in a to-be-identified airport area, and sends the first video picture to the landing gear video gateway through a first interface; The landing gear video gateway performs format conversion on the first video picture to obtain a second video picture in a second format, and sends the second video picture to the landing gear identification tracker through a second interface; The landing gear identification tracker identifies the second video picture, generates a first switching instruction or first switching information when an aircraft landing gear is identified, and the first switching information is used to prompt a user to manually switch the visible light sensor or the night vision sensor to the high frame rate sensor; and the landing gear video gateway sends the first switching instruction to the landing gear visual probe. In response to the first switching instruction, the landing gear visual probe automatically switches the visible light sensor or the night vision sensor to the high frame rate sensor; controls the high frame rate sensor to collect a third video picture in the first format, and sends the third video picture to the landing gear video gateway through the first interface, the third video picture being a next frame of the first video picture; The landing gear video gateway converts the format of the third video picture to obtain a fourth video picture in the second format, and sends the fourth video picture to the landing gear recognition tracker through the second interface; The landing gear recognition tracker tracks and labels the airplane landing gear in the fourth video picture to obtain a target video picture.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the airplane landing gear recognition and tracking method in claim 9.

Images