Aircraft ground service equipment service joint safety interlock control method and related equipment

Abstract

The invention provides an aircraft ground service equipment service joint safety interlocking control method and related equipment, and is applied to the technical field of automatic control. The method comprises the following steps: acquiring a real-time image of a target area; detecting and identifying the real-time image to obtain a service joint state identification result of the ground service equipment; based on the service connector state recognition result of the ground service equipment, an interlocking control instruction is output, and the interlocking control instruction is used for controlling the aircraft and / or the ground service equipment. Therefore, the action of the equipment or the system is controlled according to the actual state of the service joint, misoperation is avoided, and the safety of collaborative operation of the aircraft and the ground service equipment can be improved.

Description

Background Art

[0002] In the collaborative operation scenarios of airport aircraft and ground service equipment, in order to eliminate the risk of equipment collision or mechanical damage caused by dynamic docking failure, industry specifications have clearly stated that ground service equipment must forcibly lock the platform displacement when the physical connection is established and maintain functional stability in extreme environments.

[0003] Related technologies rely on manual visual inspection of the service joint status of ground service equipment or use sensors to detect the service joint status of ground service equipment. However, manual visual inspection has many problems, such as high missed detection rate due to fatigue of ground staff and limited viewing angle, and accidents may also be caused by communication errors. Relying on traditional sensor solutions is not only difficult to resist environmental interference such as rain, fog, and reflections, but also cannot be compatible with the diversity of interface structures of different devices, which ultimately leads to insufficient robustness of status detection. Therefore, how to improve the safety of collaborative operations between aircraft and ground service equipment is a technical problem that needs to be solved urgently.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present invention provides a method and related equipment for controlling the safety interlocking of service joints of aircraft ground service equipment, which can improve the safety of collaborative operations between aircraft and ground service equipment.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a method for safety interlock control of a service joint of aircraft ground service equipment is provided, comprising: acquiring a real-time image of a target area; detecting and identifying the real-time image to obtain a service joint status identification result of the ground service equipment; and outputting an interlock control instruction based on the service joint status identification result of the ground service equipment, wherein the interlock control instruction is used to control the aircraft and / or the ground service equipment.

[0008] In some embodiments, the target area includes an aircraft ground service interface area and / or a ground service equipment service connector storage bin area, and acquiring a real-time image of the target area includes: collecting real-time multispectral images of the aircraft ground service interface area and / or the ground service equipment service connector storage bin area.

[0009] In some embodiments, when the target area is the aircraft ground service interface area, the detecting and recognizing the real-time image to obtain the service joint state recognition result of the ground service equipment includes: inputting the real-time multi-spectral image of the aircraft ground service interface area into a visual detection model, and outputting the service joint state recognition result of the ground service equipment, where the visual detection model is trained according to the historical multi-spectral images of the aircraft ground service interface area.

[0010] In some embodiments, the inputting the real-time multi-spectral image of the aircraft ground service interface area into a visual detection model and outputting the service joint state recognition result of the ground service equipment includes: extracting a mixed feature in the real-time multi-spectral image of the aircraft ground service interface area, where the mixed feature includes at least one of the following: color feature, texture feature, and shape feature; determining whether there is a service joint of the ground service equipment in the real-time multi-spectral image according to at least one of the color feature, texture feature, and shape feature; when there is a service joint of the ground service equipment in the real-time multi-spectral image, outputting a first joint state recognition result; when there is no service joint of the ground service equipment in the real-time multi-spectral image, outputting a second joint state recognition result.

[0011] In some embodiments, when the target area is the service joint storage bin area of the ground service equipment, the detecting and recognizing the real-time image to obtain the service joint state recognition result of the ground service equipment includes: detecting and recognizing the real-time multi-spectral image of the service joint storage bin area of the ground service equipment to determine whether there is preset reference information in the real-time multi-spectral image of the service joint storage bin area of the ground service equipment, where the preset reference information includes at least one of the following: a service joint of the ground service equipment, a two-dimensional code on the service joint of the ground service equipment, and a color card identifier on the service joint of the ground service equipment; when there is preset reference information in the real-time multi-spectral image of the service joint storage bin area of the ground service equipment, obtaining a third joint state recognition result; when there is no preset reference information in the real-time multi-spectral image of the service joint storage bin area of the ground service equipment, obtaining a fourth joint state recognition result.

[0012] In some embodiments, the detecting and recognizing the real-time image to obtain the service joint state recognition result of the ground service equipment includes: detecting and recognizing the real-time multi-spectral image of the aircraft ground service interface area to obtain a first sub-joint state recognition result; detecting and recognizing the real-time multi-spectral image of the service joint storage bin area of the ground service equipment to obtain a second sub-joint state recognition result; and obtaining the service joint state recognition result of the ground service equipment according to the first sub-joint state recognition result and the second sub-joint state recognition result.

[0013] In some embodiments, the interface for outputting the interlock control instruction supports digital signals, serial remote terminal units / transmission control protocols, and programmable logic controller instructions.

[0014] According to another aspect of the present disclosure, there is also provided a safety interlock control device for a service connection of an aircraft ground service device. The device includes: an acquisition module configured to acquire a real-time image of a target area; a detection and recognition module configured to perform detection and recognition on the real-time image to obtain a recognition result of the state of the service connection of the ground service device; and an output module configured to output an interlock control instruction based on the recognition result of the state of the service connection of the ground service device, where the interlock control instruction is used to control the aircraft and / or the ground service device.

[0015] According to another aspect of the present disclosure, there is also provided an electronic device. The electronic device includes: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the safety interlock control method for the service connection of the aircraft ground service device according to any one of the above by executing the executable instructions.

[0016] According to another aspect of the present disclosure, there is also provided a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the safety interlock control method for the service connection of the aircraft ground service device according to any one of the above is implemented.

[0017] According to another aspect of the present disclosure, there is also provided a computer program product, including: a computer program or instruction. When the computer program or instruction is executed by a processor, the safety interlock control method for the service connection of the aircraft ground service device according to any one of the above is implemented.

[0018] The present disclosure provides a safety interlock control method and related devices for a service connection of an aircraft ground service device. The method includes: acquiring a real-time image of a target area; performing detection and recognition on the real-time image to obtain a recognition result of the state of the service connection of the ground service device; and outputting an interlock control instruction based on the recognition result of the state of the service connection of the ground service device, where the interlock control instruction is used to control the aircraft and / or the ground service device. In this way, the operation of the device or system is controlled according to the actual state of the service connection, avoiding misoperation, and the safety of the collaborative operation between the aircraft and the ground service device can be improved.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0021] Figure 1 Schematic diagram of the system architecture of a method for safety interlock control of a service connection of an aircraft ground service equipment in an embodiment of the present disclosure;

[0022] Figure 2 Flowchart of a method for safety interlock control of a service connection of an aircraft ground service equipment in an embodiment of the present disclosure;

[0023] Figure 3 Schematic diagram of an aircraft ground service interface area in an embodiment of the present disclosure;

[0024] Figure 4 Flowchart of a method for detecting and recognizing a real-time image in an embodiment of the present disclosure;

[0025] Figure 5 Flowchart of a method for detecting and recognizing a real-time image in an embodiment of the present disclosure;

[0026] Figure 6 Schematic diagram of an area of a storage bin for a service connection of a ground service equipment in an embodiment of the present disclosure;

[0027] Figure 7 Flowchart of a method for detecting and recognizing a real-time image in an embodiment of the present disclosure;

[0028] Figure 8 Schematic diagram of a vision detection architecture in an embodiment of the present disclosure;

[0029] Figure 9 Schematic diagram of a safety interlock control device for a service connection of an aircraft ground service equipment in an embodiment of the present disclosure;

[0030] Figure 10 Block diagram of the structure of an electronic device in an embodiment of the present disclosure. Detailed implementation manners

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0032] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0033] For ease of understanding, before introducing the embodiments of the present disclosure, the following explanations are first given to the terms involved in the embodiments of the present disclosure:

[0034] A convolutional neural network is a deep learning model specifically designed to process data with a grid structure (such as images). It mainly consists of a convolutional layer, a pooling layer, and a fully connected layer. The convolutional layer performs a convolution operation by sliding a convolution kernel over the image, automatically extracting local features of the image and reducing the number of parameters. The pooling layer is used to downsample the feature map after convolution, reducing the data dimension, retaining the main features, and reducing the computational amount. The fully connected layer then integrates the features extracted previously and is used for tasks such as classification or regression. The convolutional neural network has strong feature learning capabilities and can be applied to fields such as image recognition and object detection.

[0035] A support vector machine is a supervised learning model. Its basic idea is to find an optimal hyperplane in the feature space to separate data points of different classes as much as possible. For linearly separable data, it can find a hyperplane that can completely distinguish the two classes of data and maximize the margin between the two classes of data to this hyperplane. For linearly inseparable data, the data can be mapped to a high-dimensional space through a kernel function to make it linearly separable in the high-dimensional space and then classified. The support vector machine has a wide range of applications in the field of image recognition. It has good generalization ability and can effectively classify and predict unknown data with a limited number of training samples.

[0036] Random forest is an ensemble learning method. It consists of multiple decision trees. When constructing, first perform sampling with replacement (bootstrap sampling) on the training data to obtain different subsets of data for constructing different decision trees. At the same time, when splitting nodes in each decision tree, instead of considering all features, a part of the features are randomly selected, which can increase the diversity of the trees. Multiple decision trees classify the samples respectively, and finally the final result is determined through a voting mechanism. Random forest has advantages such as high accuracy, the ability to handle high-dimensional data, and being not easily overfitted, and has good generalization ability.

[0037] Aircraft ground service equipment at least includes aircraft ground power supply units, aircraft ground static inverters, aircraft ground air source units, aircraft ground air-conditioning units, aircraft fresh water trucks, aircraft sewage trucks, aircraft oxygen filling equipment, pipeline refueling equipment, tank refueling equipment, passenger boarding bridges, etc. In this disclosure, aircraft ground service equipment is simply referred to as ground service equipment.

[0038] The following will describe in detail the specific implementation manners of the embodiments of this disclosure with reference to the accompanying drawings.

[0039] Figure 1 An exemplary application system architecture diagram is shown in which the aircraft ground service equipment service connection safety interlock control method in the embodiments of this disclosure can be applied. As Figure 1 shown, the system architecture may include a terminal device 101, a network 102, and a server 103.

[0040] The network 102 is used to provide a medium for the communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.

[0041] Optionally, the above-mentioned wireless network or wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network. In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent the data exchanged through the network. Additionally, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec), etc. can be used to encrypt all or some of the links. In other embodiments, customized and / or proprietary data communication technologies can also be used to replace or supplement the above data communication technologies.

[0042] The terminal device 101 can be various visual perception devices, including but not limited to smartphones, tablets, laptop computers, smart watches, wearable devices, augmented reality devices, virtual reality devices, industrial multispectral cameras, intelligent edge computing cameras, embedded collaborative vision terminals, etc.

[0043] Optionally, the clients of the application programs installed in different terminal devices 101 are the same, or are clients of the same type of application programs based on different operating systems. Depending on the different terminal platforms, the specific form of the client of the application program can also be different. For example, the client of the application program can be a mobile client, a PC client, etc.

[0044] The server 103 can be a server that provides various services, such as a back-end management server that supports the operations performed by the user using the terminal device 101. The back-end management server can analyze and process data such as requests received, and feedback the processing results to the terminal device.

[0045] Optionally, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0046] Those skilled in the art can be aware that Figure 1 the numbers of the terminal devices, networks, and servers in are merely illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers. The embodiments of the present disclosure do not limit this.

[0047] Under the above system architecture, an aircraft ground service equipment service connection safety interlock control method is provided in the embodiments of the present disclosure. This method can be executed by any electronic device with computing and processing capabilities.

[0048] In some embodiments, the aircraft ground service equipment service connection safety interlock control method provided in the embodiments of the present disclosure can be executed by the terminal device of the above system architecture; in some other embodiments, the aircraft ground service equipment service connection safety interlock control method provided in the embodiments of the present disclosure can be executed by the server in the above system architecture; in some other embodiments, the aircraft ground service equipment service connection safety interlock control method provided in the embodiments of the present disclosure can be implemented by the interaction between the terminal device and the server in the above system architecture.

[0049] Figure 2 The flowchart of an aircraft ground service equipment service connection safety interlock control method in the embodiments of the present disclosure is shown. As Figure 2 shown, the aircraft ground service equipment service connection safety interlock control method provided in the embodiments of the present disclosure includes the following steps:

[0050] S202, obtain a real-time image of the target area.

[0051] In some embodiments, the target area includes the aircraft ground service interface area and / or the ground service equipment service connection storage bin area. Obtaining a real-time image of the target area includes: collecting real-time multi-spectral images of the aircraft ground service interface area and / or the ground service equipment service connection storage bin area.

[0052] In this embodiment, during the identification process of the service connection state of the ground service equipment, the target area refers to the key operation range that needs to be monitored in real time, mainly including the aircraft ground service interface area and the storage bin area of the service connection of the ground service equipment. The aircraft ground service interface area is usually a standardized connection port preset on the surface of the aircraft. For example, in the embodiment of the present disclosure Figure 3 shows a schematic diagram of an aircraft ground service interface area, in combination with Figure 3 as shown, the aircraft ground service interface area can be a metal interface at the belly of the aircraft for connecting the ground power supply or air conditioning pipeline. Its structure mostly adopts a guide groove and a contact array design, and the surface may be covered with an openable and closable protective cover. The storage bin area of the service connection of the ground service equipment refers to the enclosed cabin on the ground support equipment (such as a power supply vehicle, an air conditioning vehicle) for storing and retrieving the service connection. A slide rail mechanism and a mechanical locking device are often configured inside to ensure the safe storage of the service connection in the non-operating state.

[0053] It should be noted that in this embodiment, real-time images can be collected by an image acquisition module. Specifically, the image acquisition module realizes the dynamic monitoring of the target area by deploying a visible light camera (resolution ≥ 2 million pixels, frame rate ≥ 30fps). Exemplarily, for the aircraft ground service interface area, the camera enables the HDR imaging mode (dynamic range ≥ 100dB) in a strong light environment, eliminates the highlight overflow on the metal surface through multi-frame synthesis technology, and at the same time retains the buckle alignment details in the shadow area; in rainy and foggy weather, it switches to the infrared fill light mode, uses 850nm wavelength light to penetrate the water film, and accurately identifies the opening and closing state of the interface protective cover. Exemplarily, for the storage bin area of the service connection of the ground service equipment, the annular fill light system automatically matches the lighting strategy according to the ambient color temperature (adjustable from 3000K to 6500K): high color temperature white light is used during the day to enhance the texture contrast of the mechanical structure inside the cabin, and it switches to the low color temperature mode at night to cooperate with infrared imaging to ensure that the contact state between the end of the connection and the limit sensor can still be detected in a dark field environment.

[0054] Taking the docking of an aircraft with a ground air conditioning vehicle as an example, the camera first captures the image of the ground service interface area on the belly of the aircraft: in the white light mode, the projection coincidence degree of the guide groove and the connection is analyzed through a geometric matching algorithm to detect micron-level offsets; if the ambient illumination changes suddenly (such as the instantaneous brightness change caused by cloud cover), the HDR imaging can maintain stable feature extraction and avoid misjudgment caused by overexposure or underexposure. At the same time, the system can continuously monitor the storage bin area of the ground air conditioning vehicle and capture the movement trajectory during the connection recovery process through a 30fps high frame rate video stream.

[0055] In this embodiment, through the multi - spectral fusion and adaptive imaging mechanism, traditional detection pain points such as metal reflection, water vapor interference, and sudden illumination changes are effectively overcome, and the safety risks caused by visual blind spots or environmental interference are significantly reduced, providing reliable guarantee for dynamic docking scenarios with high safety requirements.

[0056] S204. Detect and identify the real - time image to obtain the recognition result of the service connection state of the ground service equipment.

[0057] In this embodiment, detection and identification refer to the process of analyzing the real - time image based on computer vision algorithms, extracting key features (such as geometric contours, texture contrasts, spatial position relationships), and converting the image data into structured state labels through classification models (such as deep learning networks or traditional feature matching). The recognition result of the service connection state of the ground service equipment is a global determination of the spatial state of the service connection of the ground service equipment in different operation stages, and the state categories are dynamically defined based on the functional attributes of the target area. For example: when the target area is the docking operation area (such as the ground service interface of the aircraft), the state determination focuses on the coupling integrity of the connection and the interface (such as "extended / not extended"); when the target area is the storage and management area (such as the storage compartment of the service connection of the ground service equipment), the state determination emphasizes the return safety of the connection (such as "retracted / not retracted").

[0058] In this embodiment, the spatial pose of the connection in a specific functional scenario is analyzed through multi - spectral images, and a risk - level label strongly related to the current operation stage is output, providing a scenario - based decision - making basis for interlock control and improving the safety and efficiency of the collaborative operation between the aircraft and the ground service equipment.

[0059] S206. Based on the recognition result of the service connection state of the ground service equipment, output an interlock control instruction, where the interlock control instruction is used to control the aircraft and / or the ground service equipment.

[0060] In this embodiment, the interlock control instruction refers to a digital control signal generated based on the connection state recognition result, which is used to forcibly restrict the collaborative action permissions of the aircraft and the ground service equipment (such as power supply vehicles, air - conditioning vehicles), preventing mechanical collisions or equipment damage caused by abnormal states. Its control objects include: restricting the displacement of the aircraft (such as prohibiting backward movement), power output (such as cutting off the drive), etc.; locking the movement of the ground service equipment, the movement of the robotic arm (such as freezing the connection expansion and contraction), closing the energy transmission (such as disconnecting the power supply / fluid path), etc. Specific triggering conditions can include risk states (such as "extended", "not retracted"): immediately output a "lock instruction" to terminate the equipment operation; safety states (such as "not extended", "retracted"): maintain or restore the normal operation permissions of the equipment.

[0061] In some embodiments, the interface for outputting interlock control instructions supports digital signals, serial remote terminal units / transmission control protocols, and programmable logic controller instructions. Due to the support for multiple signal types, this interface has strong versatility and can output interlock control instructions among different devices, different communication requirements, and different automation systems, improving the flexibility, compatibility, and expandability of the system.

[0062] In this embodiment, controlling the actions of devices or systems according to the actual state of the service connection can avoid misoperations and improve the safety of the collaborative operation between the aircraft and ground service equipment.

[0063] In some embodiments, Figure 4 The flowchart of a method for detecting and recognizing real-time images in an embodiment of the present disclosure is shown. As Figure 4 shown, when the target area is the aircraft ground service interface area, the method for detecting and recognizing real-time images provided in the embodiment of the present disclosure includes the following steps:

[0064] S402: Input the real-time multi-spectral image of the aircraft ground service interface area into the visual detection model to output the recognition result of the service connection state of the ground service equipment, where the visual detection model is trained according to the historical multi-spectral images of the aircraft ground service interface area.

[0065] In this embodiment, the visual detection model is trained using a historical multi-spectral image library, which covers diverse working condition data of the service connections of ground service equipment such as day / night, sunny / rainy, extended / unextended, etc. The visual detection model can adopt a single model machine learning model, and specifically, the most suitable algorithm can be selected according to the scenario requirements. For example, when extremely high real-time performance is required (such as response time < 50 ms), a lightweight convolutional neural network is used to directly complete end-to-end joint positioning and state classification; if the training data volume is limited and high interpretability is required, a support vector machine can be used for state discrimination. The visual detection model can also adopt a hybrid model architecture. Specifically, the deep semantic features of the joint can be extracted through a convolutional neural network, combined with a support vector machine for geometric spatial distribution matching, and a random forest for texture anomaly classification to form a hybrid feature analysis ability. The model output can include the joint position bounding box (Bbox = (x, y, w, h)) and the state confidence level (Ps ∈ [0, 1]). For example, when the center offset of the detection box ≤ 2 pixels and Ps ≥ 0.95, it is determined as "extended", while Ps < 0.8 or offset > 5 pixels triggers a "not extended" warning.

[0066] It should be noted that when the target area is the aircraft ground service interface area, the interlock control instructions can be "prohibit the aircraft / ground service equipment from moving backward" and "prohibit the aircraft from moving".

[0067] In some embodiments, the real-time multi-spectral image of the aircraft ground service interface area is input into a visual detection model, and the recognition result of the service joint state of the ground service equipment is output, including: extracting mixed features from the real-time multi-spectral image of the aircraft ground service interface area, where the mixed features include at least one of the following: color feature, texture feature, and shape feature; determining whether there is a service joint of the ground service equipment in the real-time multi-spectral image according to at least one of the color feature, texture feature, and shape feature; when there is a service joint of the ground service equipment in the real-time multi-spectral image, outputting a first joint state recognition result; when there is no service joint of the ground service equipment in the real-time multi-spectral image, outputting a second joint state recognition result.

[0068] In this embodiment, the mixed features include three-dimensional information of color, texture, and shape: the color feature analyzes the chromaticity difference between the metal surface of the interface and the oxidized / polluted area, the texture feature quantifies the arrangement regularity of the contact array through the gray-level co-occurrence matrix, and the shape feature calculates the projection offset of the guide groove contour by using the template matching algorithm.

[0069] Specifically, the visual detection model inputs the mixed features into a multi-algorithm fusion architecture for state discrimination. When the color feature shows significant metal reflection in the interface area, the texture feature detects that the contacts are completely covered, and the shape feature matching degree is greater than a preset threshold (for example, ≥ 95%), the model determines it as the first joint state recognition result (the service joint extends), triggering the interlock control instructions "prohibit the ground service equipment from retreating" and "prohibit the aircraft from moving" to prevent mechanical detachment of the coupled equipment due to displacement. On the contrary, if the shape feature does not detect an effective contour or the texture feature does not show a large area of joint shadow in the target area, it is determined as the second joint state (the service joint does not extend).

[0070] In some embodiments, Figure 5 The flowchart of a method for detecting and recognizing a real-time image in an embodiment of the present disclosure is shown. As Figure 5 shown, when the target area is the storage bin area of the service joint of the ground service equipment, the method for detecting and recognizing a real-time image provided in the embodiment of the present disclosure includes the following steps:

[0071] S502, detecting and recognizing the real-time multi-spectral image of the storage bin area of the service joint of the ground service equipment, and determining whether there is preset reference information in the real-time multi-spectral image of the storage bin area of the service joint of the ground service equipment, where the preset reference information includes at least one of the following: the service joint of the ground service equipment, the two-dimensional code on the service joint of the ground service equipment, and the color card identifier on the service joint of the ground service equipment.

[0072] In this embodiment, the preset reference information refers to the physical identifiers or features pre-set on the servicing connectors of the ground service equipment, which are used to quickly locate and identify the connector status in the image. Its types include: Servicing connector body: directly identified by inherent features such as geometric contours and textures; QR code identifier: a standardized encoded graphic printed on the surface of the connector (such as Quick Response QR code, Data Matrix code), storing metadata such as device model and serial number; color card identifier: a color block with a specific color number, establishing the association between the connector identity and position through chromaticity-brightness features. Combining Figure 6 As shown in the schematic diagram of the area of the servicing connector storage bin of the ground service equipment, real-time multi-spectral image detection and recognition means collecting the image of the servicing connector storage bin through visible light and infrared cameras, and combining algorithm analysis to determine whether there is preset reference information (such as QR code, color card) in the image, so as to determine whether the servicing connector of the ground service equipment is retracted.

[0073] Specifically, for example, a high-contrast QR code (such as a black and white DataMatrix code) is printed on the surface of the servicing connector of the ground service equipment. After the system captures the image of the storage bin through a multi-spectral camera, the following process is executed: judging the type of QR code (QR / Data Matrix, etc.) based on a deep learning model (such as ResNet-18); performing perspective transformation correction on the distorted image according to the pre-stored installation angle parameters of the servicing connector (such as pitch angle ±15°, yaw angle ±10°); calling an adapted decoding engine to extract the encoded data Dcode and output the decoding credibility Pc (such as Pc≥0.95 is judged as valid).

[0074] In some embodiments, data can be restored through an error correction algorithm when the QR code is partially damaged; and it supports dynamic switching of multiple code systems to adapt to different brand connector identification systems; Dcode can also be associated with maintenance records to realize the life cycle management of the servicing connector.

[0075] Another example is spraying a weather-resistant color card (such as fluorescent orange) on the top of the servicing connector. The system performs joint chromaticity-brightness analysis, specifically including gamut mapping: converting the image from RGB to the CIELAB color space and separating the luminance (L*) and chromaticity (a*, b*) components; color difference calculation: quantifying the difference between the real-time color card and the standard color number based on the ΔE00 formula (ΔE00<1.5 is considered a match); dynamic matching: outputting the color matching degree Pcolor = 1-(ΔE00 / threshold), and when Pcolor≥0.9, it is judged that the color card is visible. Among them, the ΔE00 formula is the CIEDE2000 color difference formula, which is used to measure the degree of difference between two colors.

[0076] In this embodiment, multi-spectral imaging can resist illumination changes (such as identifying the thermal radiation characteristics of a color card in the night infrared mode); the color difference calculation has low complexity and can achieve passive identification, that is, without electronic components, and is applicable to harsh working conditions such as high temperature and high humidity.

[0077] It should be noted that in some embodiments, the two-dimensional code and color card solutions can complement each other. The two-dimensional code provides digital information interaction capabilities, while the color card provides a redundant detection channel in extreme environments. Both convert the identification of the adapter state into a dual verification of target detection + feature verification through preset reference information, reducing the misjudgment rate compared to a single contour detection scheme and supporting rapid adaptation of cross-brand devices.

[0078] S504. When the preset reference information exists in the real-time multi-spectral image of the ground service equipment service adapter storage area, obtain the third adapter state identification result.

[0079] In this embodiment, the third adapter state identification result indicates that the service adapter of the ground service equipment is retracted.

[0080] S506. When the preset reference information does not exist in the real-time multi-spectral image of the ground service equipment service adapter storage area, obtain the fourth adapter state identification result.

[0081] In this embodiment, the fourth adapter state identification result indicates that the service adapter of the ground service equipment is not retracted.

[0082] In some embodiments, Figure 7 The flowchart of a method for detecting and identifying a real-time image in an embodiment of the present disclosure is shown. As Figure 7 shown, when the target area is the aircraft ground service interface area and the ground service equipment service adapter storage area, the method for detecting and identifying a real-time image provided in the embodiment of the present disclosure includes the following steps:

[0083] S702. Detect and identify the real-time multi-spectral image of the aircraft ground service interface area to obtain the first sub-adapter state identification result.

[0084] In this embodiment, the first sub-adapter state identification result includes whether the service adapter of the ground service equipment extends or does not extend. The specific detection and identification method refers to the foregoing description and will not be elaborated here.

[0085] S704. Detect and identify the real-time multi-spectral image of the ground service equipment service adapter storage area to obtain the second sub-adapter state identification result.

[0086] In this embodiment, the second sub-adapter state identification result includes whether the service adapter of the ground service equipment is retracted or not retracted. The specific detection and identification method refers to the foregoing description and will not be elaborated here.

[0087] S706. Based on the recognition results of the first sub-connector status and the second sub-connector status, obtain the recognition result of the service connector status of the ground service equipment.

[0088] In this embodiment, Table 1 is a correspondence table between the recognition results of the service connector status of the ground service equipment and the interlock control instructions provided by the present disclosure embodiment.

[0089] Table 1

[0090]

[0091] In this embodiment, Scheme 1 is the recognition result of the service connector status of the ground service equipment when the target area is the aircraft ground service interface area; Scheme 2 is the recognition result of the service connector status of the ground service equipment when the target area is the storage bin area of the service connector of the ground service equipment and the preset reference information is a two-dimensional code; Scheme 3 is the recognition result of the service connector status of the ground service equipment when the target area is the storage bin area of the service connector of the ground service equipment and the preset reference information is a color card. The multi-modal enhancement combination refers to the combination of multiple schemes.

[0092] In this embodiment, the multi-modal enhancement mode can combine Scheme 1 to Scheme 3 and apply them as inputs to the same service connector of the ground service equipment, and dynamically adjust the output threshold according to the weighted fusion ratio of multi-source data to reduce the risk of misjudgment.

[0093] In some embodiments, Figure 8 is a schematic diagram of a visual detection architecture provided by the present disclosure embodiment. As shown in combination Figure 8 The visual detection architecture includes an image acquisition module 81, an image processing module 82, an image processing module 83, an image processing module 84, and an interlock signal output module 85, an interlock signal output module 86, and an interlock signal output module 87. Among them, the image processing module 83 and the image processing module 84 are respectively connected to the image acquisition module 81, the interlock signal output module 85 is connected to the image processing module 82, the interlock signal output module 86 is connected to the image processing module 83, and the interlock signal output module 87 is connected to the image processing module 84.

[0094] In this embodiment, the image acquisition module 81 is used to acquire real-time images of the target area. The image processing module 82 runs a visual detection model based on image feature analysis. The image processing module 83 runs a general recognition model based on two-dimensional codes. The image processing module 84 runs a joint detection model based on chromaticity-chromatic aberration. The interlock signal output modules 85, 86, and 87 all support switch quantity signals, serial remote terminal units / transmission control protocols, and programmable logic controller instructions, and are used to output interlock signals. The apron equipment executes relevant interlock control instructions by receiving the interlock signals.

[0095] Based on the same inventive concept, an aircraft ground service equipment servicing joint safety interlock control device is also provided in the embodiments of the present disclosure, as described in the following embodiments. Since the principle of problem-solving in the device embodiments is similar to that of the above method embodiments, the implementation of the device embodiments can refer to the implementation of the above method embodiments, and the repeated parts will not be described again.

[0096] Figure 9 The figure shows a schematic diagram of an aircraft ground service equipment servicing joint safety interlock control device in the embodiments of the present disclosure, as Figure 9 shown, the device includes: an acquisition module 91, a detection and recognition module 92, and an output module 93.

[0097] The acquisition module 91 is configured to acquire a real-time image of a target area; the detection and recognition module 92 is configured to perform detection and recognition on the real-time image to obtain a recognition result of the state of the servicing joint of the ground service equipment; the output module 93 is configured to output an interlock control instruction based on the recognition result of the state of the servicing joint of the ground service equipment, where the interlock control instruction is used to control the aircraft and / or the ground service equipment.

[0098] In some embodiments, the target area includes an aircraft ground servicing interface area and / or a ground service equipment servicing joint storage bin area, and the acquisition module 91 is configured to: collect real-time multi-spectral images of the aircraft ground servicing interface area and / or the ground service equipment servicing joint storage bin area.

[0099] In some embodiments, when the target area is the aircraft ground servicing interface area, the detection and recognition module 92 is configured to: input the real-time multi-spectral image of the aircraft ground servicing interface area into a visual detection model, and output the recognition result of the state of the servicing joint of the ground service equipment, where the visual detection model is trained according to the historical multi-spectral images of the aircraft ground servicing interface area.

[0100] In some embodiments, the detection and recognition module 92 is configured to: extract mixed features from the real-time multi-spectral image of the aircraft ground servicing interface area, where the mixed features include at least one of the following: color feature, texture feature, and shape feature; determine whether there is a servicing joint of the ground service equipment in the real-time multi-spectral image according to at least one of the color feature, texture feature, and shape feature; when there is a servicing joint of the ground service equipment in the real-time multi-spectral image, output the first joint state recognition result; when there is no servicing joint of the ground service equipment in the real-time multi-spectral image, output the second joint state recognition result.

[0101] In some embodiments, when the target area is the storage bin area of the ground service equipment service connection, the detection and recognition module 92 is configured to: detect and recognize the real-time multispectral image of the storage bin area of the ground service equipment service connection to determine whether there is preset reference information in the real-time multispectral image of the storage bin area of the ground service equipment service connection, where the preset reference information includes at least one of the following: the service connection of the ground service equipment, the two-dimensional code on the ground service equipment service connection, and the color card identifier on the ground service equipment service connection; when there is preset reference information in the real-time multispectral image of the storage bin area of the ground service equipment service connection, obtain the third joint status recognition result; when there is no preset reference information in the real-time multispectral image of the storage bin area of the ground service equipment service connection, obtain the fourth joint status recognition result.

[0102] In some embodiments, the detection and recognition module 92 is configured to: detect and recognize the real-time multispectral image of the aircraft ground service interface area to obtain the first sub-joint status recognition result; detect and recognize the real-time multispectral image of the storage bin area of the ground service equipment service connection to obtain the second sub-joint status recognition result; and obtain the service connection status recognition result of the ground service equipment according to the first sub-joint status recognition result and the second sub-joint status recognition result.

[0103] In some embodiments, the interface for outputting the interlock control instruction supports digital input / output signals, serial remote terminal unit / transmission control protocol, and programmable logic controller instructions.

[0104] It should be noted here that the functions of the various modules in the above device embodiments are the same as the corresponding steps in the method embodiments in terms of the examples and application scenarios they implement, but are not limited to the content disclosed in the above method embodiments. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0105] Those skilled in the art can understand that various aspects of the present disclosure can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module" or "system" here.

[0106] Based on the same inventive concept, an electronic device is further provided in the embodiments of the present disclosure. The electronic device includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the aircraft ground service equipment service connection safety interlock control method of any one of the above via executing the executable instructions. Since the principle of solving problems in the embodiment of this electronic device is similar to that in the above method embodiment, the implementation of the embodiment of this electronic device can refer to the implementation of the above method embodiment, and the repeated parts will not be elaborated.

[0107] The following will describe the electronic device 1000 according to this embodiment of the present disclosure with reference to Figure 10 this. Figure 10 The shown electronic device 1000 is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0108] As Figure 10 shown, the electronic device 1000 is presented in the form of a general computing device. The components of the electronic device 1000 may include but are not limited to: the at least one processing unit 1010 described above, the at least one storage unit 1020 described above, and a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010).

[0109] Among them, the storage unit stores program codes, and the program codes can be executed by the processing unit 1010, so that the processing unit 1010 executes the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the processing unit 1010 may execute the following steps of the above method embodiment: obtain a real-time image of a target area; perform detection and recognition on the real-time image to obtain a recognition result of the service connection state of the ground service equipment; based on the recognition result of the service connection state of the ground service equipment, output an interlock control instruction, wherein the interlock control instruction is used to control the aircraft and / or the ground service equipment. In this way, controlling the actions of the equipment or system according to the actual state of the service connection can avoid misoperations and improve the safety of the collaborative operation between the aircraft and the ground service equipment.

[0110] The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 10201 and / or a cache storage unit 10202, and may further include a read-only storage unit (ROM) 10203.

[0111] The storage unit 1020 may also include a program / utility 10204 having a set (at least one) of program modules 10205. Such program modules 10205 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0112] The bus 1030 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus structures.

[0113] The electronic device 1000 may also communicate with one or more external devices 1040 (such as a keyboard, a pointing device, a Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1000, and / or may communicate with any device that enables the electronic device 1000 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication may be carried out through the input / output (I / O) interface 1050. Also, the electronic device 1000 may communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 1060. As shown in the figure, the network adapter 1060 communicates with other modules of the electronic device 1000 through the bus 1030. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0114] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or can be implemented by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0115] Based on the same inventive concept, embodiments of the present disclosure also provide a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, it implements the aircraft ground service equipment service connection safety interlock control method of any one of the above. Since the principle of solving problems in the embodiments of this computer-readable storage medium is similar to that of the above method embodiments, the implementation of the embodiments of this computer-readable storage medium can refer to the implementation of the above method embodiments, and the repeated parts will not be elaborated.

[0116] More specific examples of the computer-readable storage medium in the present disclosure may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0117] In the present disclosure, the computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, and this readable medium may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0118] Optionally, the program code contained on the computer-readable storage medium may be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0119] In specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages - such as Java, C++, etc., and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0120] Based on the same inventive concept, embodiments of the present disclosure also provide a computer program product, including: a computer program or instruction, which, when executed by a processor, implements the aircraft ground service equipment service connection safety interlock control method of any one of the above method embodiments. Since the principle of solving problems in this computer program product embodiment is similar to that of the above method embodiments, the implementation of this computer program product embodiment can refer to the implementation of the above method embodiments, and the repeated parts will not be elaborated.

[0121] It should be noted that although several modules or units of the devices for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-mentioned modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0122] In addition, although the steps of the methods in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in this specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0123] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0124] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

Claims

1. A method for safety interlock control of a service connection of aircraft ground service equipment, characterized in that, Including: Obtain a real-time image of the target area; Perform detection and recognition on the real-time image to obtain a recognition result of the service joint status of the ground service equipment; Based on the recognition result of the service joint status of the ground service equipment, output an interlock control instruction, where the interlock control instruction is used to control the aircraft and / or the ground service equipment.

2. The method for safety interlock control of the service connection of aircraft ground service equipment according to claim 1, characterized in that, The target area includes the aircraft ground service interface area and / or the service joint storage bin area of the ground service equipment. The obtaining of the real-time image of the target area includes: Collect real-time multi-spectral images of the aircraft ground service interface area and / or the service joint storage bin area of the ground service equipment.

3. The aircraft ground service equipment servicing joint safety interlock control method according to claim 2, wherein When the target area is the aircraft ground service interface area, the performing detection and recognition on the real-time image to obtain the recognition result of the service joint status of the ground service equipment includes: Input the real-time multi-spectral image of the aircraft ground service interface area into a visual detection model, and output the recognition result of the service joint status of the ground service equipment, where the visual detection model is trained according to the historical multi-spectral images of the aircraft ground service interface area.

4. The aircraft ground service equipment servicing joint safety interlock control method according to claim 3, wherein The inputting the real-time multi-spectral image of the aircraft ground service interface area into the visual detection model and outputting the recognition result of the service joint status of the ground service equipment includes: Extract mixed features in the real-time multi-spectral image of the aircraft ground service interface area, where the mixed features include at least one of the following: color feature, texture feature, and shape feature; Determine whether there is a service joint of the ground service equipment in the real-time multi-spectral image according to at least one of the color feature, texture feature, and shape feature; When there is a service joint of the ground service equipment in the real-time multi-spectral image, output a first joint status recognition result; When there is no service joint of the ground service equipment in the real-time multi-spectral image, output a second joint status recognition result.

5. The aircraft ground service equipment servicing joint safety interlock control method according to claim 2, characterized in that, When the target area is the service joint storage bin area of the ground service equipment, the performing detection and recognition on the real-time image to obtain the recognition result of the service joint status of the ground service equipment includes: Perform detection and recognition on the real-time multi-spectral image of the service joint storage bin area of the ground service equipment to determine whether there is preset reference information in the real-time multi-spectral image of the service joint storage bin area of the ground service equipment, where the preset reference information includes at least one of the following: the service joint of the ground service equipment, the two-dimensional code on the service joint of the ground service equipment, and the color card identifier on the service joint of the ground service equipment; When there is preset reference information in the real-time multi-spectral image of the service joint storage bin area of the ground service equipment, obtain a third joint status recognition result; When there is no preset reference information in the real-time multi-spectral image of the service joint storage bin area of the ground service equipment, obtain a fourth joint status recognition result.

6. The aircraft ground service equipment servicing joint safety interlock control method according to claim 2, wherein The performing detection and recognition on the real-time image to obtain the recognition result of the service joint status of the ground service equipment includes: Perform detection and recognition on the real-time multi-spectral image of the aircraft ground service interface area to obtain a first sub-joint status recognition result; Detect and identify the real-time multi-spectral image of the area of the service connection storage bin of the ground service equipment to obtain the recognition result of the second sub-connection state; Obtain the recognition result of the service connection state of the ground service equipment according to the recognition result of the first sub-connection state and the recognition result of the second sub-connection state.

7. The method for controlling the safety interlock of the service connection of the aircraft ground service equipment according to claim 1, wherein the interface for outputting the interlock control instruction supports digital quantity signals, serial remote terminal units / transmission control protocols, and programmable logic controller instructions.

8. A safety interlock control device for a servicing connector of aircraft ground service equipment, characterized in that, The device includes: An acquisition module, configured to acquire a real-time image of a target area; A detection and identification module, configured to detect and identify the real-time image to obtain the recognition result of the service connection state of the ground service equipment; An output module, configured to output an interlock control instruction based on the recognition result of the service connection state of the ground service equipment, wherein the interlock control instruction is used to control the aircraft and / or the ground service equipment.

9. An electronic device, characterized in that, including: A processor; and A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the method for controlling the safety interlock of the service connection of the aircraft ground service equipment according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for controlling the safety interlock of the service connection of the aircraft ground service equipment according to any one of claims 1 to 7.

Images