Collaborative decision-making method, device and equipment for airports in gallery bridge

By using improved image processing technology and feature recognition methods in the A-CDM system, the problem of low recognition accuracy of aviation staff and passengers in the bridge is solved, and accurate identification of aviation staff and passengers and working status monitoring is achieved. , improves information sharing and flight operation efficiency of coordinated decision-making at airports.

CN119090285APending Publication Date: 2024-12-06GAOZHONG INFORMATION TECH GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410277066.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-03-12
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing A-CDM system cannot effectively obtain the working status information of aviation staff and passengers in the bridge, resulting in insufficient resource utilization and low flight operation efficiency. The recognition accuracy is low due to similar personnel dresses, which affects information sharing.

Method used

By obtaining image data of the bridge docking scene, target detection and feature extraction are performed, the improved Yolov5 network and adaptive feature fusion network are used to determine the location and category of the target box, and accurately identify the pre-stored bottom library features, and target tracking is performed. To determine the working status information, it is synchronously shared with the airport collaborative decision-making system.

Benefits of technology

It realizes accurate identification and working status monitoring of aviation staff and passengers, improves information sharing and flight operation efficiency in coordinated decision-making at airports, and ensures reasonable allocation of resources and maximum flight efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090285A_ABST
    Figure CN119090285A_ABST
Patent Text Reader

Abstract

The embodiment of the invention relates to the technical field of video monitoring, and discloses a collaborative decision-making method for an airport in a gallery bridge, and the method comprises the steps: obtaining image data in a target scene; the target scene is a gallery bridge docking scene; performing target detection on the image data to obtain a target frame image; the target frame image is a human body image of which the image data comprises wearing features; extracting target frame features of the target frame image; determining a target category of the target frame image according to the target frame feature and a pre-stored base library feature; the pre-stored bottom library features are determined in advance according to image data of aviation workers and passengers; the target category comprises aviation workers and passengers; performing target tracking on the target in the target frame image according to the target category, and determining working state information of the target; and according to the working state information, carrying out airport collaborative decision-making through an airport collaborative decision-making system. Through the above mode, the method achieves the recognition of aviation workers and passengers in the gallery bridge, and carries out the collaborative decision-making of an airport.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of deep learning and intelligent video surveillance, and specifically to a method, device, equipment and computer-readable storage medium for collaborative decision-making in an airport within a jet bridge. Background Art

[0002] As my country's economy continues to develop at a high speed, air passenger transport has become increasingly important, and the safety and efficiency of airport operations, which are directly related to this, have also received increasing attention. How to improve the efficiency of airport operations and the flexibility to deal with disturbances has become crucial. A-CDM (Airport Collaborative Decision-Making System) integrates airports, air traffic control, airlines and other relevant parties into a unified platform to achieve the purpose of collaborative decision-making in airport operations management. A-CDM realizes coordinated operations between airports, air traffic control and airlines centered on the airport. The airport collaborative decision-making system integrates data from all parties and shares the data to achieve the rational allocation of resources of various operators centered on the airport, improve overall operating efficiency and ensure flight operations.

[0003] However, the existing A-CDM cannot obtain the working status information of the aviation staff and passengers after the aircraft hatch is received, such as the passengers starting to disembark, the passengers completing disembarkation, the boarding start, the boarding end, the crew arriving, the cleaning start, the cleaning end, etc., so A-CDM cannot complete the corresponding airport collaborative decision-making in time through information sharing, the resource utilization rate is insufficient, and the maximum flight operation efficiency cannot be achieved. In addition, the inventor of this application found that some aviation staff and passengers are dressed very similarly, such as the suits of aviation service personnel, the white shirts of crew members, the work clothes of cleaning staff, etc., which will cause misdetection and false detection, thereby affecting the information sharing of A-CDM. The existing technology lacks effective means of identifying aviation staff and passengers in the docking scenario of the jet bridge. Summary of the invention

[0004] In view of the above problems, an embodiment of the present invention provides an airport collaborative decision-making method, device, equipment and computer-readable storage medium within the jet bridge, which are used to solve the problems in the prior art of lack of airport collaborative decision-making based on image data within the jet bridge, and low accuracy in identifying aviation staff and passengers.

[0005] According to one aspect of an embodiment of the present invention, a method for collaborative decision-making in an airport within a corridor is provided, the method comprising:

[0006] Acquire image data of a target scene; the target scene is a corridor bridge docking scene;

[0007] Performing target detection on the image data to obtain a target frame image; the target frame image is a human body image including wearing features in the image data;

[0008] Extracting target frame features of the target frame image;

[0009] Determining the target category of the target frame image according to the target frame features and pre-stored base database features; the pre-stored base database features are pre-determined according to the image data of the aviation staff and the passengers; the target categories include aviation staff and passengers;

[0010] Tracking the target in the target frame image according to the target category to determine the working state information of the target;

[0011] Airport collaborative decision-making is performed through an airport collaborative decision-making system according to the working status information.

[0012] In an optional manner, tracking the target in the target frame image according to the target category to determine the working status information of the target includes:

[0013] Tracking the target in the target frame image according to the target category to obtain trajectory information of the target;

[0014] The working state information of the target is determined according to the target category and the trajectory information of the target.

[0015] In an optional manner, performing airport collaborative decision-making through an airport collaborative decision-making system according to the working status information includes:

[0016] Determine the transfer process information of the current flight according to the working status information;

[0017] Determine the airport operation management plan based on the transfer process information of the current flight;

[0018] Conduct airport collaborative decision making based on the airport operation management plan.

[0019] In an optional manner, performing target detection on the image data to obtain a target frame image includes:

[0020] Performing target detection on the image data to obtain position information of the target frame;

[0021] The target frame image is determined from the image data according to the position information of the target frame.

[0022] In an optional manner, the position information of the target box is obtained based on an improved Yolov5 network, and the structure of the improved Yolov5 network includes an input end, BackBone, Neck and Head;

[0023] The input end is used to preprocess the image data to obtain preprocessed image data;

[0024] The BackBone includes an SPD-Conv module, and the BackBone is used to extract features from the preprocessed image data through the SPD-Conv module to obtain first feature data;

[0025] The Neck includes a multi-branch convolution module, a phantom convolution module and a transposed convolution module; the Neck performs feature fusion on the first feature data through the multi-branch convolution module, the phantom convolution module and the transposed convolution module to obtain second feature data of three different levels;

[0026] The Head includes an adaptive feature fusion network, and the Head is used to perform feature fusion on the second feature data of three different levels through the adaptive feature fusion network to obtain fusion features of three different levels, and obtain the position information of the target frame according to the fusion features of three different levels.

[0027] In an optional manner, determining the target category of the target frame image according to the target frame feature and the pre-stored base library feature includes:

[0028] Calculating a first similarity between the target frame feature and the pre-stored base database feature;

[0029] If the first similarity is greater than a preset threshold, the target category of the target frame image is determined according to the first similarity.

[0030] In an optional manner, the pre-stored base library feature includes a plurality of base library sub-features, the target category includes a plurality of target sub-categories, and the first base library sub-feature corresponds to one target sub-category; and determining the target category of the target frame image according to the target frame feature and the pre-stored base library feature includes:

[0031] Calculating a second similarity between the target frame feature and each of the base library sub-features;

[0032] Determine the second similarity with the largest value as the first similarity;

[0033] If the first similarity is greater than a preset threshold, the target category of the target frame image is determined according to the base library sub-feature and the sub-category corresponding to the first similarity.

[0034] According to another aspect of an embodiment of the present invention, a device for collaborative decision making in an airport in a corridor is provided, the device comprising:

[0035] A data acquisition module is used to acquire image data in a target scene; the target scene is a corridor bridge docking scene;

[0036] A target detection module, used to perform target detection on the image data to obtain a target frame image; the target frame image is a human body image including clothing features in the image data;

[0037] A feature extraction module, used for extracting target frame features of the target frame image;

[0038] A target determination module, used to determine the target category of the target frame image according to the target frame features and pre-stored base library features; the pre-stored base library features are pre-determined according to the image data of the aviation staff and passengers; the target categories include aviation staff and passengers;

[0039] A target tracking module, used to track the target in the target frame image according to the target category, and determine the working state information of the target;

[0040] The airport collaborative decision-making module is used to perform airport collaborative decision-making through the airport collaborative decision-making system according to the working status information.

[0041] According to another aspect of an embodiment of the present invention, there is provided an airport collaborative decision-making device in a corridor bridge, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0042] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the airport collaborative decision-making method within the jet bridge as described in any one of the above.

[0043] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein at least one executable instruction is stored in the storage medium. When the executable instruction is executed on an in-bridge airport collaborative decision-making device, the in-bridge airport collaborative decision-making device enables the in-bridge airport collaborative decision-making device to perform operations of the in-bridge airport collaborative decision-making method as described in any one of the above.

[0044] The embodiment of the present invention determines the target category of the target frame image through the target frame features and the pre-stored bottom-up data, and can realize the accurate identification of aviation staff and passengers in the jet bridge docking scenario, so that the target category can be synchronously shared to the airport collaborative decision-making system, facilitating the formulation and execution of airport collaborative decision-making.

[0045] Furthermore, the embodiment of the present invention tracks the target in the target frame image, determines the working status information of the target, and makes airport collaborative decisions based on the working status information. It can monitor the jet bridge docking scene in real time during the entire transfer operation process and make decisions in time to achieve maximum operating efficiency of the flight.

[0046] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0048] Figure 1 A schematic diagram of the process of the airport collaborative decision-making method in the corridor provided by an embodiment of the present invention is shown;

[0049] Figure 2 A schematic diagram showing the structure of an improved Yolov5 network provided by an embodiment of the present invention is shown;

[0050] Figure 3 A schematic diagram of the structure of an airport collaborative decision-making device in a corridor bridge provided by an embodiment of the present invention is shown;

[0051] Figure 4 A schematic diagram of the structure of an airport collaborative decision-making device in a jet bridge provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0052] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0053] Figure 1 The flowchart of the airport collaborative decision-making method in the corridor provided by the embodiment of the present invention is shown. The method is executed by the airport collaborative decision-making device in the corridor. The device can be an intelligent terminal device, tablet, computer, etc. installed in the airport. The device is equipped with an airport collaborative decision-making system. Figure 1 As shown, the method comprises the following steps:

[0054] Step 110: Acquire image data of a target scene; the target scene is a corridor bridge docking scene.

[0055] The image data may be acquired by a camera device disposed in the corridor bridge docking scene. Specifically, the camera device is used to acquire a video stream of the corridor bridge docking scene, and step 110 further includes parsing the video stream into image data.

[0056] Step 120: Perform target detection on the image data to obtain a target frame image; the target frame image is a human body image including clothing features in the image data.

[0057] The performing target detection on the image data to obtain a target frame image specifically includes: performing target detection on the image data to obtain position information of the target frame; and determining the target frame image from the image data according to the position information of the target frame.

[0058] The location information of the target box is obtained based on the improved Yolov5 network, please refer to Figure 2 , Figure 2 The schematic diagram of the structure of the improved Yolov5 network provided by the embodiment of the present invention is shown. The structure of the improved Yolov5 network includes an input end, a BackBone, a Neck and a Head.

[0059] The input end is used to preprocess the image data to obtain preprocessed image data. Specifically, the input end preprocesses the image data by Mosaic data enhancement, adaptive anchor frame calculation, adaptive image scaling and other methods to obtain preprocessed image data. Specifically, the preprocessed image data is a 640*640 three-channel image, that is, 3*640*640 matrix data.

[0060] Among them, the BackBone includes a ConvBNSILU module, a DCNv2 module, a SPD-Conv module, a C3 (CSPBottleneck with 3 convolutions) module and a SPPF (Spatial Pyramid Pooling-Fast) module. The BackBone is used to extract features from the preprocessed image data through the ConvBNSILU module, the DCNv2 module, the SPD-Conv module, the C3 module and the SPPF module to obtain first feature data; the SPD-Conv module includes an SPD (depth) layer and a Conv (non-Stride convolution) layer.

[0061] The embodiment of the present invention integrates the SPD-Conv module on the basis of the original BackBone, and rearranges the spatial blocks of the image data into the depth dimension through the SPD layer, which can reduce the spatial resolution while increasing the number of channels without losing information; the Conv layer uses a convolution operation with a step size of 1, which means that during the convolution process, the convolution kernel will move pixel by pixel on the input image data without skipping any pixels, which can retain information to the greatest extent and obtain better feature expression. Especially when processing small objects or low-resolution images, important information is often lost in the traditional CNN structure, and the present invention improves the representation of features by applying the SPD-Conv module.

[0062] Among them, the Neck includes a multi-branch convolution module (RFBCSP), a ghost convolution (GhostConv) module and a transposed convolution (ConvTranspose2d) module; the Neck performs feature fusion on the first feature data through the multi-branch convolution module, the ghost convolution module and the transposed convolution module to obtain second feature data of three different levels.

[0063] Specifically, in an embodiment of the present invention, second feature data of three different levels, level-0, level-1 and level-2, are provided. The second feature data of each level has different spatial resolutions. For example, the second feature data of level-0 is set to a 20*20 grid, and information of 1024*20*20 large-size detection targets is output. The second feature data of level-1 is a 40*40 grid, and information of 512*40*40 medium-size detection targets is output. The second feature data of level-2 is a 80*80 grid, and information of 256*80*80 small-size detection targets is output.

[0064] The embodiment of the present invention applies a multi-branch convolution module to Neck, that is, different branches apply different convolution kernels, so as to obtain different receptive fields for obtaining higher resolution features; and the implementation of the present invention performs upsampling by applying a transposed convolution module. Compared with traditional nearest neighbor interpolation, bilinear difference and other methods, the parameters of the transposed convolution module are learnable and do not require some artificially predefined methods.

[0065] Wherein, the Head includes an adaptive feature fusion network (ASFF), and the Head is used to perform feature fusion on the second feature data of three different levels through the adaptive feature fusion network to obtain fusion features of three different levels, and obtain the position information of the target frame according to the fusion features of three different levels. Specifically, the adaptive feature fusion network includes ASFF-1, ASFF-2 and ASFF-3 with similar network structures, representing the feature fusion of different levels using the ASFF mechanism; ASFF-1 performs feature fusion on the second feature data of three different levels of level-0, level-1 and level-2, ASFF-2 performs feature fusion on the second feature data of three different levels of level-0, level-1 and level-2, and ASFF-3 performs feature fusion on the second feature data of three different levels of level-0, level-1 and level-2 to obtain fusion features of three different levels.

[0066] The obtaining of the position information of the target frame according to the fused features at three different levels specifically includes the following steps: weighted fusion of the fused features at three different levels to generate features for prediction, and obtaining the position information (x1, y1, x2, y2) of the target frame according to the feature prediction, where (x1, y1, x2, y2) represents the upper left corner coordinate information (x1, y1) and the lower right corner coordinate information (x2, y2) of the target frame.

[0067] Wherein, determining the target frame image from the image data according to the position information of the target frame specifically includes: according to the position information (x1, y1, x2, y2) of the target frame, intercepting the image at the position of the target frame from the image data to obtain the target frame image.

[0068] The embodiment of the present invention replaces the traditional Detect detection head with an adaptive feature fusion network, which can filter conflicting information in the spatial domain to suppress inconsistent features, improve the network's feature fusion capability for targets of different scales, and enable the network to directly learn how to spatially filter features at other levels, thereby retaining only useful information for combination.

[0069] Step 130: extracting target frame features of the target frame image.

[0070] Among them, step 130 is to extract the target box features of the target box image through a classification network. Specifically, the classification network is a VGG16 (Visual Geometry Group-16) network. In addition to the datasets of aviation staff and passengers in the apron bridge docking scenario, the VGG16 network also adds STL-10, CIFAR-100, CIFAR-10, and ImageNet datasets for training.

[0071] Among them, the loss function of the VGG16 network adopts Circle Loss, and the loss function L circle is as follows:

[0072]

[0073] Among them, K is the number of positive samples, L is the number of negative samples, α j n and α j p are non-negative weight factors, γ is a scaling factor, s p is the intra-class similarity, s n is the inter-class similarity, is the i-th inter-class similarity, is the j-th inter-class similarity.

[0074] Define the optimal value of s p as Op, the optimal value of s n as On, and On < Op; when a similarity score s p or s n deviates far from the optimal value Op or On, Circle Loss will assign a large weight to further adaptively optimize and update it. Among them, the definition of self-paced is as follows:

[0075]

[0076]

[0077] Among them, [·] + is a 0 truncation operation to ensure that α j n and α j p are non-negative. Circleloss then performs an independent weighting based on the classification value of each similarity, allowing different learning amplitudes, enabling each similarity score to adjust the learning amplitude according to the current optimization state, and being more flexible.

[0078] The present invention adds multiple data sets and performs transfer learning based on the VGG16 network, so that the trained VGG16 network can efficiently extract the characteristics of aviation staff and passengers.

[0079] Step 140: Determine the target category of the target frame image based on the target frame features and pre-stored base database features; the pre-stored base database features are pre-determined based on image data of airline staff and passengers; the target categories include airline staff and passengers.

[0080] Among them, the pre-stored bottom database features are the last layer features extracted by the classification network based on the image data of aviation staff and passengers; the aviation staff include flight attendants, cleaning staff, aviation agents, security officers, firefighters and other crew members and maintenance personnel.

[0081] Among them, determining the target category of the target frame image based on the target frame features and the pre-stored base library features specifically includes: calculating a first similarity between the target frame features and the pre-stored base library features; if the first similarity is greater than a preset threshold, determining the target category of the target frame image based on the first similarity.

[0082] Among them, the calculation formula of the first similarity cos(θ) is as follows:

[0083]

[0084] Among them, ɑ is the target box feature vector, and b is the pre-stored base library feature vector.

[0085] Among them, the pre-stored base database feature includes multiple base database sub-features, the target category includes multiple target sub-categories, and the first base database sub-feature corresponds to one target sub-category; determining the target category of the target frame image according to the target frame feature and the pre-stored base database feature includes: calculating the second similarity between the target frame feature and each of the base database sub-features; determining the second similarity with the largest value as the first similarity; if the first similarity is greater than a preset threshold, determining the target category of the target frame image according to the base database sub-feature and the sub-category corresponding to the first similarity.

[0086] Preferably, the calculated second similarities are sorted from large to small, and TOP1 with the largest value is taken as the first similarity.

[0087] In the embodiment of the present invention, the preset threshold is set to 0.7.

[0088] Wherein, after determining the target category of the target frame image according to the base library sub-features and the sub-category corresponding to the first similarity, step 140 further includes: synchronizing the target frame features of the target category of the target frame image to the airport collaborative decision-making system.

[0089] The embodiment of the present invention determines whether a target hits a category in the base database data by a feature similarity calculation method, which can better compensate for the fault tolerance of aviation staff and passenger detection.

[0090] The embodiments of the present invention can improve information sharing with the airport collaborative decision-making system by accurately identifying aviation staff and passengers in the jet bridge and synchronizing the data to the airport collaborative decision-making system in a timely manner, and the application of feature similarity calculation method can better compensate for the fault tolerance of aviation staff and passenger detection.

[0091] Step 150: Track the target in the target frame image according to the target category to determine the working status information of the target.

[0092] Among them, tracking the target in the target frame image according to the target category to determine the working status information of the target includes: tracking the target in the target frame image according to the target category to obtain trajectory information of the target; determining the working status information of the target according to the target category and the trajectory information of the target.

[0093] The working status information of the target includes the start of passenger boarding, the end of passenger boarding, the start of passenger disembarkation, the end of passenger disembarkation, the start of cleaning, the end of cleaning, the arrival of the crew, etc.

[0094] In an optional manner, step 150 can obtain the final position of the target through target tracking, and determine the working status information of the target based on the final position. For example, when it is detected that the final position of the passenger is inside the aircraft, the passenger's working status is judged to be the start of boarding; when it is detected that the final positions of all passengers are inside the aircraft or arrive at the scheduled time, the passenger's working status is judged to be the end of boarding; when it is detected that the final position of the cleaning staff is inside the aircraft, the cleaning staff's working status is judged to be the start of cleaning; when it is detected that the final position of the cleaning staff is inside the jet bridge, the cleaning staff's working status is judged to be the end of cleaning, which are not listed one by one in the present invention.

[0095] Step 160: Perform airport collaborative decision-making through the airport collaborative decision-making system based on the working status information.

[0096] Among them, the airport collaborative decision-making is performed through the airport collaborative decision-making system based on the working status information, including: determining the transfer process information of the current flight based on the working status information; determining the airport operation management plan based on the transfer process information of the current flight; and performing airport collaborative decision-making based on the airport operation management plan.

[0097] The transition process information includes the start of passenger boarding, the end of passenger boarding, the start of cleaning, the end of cleaning, the arrival of the crew, etc.

[0098] Among them, the airport collaborative decision-making includes event reminders. Specifically, according to the working status information, the transfer process information of the current flight is determined; according to the transfer process information of the current flight, the airport operation management plan is determined; if the transfer process information of the current flight is normal, the airport operation management plan is determined to be a normal process reminder, and according to the airport operation management plan, event reminders are performed to facilitate the rapid handover of the transfer process; if the transfer process information of the current flight is abnormal, the airport operation management plan is determined to be an abnormal process reminder, and according to the airport operation management plan, event reminders are performed to facilitate the airport to quickly investigate and resolve abnormalities and avoid the situation where flights cannot complete the transfer process on time. For example, when it is detected that passengers have started boarding, the boarding status of passengers in the jet bridge is monitored in real time by video cameras until the end of boarding, to determine whether all passengers have arrived. If not all passengers have arrived, the crew is reminded that there is a passenger non-arrival event, so that the crew can check the boarding status of passengers in time. For another example, when it is detected that cleaning has started, the flow of people in the jet bridge is monitored in real time by video cameras until the end of cleaning, to determine whether all cleaning staff have left. If not all cleaning staff have left, the crew is reminded to urge them in time, and to determine whether there are non-cleaning staff entering or leaving during the cleaning period. If so, the crew is reminded to queue up in time to correct the abnormality.

[0099] The embodiment of the present invention tracks the target in the target frame image, determines the working status information of the target, and makes airport collaborative decisions based on the working status information. It can monitor the jet bridge docking scene in real time during the entire transfer operation process and make decisions in time to achieve maximum flight operation efficiency.

[0100] Figure 3 FIG. 1 shows a schematic diagram of the structure of the airport collaborative decision-making in the corridor provided by an embodiment of the present invention. Figure 3 As shown, the device 300 includes: a data acquisition module 310, a target detection module 320, a feature extraction module 330, a target determination module 340, a target tracking module 350 and an airport collaborative decision-making module 360.

[0101] The data acquisition module 310 is used to acquire image data in a target scene; the target scene is a corridor bridge docking scene;

[0102] The target detection module 320 is used to perform target detection on the image data to obtain a target frame image; the target frame image is a human body image including wearing features in the image data;

[0103] Wherein, the feature extraction module 330 is used to extract the target frame features of the target frame image;

[0104] The target determination module 340 is used to determine the target category of the target frame image according to the target frame features and pre-stored base library features; the pre-stored base library features are pre-determined according to the image data of the aviation staff and passengers; the target categories include aviation staff and passengers;

[0105] The target tracking module 350 is used to track the target in the target frame image according to the target category and determine the working status information of the target;

[0106] The airport collaborative decision module 360 ​​is used to perform airport collaborative decision-making through the airport collaborative decision system according to the working status information.

[0107] In an optional manner, the target is tracked in the target frame image according to the target category to determine the working status information of the target, including: tracking the target in the target frame image according to the target category to obtain trajectory information of the target; and determining the working status information of the target according to the target category and the trajectory information of the target.

[0108] In an optional manner, the airport collaborative decision-making is performed through an airport collaborative decision-making system based on the working status information, including: determining the transfer process information of the current flight based on the working status information; determining the airport operation management plan based on the transfer process information of the current flight; and performing airport collaborative decision-making based on the airport operation management plan.

[0109] In an optional manner, performing target detection on the image data to obtain a target frame image includes: performing target detection on the image data to obtain position information of the target frame; and determining the target frame image from the image data based on the position information of the target frame.

[0110] In an optional manner, the position information of the target frame is obtained based on an improved Yolov5 network, and the structure of the improved Yolov5 network includes an input end, BackBone, Neck and Head; the input end is used to preprocess the image data to obtain preprocessed image data; the BackBone includes an SPD-Conv module, and the BackBone is used to extract features of the preprocessed image data through the SPD-Conv module to obtain first feature data; the Neck includes a multi-branch convolution module, a phantom convolution module and a transposed convolution module; the Neck performs feature fusion on the first feature data through the multi-branch convolution module, the phantom convolution module and the transposed convolution module to obtain second feature data of three different levels; the Head includes an adaptive feature fusion network, and the Head is used to perform feature fusion on the second feature data of three different levels through the adaptive feature fusion network to obtain fusion features of three different levels, and obtain the position information of the target frame according to the fusion features of three different levels.

[0111] In an optional manner, determining the target category of the target frame image based on the target frame features and pre-stored base library features includes: calculating a first similarity between the target frame features and the pre-stored base library features; if the first similarity is greater than a preset threshold, determining the target category of the target frame image based on the first similarity.

[0112] In an optional manner, the pre-stored base database feature includes multiple base database sub-features, the target category includes multiple target sub-categories, and the first base database sub-feature corresponds to one target sub-category; determining the target category of the target frame image based on the target frame feature and the pre-stored base database feature includes: calculating the second similarity between the target frame feature and each of the base database sub-features; determining the second similarity with the largest value as the first similarity; if the first similarity is greater than a preset threshold, determining the target category of the target frame image based on the base database sub-feature and the sub-category corresponding to the first similarity.

[0113] The embodiment of the present invention determines the target category of the target frame image through the target frame features and the pre-stored bottom-up data, and can realize the accurate identification of aviation staff and passengers in the jet bridge docking scenario, so that the target category can be synchronously shared to the airport collaborative decision-making system, facilitating the formulation and execution of airport collaborative decision-making.

[0114] Furthermore, the embodiment of the present invention tracks the target in the target frame image, determines the working status information of the target, and makes airport collaborative decisions based on the working status information. It can monitor the jet bridge docking scene in real time during the entire transfer operation process and make decisions in time to achieve maximum operating efficiency of the flight.

[0115] Figure 4 A schematic diagram of the structure of an airport collaborative decision-making device in a jet bridge provided by an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the airport collaborative decision-making device in a jet bridge.

[0116] like Figure 4 As shown, the airport collaborative decision-making device in the jet bridge may include: a processor 402 , a communications interface 404 , a memory 406 , and a communication bus 408 .

[0117] The processor 402, the communication interface 404, and the memory 406 communicate with each other via the communication bus 408. The communication interface 404 is used to communicate with other devices such as a client or other server network elements. The processor 402 is used to execute the program 410, which can specifically execute the relevant steps in the above-mentioned embodiment of the airport collaborative decision-making method in the corridor.

[0118] Specifically, the program 410 may include program code including computer executable instructions.

[0119] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention. The one or more processors included in the airport collaborative decision-making device in the air bridge may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0120] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0121] Program 410 can be specifically called by processor 402 to enable the airport collaborative decision-making device in the jet bridge to perform the following operations:

[0122] Acquire image data of a target scene; the target scene is a corridor bridge docking scene;

[0123] Performing target detection on the image data to obtain a target frame image; the target frame image is a human body image including wearing features in the image data;

[0124] Extracting target frame features of the target frame image;

[0125] Determining the target category of the target frame image according to the target frame features and pre-stored base database features; the pre-stored base database features are pre-determined according to the image data of the aviation staff and the passengers; the target categories include aviation staff and passengers;

[0126] Tracking the target in the target frame image according to the target category to determine the working state information of the target;

[0127] Airport collaborative decision-making is performed through an airport collaborative decision-making system according to the working status information.

[0128] In an optional manner, the target is tracked in the target frame image according to the target category to determine the working status information of the target, including: tracking the target in the target frame image according to the target category to obtain trajectory information of the target; and determining the working status information of the target according to the target category and the trajectory information of the target.

[0129] In an optional manner, the airport collaborative decision-making is performed through an airport collaborative decision-making system based on the working status information, including: determining the transfer process information of the current flight based on the working status information; determining the airport operation management plan based on the transfer process information of the current flight; and performing airport collaborative decision-making based on the airport operation management plan.

[0130] In an optional manner, performing target detection on the image data to obtain a target frame image includes: performing target detection on the image data to obtain position information of the target frame; and determining the target frame image from the image data based on the position information of the target frame.

[0131] In an optional manner, the position information of the target frame is obtained based on an improved Yolov5 network, and the structure of the improved Yolov5 network includes an input end, BackBone, Neck and Head; the input end is used to preprocess the image data to obtain preprocessed image data; the BackBone includes an SPD-Conv module, and the BackBone is used to extract features of the preprocessed image data through the SPD-Conv module to obtain first feature data; the Neck includes a multi-branch convolution module, a phantom convolution module and a transposed convolution module; the Neck performs feature fusion on the first feature data through the multi-branch convolution module, the phantom convolution module and the transposed convolution module to obtain second feature data of three different levels; the Head includes an adaptive feature fusion network, and the Head is used to perform feature fusion on the second feature data of three different levels through the adaptive feature fusion network to obtain fusion features of three different levels, and obtain the position information of the target frame according to the fusion features of three different levels.

[0132] In an optional manner, determining the target category of the target frame image based on the target frame features and pre-stored base library features includes: calculating a first similarity between the target frame features and the pre-stored base library features; if the first similarity is greater than a preset threshold, determining the target category of the target frame image based on the first similarity.

[0133] In an optional manner, the pre-stored base database feature includes multiple base database sub-features, the target category includes multiple target sub-categories, and the first base database sub-feature corresponds to one target sub-category; determining the target category of the target frame image based on the target frame feature and the pre-stored base database feature includes: calculating the second similarity between the target frame feature and each of the base database sub-features; determining the second similarity with the largest value as the first similarity; if the first similarity is greater than a preset threshold, determining the target category of the target frame image based on the base database sub-feature and the sub-category corresponding to the first similarity.

[0134] The embodiment of the present invention determines the target category of the target frame image through the target frame features and the pre-stored bottom-up data, and can realize the accurate identification of aviation staff and passengers in the jet bridge docking scenario, so that the target category can be synchronously shared to the airport collaborative decision-making system, facilitating the formulation and execution of airport collaborative decision-making.

[0135] Furthermore, the embodiment of the present invention tracks the target in the target frame image, determines the working status information of the target, and makes airport collaborative decisions based on the working status information. It can monitor the jet bridge docking scene in real time during the entire transfer operation process and make decisions in time to achieve maximum operating efficiency of the flight.

[0136] An embodiment of the present invention provides a computer-readable storage medium storing at least one executable instruction. When the executable instruction is executed on an in-bridge airport collaborative decision-making device, the in-bridge airport collaborative decision-making device enables the in-bridge airport collaborative decision-making method to execute any of the above-mentioned method embodiments.

[0137] The executable instructions can be used to enable the airport collaborative decision-making device in the bridge to perform the following operations:

[0138] Acquire image data of a target scene; the target scene is a corridor bridge docking scene;

[0139] Performing target detection on the image data to obtain a target frame image; the target frame image is a human body image including wearing features in the image data;

[0140] Extracting target frame features of the target frame image;

[0141] Determining the target category of the target frame image according to the target frame features and pre-stored base database features; the pre-stored base database features are pre-determined according to the image data of the aviation staff and the passengers; the target categories include aviation staff and passengers;

[0142] Tracking the target in the target frame image according to the target category to determine the working state information of the target;

[0143] Airport collaborative decision-making is performed through an airport collaborative decision-making system according to the working status information.

[0144] In an optional manner, the target is tracked in the target frame image according to the target category to determine the working status information of the target, including: tracking the target in the target frame image according to the target category to obtain trajectory information of the target; and determining the working status information of the target according to the target category and the trajectory information of the target.

[0145] In an optional manner, the airport collaborative decision-making is performed through an airport collaborative decision-making system based on the working status information, including: determining the transfer process information of the current flight based on the working status information; determining the airport operation management plan based on the transfer process information of the current flight; and performing airport collaborative decision-making based on the airport operation management plan.

[0146] In an optional manner, performing target detection on the image data to obtain a target frame image includes: performing target detection on the image data to obtain position information of the target frame; and determining the target frame image from the image data based on the position information of the target frame.

[0147] In an optional manner, the position information of the target frame is obtained based on an improved Yolov5 network, and the structure of the improved Yolov5 network includes an input end, BackBone, Neck and Head; the input end is used to preprocess the image data to obtain preprocessed image data; the BackBone includes an SPD-Conv module, and the BackBone is used to extract features of the preprocessed image data through the SPD-Conv module to obtain first feature data; the Neck includes a multi-branch convolution module, a phantom convolution module and a transposed convolution module; the Neck performs feature fusion on the first feature data through the multi-branch convolution module, the phantom convolution module and the transposed convolution module to obtain second feature data of three different levels; the Head includes an adaptive feature fusion network, and the Head is used to perform feature fusion on the second feature data of three different levels through the adaptive feature fusion network to obtain fusion features of three different levels, and obtain the position information of the target frame according to the fusion features of three different levels.

[0148] In an optional manner, determining the target category of the target frame image based on the target frame features and pre-stored base library features includes: calculating a first similarity between the target frame features and the pre-stored base library features; if the first similarity is greater than a preset threshold, determining the target category of the target frame image based on the first similarity.

[0149] In an optional manner, the pre-stored base database feature includes multiple base database sub-features, the target category includes multiple target sub-categories, and the first base database sub-feature corresponds to one target sub-category; determining the target category of the target frame image based on the target frame feature and the pre-stored base database feature includes: calculating the second similarity between the target frame feature and each of the base database sub-features; determining the second similarity with the largest value as the first similarity; if the first similarity is greater than a preset threshold, determining the target category of the target frame image based on the base database sub-feature and the sub-category corresponding to the first similarity.

[0150] The embodiment of the present invention determines the target category of the target frame image through the target frame features and the pre-stored bottom-up data, and can realize the accurate identification of aviation staff and passengers in the jet bridge docking scenario, so that the target category can be synchronously shared to the airport collaborative decision-making system, facilitating the formulation and execution of airport collaborative decision-making.

[0151] Furthermore, the embodiment of the present invention tracks the target in the target frame image, determines the working status information of the target, and makes airport collaborative decisions based on the working status information. It can monitor the jet bridge docking scene in real time during the entire transfer operation process and make decisions in time to achieve maximum operating efficiency of the flight.

[0152] An embodiment of the present invention provides an in-bridge airport collaborative decision-making device, which is used to execute the in-bridge airport collaborative decision-making method mentioned above.

[0153] An embodiment of the present invention provides a computer program, which can be called by a processor to enable an in-bridge airport collaborative decision-making device to execute the in-bridge airport collaborative decision-making method in any of the above method embodiments.

[0154] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are run on a computer, the computer executes the XXXX method in any of the above method embodiments.

[0155] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0156] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0157] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than those expressly recited in each claim.

[0158] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0159] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A collaborative decision-making method for an airport in a corridor bridge, characterized in that: The method comprises: Acquire image data of a target scene; the target scene is a corridor bridge docking scene; Performing target detection on the image data to obtain a target frame image; the target frame image is a human body image including wearing features in the image data; Extracting target frame features of the target frame image; Determining the target category of the target frame image according to the target frame features and pre-stored base database features; the pre-stored base database features are pre-determined according to the image data of the aviation staff and the passengers; the target categories include aviation staff and passengers; Tracking the target in the target frame image according to the target category to determine the working state information of the target; Airport collaborative decision-making is performed through an airport collaborative decision-making system according to the working status information.

2. The method according to claim 1, characterized in that: Tracking the target in the target frame image according to the target category to determine the working status information of the target includes: Tracking the target in the target frame image according to the target category to obtain trajectory information of the target; The working state information of the target is determined according to the target category and the trajectory information of the target.

3. The method according to claim 1, characterized in that: The performing airport collaborative decision-making by an airport collaborative decision-making system according to the working status information includes: Determine the transfer process information of the current flight according to the working status information; Determine the airport operation management plan based on the transfer process information of the current flight; Conduct airport collaborative decision making based on the airport operation management plan.

4. The method according to claim 1, characterized in that: The performing target detection on the image data to obtain a target frame image includes: Performing target detection on the image data to obtain position information of the target frame; The target frame image is determined from the image data according to the position information of the target frame.

5. The method according to claim 4, characterized in that The position information of the target frame is obtained based on an improved Yolov5 network, wherein the structure of the improved Yolov5 network includes an input end, a BackBone, a Neck, and a Head; The input end is used to preprocess the image data to obtain preprocessed image data; The BackBone includes an SPD-Conv module, and the BackBone is used to extract features from the preprocessed image data through the SPD-Conv module to obtain first feature data; The Neck includes a multi-branch convolution module, a phantom convolution module and a transposed convolution module; the Neck performs feature fusion on the first feature data through the multi-branch convolution module, the phantom convolution module and the transposed convolution module to obtain second feature data of three different levels; The Head includes an adaptive feature fusion network, and the Head is used to perform feature fusion on the second feature data of three different levels through the adaptive feature fusion network to obtain fusion features of three different levels, and obtain the position information of the target frame according to the fusion features of three different levels.

6. The method according to claim 1, characterized in that The step of determining the target category of the target frame image according to the target frame feature and the pre-stored base library feature includes: Calculating a first similarity between the target frame feature and the pre-stored base database feature; If the first similarity is greater than a preset threshold, the target category of the target frame image is determined according to the first similarity.

7. The method according to claim 6, characterized in that The pre-stored base library feature includes a plurality of base library sub-features, the target category includes a plurality of target sub-categories, and the first base library sub-feature corresponds to one target sub-category; and determining the target category of the target frame image according to the target frame feature and the pre-stored base library feature includes: Calculating a second similarity between the target frame feature and each of the base library sub-features; Determine the second similarity with the largest value as the first similarity; If the first similarity is greater than a preset threshold, the target category of the target frame image is determined according to the base library sub-feature and the sub-category corresponding to the first similarity.

8. An airport collaborative decision-making device in a corridor bridge, characterized in that: The device comprises: A data acquisition module is used to acquire image data in a target scene; the target scene is a corridor bridge docking scene; A target detection module, used to perform target detection on the image data to obtain a target frame image; the target frame image is a human body image including clothing features in the image data; A feature extraction module, used for extracting target frame features of the target frame image; A target determination module, used to determine the target category of the target frame image according to the target frame features and pre-stored base library features; the pre-stored base library features are pre-determined according to the image data of the aviation staff and passengers; the target categories include aviation staff and passengers; A target tracking module, used to track the target in the target frame image according to the target category, and determine the working state information of the target; The airport collaborative decision-making module is used to perform airport collaborative decision-making through the airport collaborative decision-making system according to the working status information.

9. An airport collaborative decision-making device in a corridor bridge, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the airport collaborative decision-making method within the jet bridge as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on the in-bridge airport collaborative decision-making device, the in-bridge airport collaborative decision-making device executes the operation of the in-bridge airport collaborative decision-making method as described in any one of claims 1 to 7.