A method for identifying apron flight support services and abnormalities

Abstract

The present invention discloses a method for identifying and abnormally identifying apron flight support services, and belongs to the field of image processing technology. A method for identifying apron flight support services includes: defining support service types, service stages, entity sets, and temporal behavior patterns of each entity; obtaining a video frame sequence corresponding to each type of support service and data annotation; generating an initial graph structure for each image in the video frame sequence, and extracting graph convolution features of each node therein through a GCN network; expanding the initial graph structure to obtain an extended graph structure, forming a temporal graph structure data to train an ST‑GCN network to obtain a type recognition model, and used for the support service type and service stage recognition in the surveillance video of the apron. A method for identifying abnormalities in apron flight support services is used for identifying abnormal events in support services. Based on the extended graph structure and the ST‑GCN network, the present invention can improve the accuracy and real-time performance of recognition, and can also improve the efficiency of abnormal behavior recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for identifying apron flight support services and anomalies. Background Art

[0002] In the airport apron flight support business, ensuring the smooth completion of each task and the safety of the support process is the core task of the airport's efficient operation. Traditional flight support business node identification mainly relies on manual input of handheld terminal devices or business node information transmitted through A-CDM (collaborative decision-making system). Although this method has achieved digital management of business nodes to a certain extent, due to its reliance on manual operation, there are problems such as data input lag, large errors, and poor real-time performance, making it difficult to fully and accurately reflect the progress of the business.

[0003] In addition, most of the current flight support business monitoring systems are based on traditional fixed rule algorithms or image recognition technology, which judge a certain business status through feature extraction and analysis of single-frame images. For example, the approach of a refueling truck to an aircraft can be identified as the beginning of the refueling process. Although these methods can capture the static state of business nodes, they cannot effectively identify dynamic changes and potential illegal operations in the business process. Especially in abnormal situations such as unreasonable equipment placement and inadequate personnel, traditional monitoring systems often find it difficult to make accurate judgments in a timely manner, posing safety risks.

[0004] Existing technologies still lack the integrated use of multi-source data, and are unable to conduct comprehensive visualization analysis and comparison of video surveillance data and detection result data, making it difficult to fully reflect the dynamic changes in the business, and unable to support intuitive display of business node backtracking, anomalies, etc. Summary of the invention

[0005] The purpose of the present invention is to provide a method for identifying and safely monitoring apron flight support services, which solves the above-mentioned problems, realizes type identification of apron flight support services based on multi-source data fusion, can improve the real-time and accuracy of data, and has dynamic monitoring and abnormal behavior identification.

[0006] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is as follows: a method for identifying apron flight support services, comprising the following steps;

[0007] S1, defines C types of security services. Each type of security service is completed by several entities in cooperation. The security service is divided into three business stages: start, ongoing, and end.

[0008] If the c-type protection business GB c Corresponding to n entities, number the entities and construct GB c The entity collection , v iis the i-th entity;

[0009] Decomposition i For each action, a temporal behavior pattern is generated , is the average duration from the k-1th action to the kth action;

[0010] S2, for each type of security service, obtain several corresponding video frame sequences, one of which is , X t is the image at the tth time step in X, N is the total number of time steps, 1≤t≤N;

[0011] S3, data annotation of video frame sequence, X t The marking method is:

[0012] If X corresponds to GB c , then according to EC c X t Perform entity annotation and business stage annotation, wherein the entity annotation includes entity type and location, and the business stage annotation includes guarantee business type and business stage;

[0013] S4, the entity set EC c The entity in the middle is used as a node to generate X1~X N The initial graph structure ~ , extract the graph convolution features of each node in the initial graph structure through the GCN network, and t The initial graph structure , V is a set of nodes and V=EC c , node v in V i The initial eigenvector x i =[Type i ,C i ,A i ,P i ], Type i , C i , A i , P i They are v i type, geometric center, area and polygon boundary, v i The graph convolution feature is , E is the set of edges E={e ij},e ij is the node v in V i 、v j The edge of

[0014] S5, yes ~ Expand and obtain the corresponding extended graph structure G1~GN , forming the timing diagram structure data , which will Expanded to G t The method comprises steps S51 to S52;

[0015] S51, preset the interaction threshold and time window length T, if v j v i Neighbor nodes of X, count the nodes with X t D in the image within the time window starting from ij The number N that is less than the interaction threshold ij , and calculate e according to the following formula ij Weight ;

[0016] ,

[0017] Where D ij v i 、v j The distance, α is the spatial distance weight, γ is the interaction weight;

[0018] S52, generated The extended feature vector ,use replace Medium i The initial eigenvector x i , we get X t The extended graph structure G t , According to the following formula:

[0019] ,

[0020] In the formula, For Node The set of neighbor nodes of v j Graph convolution features;

[0021] S6, use the time-series graph structure data to train the ST-GCN network until convergence, and obtain the type recognition model. During training, The input of the tth time step is G t , the expected output is , , X t The actual type and stage of guaranteed business;

[0022] S7, use the type recognition model to identify the security service type and service stage of the images in the surveillance video of the apron.

[0023] Preferably, the support services include jet bridge docking services, refueling services, baggage loading and unloading services, cargo loading and unloading services, and in-flight catering services.

[0024] As a preferred embodiment: in S21, the video frame sequence is collected by a camera covering the apron area and obtained after video compression, format conversion and image enhancement, and each video frame in the video frame sequence contains all targets in its corresponding security business.

[0025] As a preference: X t When labeling entities, first use the target detection network to detect the entity and obtain the target detection result, then use the image segmentation network to perform semantic segmentation on the entity and obtain the semantic segmentation result;

[0026] The target detection network and the image segmentation network are pre-trained;

[0027] The target detection result includes entity category, bounding box and detection confidence, which is used for entity labeling;

[0028] The semantic segmentation result includes a set of entity contours and segmentation confidence, which are used to generate an initial feature vector of a node.

[0029] As a preferred embodiment: in S4, In the example, the set of constructed edges is: i 、v j The distance D ij is less than the distance threshold, then in v i 、v j Create an edge between ij , we get the edge set E={e ij}.

[0030] As a preferred embodiment: S7 is specifically:

[0031] Obtain surveillance video of the apron and generate a sequence of video frames to be identified;

[0032] For the image at the tth time step , S4 and S5 generate the corresponding extended graph structure, which is sent to the type recognition model to obtain The predicted output ,in , They are Forecasted assurance business types and business stages.

[0033] As a preferred embodiment: it also includes step S8, The prediction output is visualized.

[0034] A method for identifying abnormalities in a ramp flight support service, identifying abnormal events in the support service, wherein the abnormal events include unreasonable locations;

[0035] The unreasonable position identification includes steps Sa1 to Sa3;

[0036] Sa1, for each entity of each type of security service, pre-set its specified area at each service stage;

[0037] Sa2, get The prediction output is obtained according to the type of guaranteed business and business stage. the prescribed area of ​​each entity within;

[0038] Sa3, yes Perform target detection and semantic segmentation to obtain For each entity and entity outline, determine whether each entity outline is located in the specified area. If so, skip it, otherwise alarm.

[0039] Preferably, the abnormal event also includes insufficient number of people, and the insufficient number of people identification includes steps Sb1 to Sb3;

[0040] Sb1, if a type of guarantee service GB c If there is a number requirement in a business phase, then the business phase of the guarantee business is marked as a node to be detected, and the number threshold N of the node to be detected is predicted expected ;

[0041] Sb2, get The predicted output is used to determine whether it is a node to be detected. If not, skip it. , otherwise proceed to the next step;

[0042] Sb3, yes Perform target detection and obtain the number of entities whose type is person ,like , then alarm, otherwise skip .

[0043] As a preferred embodiment: the abnormal event also includes an operation sequence error, identification and guarantee service GB c The method of operating in the wrong order includes steps Sc1 to Sc3;

[0044] Sc1, will ensure business GB c Divided into S sub-actions, marked in order of precedence ~ , and set GB c Order of operations ;

[0045] Sc2, get The forecast output is: if the service GB is to be guaranteed c At the beginning stage, And subsequent images are sent to the action recognition model in sequence until The prediction output of subsequent images is to ensure the business GB c the end stage;

[0046] Sc3, the action recognition model recognizes the actions of the input images in sequence. If the sequence of recognized actions is consistent with If different, alarm will be issued.

[0047] About the neural network used in the present invention:

[0048] Object detection network: such as YOLO network or SSD network, used to identify entities in the image, such as people, airplanes, vehicles, etc. The object detection results include entity category, bounding box and detection confidence, which are used for entity labeling.

[0049] Image segmentation network: such as Mask R-CNN, which performs fine segmentation on detected entities and extracts their precise boundaries. The geometric center C i 、Area A i and the polygon boundary P i All are obtained based on the results of semantic segmentation.

[0050] GCN (Graph Convolutional Network) is a type of neural network model specifically used to process graph data. It combines the topological structure of the graph with the node features. At each time step t, a graph convolutional network is used to extract features from the graph structure. In this invention, the node v i The features extracted by the graph convolutional network are called graph convolutional features. .

[0051] ST-GCN (Skeleton-Based Action Recognition) is a spatiotemporal graph convolutional network. It introduces temporal graph convolutional TCN into spatial graph convolutional GCN to process graph structure data with time series relationship and recognize different types of actions from it. Its network structure is divided into three parts: the first part: normalization; the second part: ST-GCN unit, which uses GCN and TCN alternately; the third part: average pooling and fully connected layer.

[0052] Compared with the prior art, the advantages of the present invention are: the present invention proposes a method for identifying and safely monitoring apron flight support services based on an extended graph structure and an ST-GCN network. The method continuously monitors the video data in the apron area and uses graph neural network technology to capture the dynamic relationship between entities such as people, vehicles, and aircraft, and identifies different stages of support services, as well as potential safety hazards or abnormal behaviors through time series analysis. It can improve the accuracy and real-time performance of identification, and also improve the efficiency of abnormal behavior identification. Specifically:

[0053] (1) Improved data real-time performance and accuracy. First, the present invention constructs a new extended graph structure. Considering that in the apron business scenario, each entity has a relatively fixed temporal behavior pattern, and the actions of each entity in the business can be decomposed according to a certain rule, the present invention introduces a temporal behavior pattern into the initial feature vector of the node in the extended graph structure. By counting the average duration between two adjacent actions of each entity, the historical business model and context information of the node are analyzed. Second, in order to better obtain the interactive relationship between each node, the present invention also introduces the feature of interactive relationship in the initial feature vector of the node in the extended graph structure, which is specifically determined by the formula in step S52 of the present invention. This interaction relationship depends on the two nodes v i 、v j Edge ij Weight , and X in X t It is related to T images in the time window of the starting point, so it can better integrate the interaction relationship and context information of each node. Third, combine the extended graph structure with the ST-GCN network. In summary, by constructing an extended graph structure, we can overcome the problems of existing flight type methods that mainly rely on manual records or single visual technology, resulting in delays, missed detections, and false detections in data collection. And based on ST-GCN and video analysis technology, each node can be automatically identified, significantly improving the real-time and accuracy of the data. Especially in complex multi-target environments, it can analyze multi-frame video data in real time, and issue timely warnings when anomalies are identified, avoiding errors caused by manual intervention.

[0054] (2) Improvement in the efficiency of abnormal behavior identification. Compared with most current flight support systems that are limited to static matching of a predetermined template library in abnormal behavior detection and have difficulty monitoring potential abnormal behaviors in dynamic changes or interactions, the present invention uses the ST-GCN network to analyze the entity interaction relationship in multi-frame video data, effectively identifying abnormal behaviors such as unreasonable equipment location, insufficient personnel, and incorrect operation sequence, avoiding the limitation of traditional systems that can only be judged based on single-frame image features. The present invention can dynamically track changes in business nodes, realize comprehensive monitoring and abnormality detection of business processes in complex scenarios, and greatly improve security and monitoring effects.

[0055] (3) Improved business process control efficiency. The ability of the present invention to automatically identify and monitor business nodes enables the operation status of each node in the apron support business to be accurately controlled, reducing the need for manual intervention and monitoring, and significantly improving overall operation efficiency. The system automatically generates alarm prompts and node status displays to help managers efficiently respond to emergencies and potential operational deviations, thereby ensuring the smooth progress of business processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described below in conjunction with embodiments and drawings.

[0058] Example 1: See Figure 1 , a method for identifying apron flight support services, comprising the following steps;

[0059] S1, defines C types of security services. Each type of security service is completed by several entities in cooperation. The security service is divided into three business stages: start, ongoing, and end.

[0060] If the c-type protection business GB c Corresponding to n entities, number the entities and construct GB c The entity collection , v i is the i-th entity;

[0061] Decomposition i For each action, a temporal behavior pattern is generated , is the average duration from the k-1th action to the kth action;

[0062] S2, for each type of security service, obtain several corresponding video frame sequences, one of which is , X t is the image at the tth time step in X, N is the total number of time steps, 1≤t≤N;

[0063] S3, data annotation of video frame sequence, X t The marking method is:

[0064] If X corresponds to GB c , then according to EC c X t Perform entity annotation and business stage annotation, wherein the entity annotation includes entity type and location, and the business stage annotation includes guarantee business type and business stage;

[0065] S4, the entity set EC c The entity in the middle is used as a node to generate X1~X N The initial graph structure ~ , extract the graph convolution features of each node in the initial graph structure through the GCN network, and t The initial graph structure , V is a set of nodes and V=EC c , node v in V i The initial eigenvector x i =[Type i ,C i ,A i ,P i ], Type i , C i , A i , P i They are v i type, geometric center, area and polygon boundary, v i The graph convolution feature is , E is the set of edges E={e ij},e ij is the node v in V i 、v j The edge of

[0066] S5, yes ~ Expand and obtain the corresponding extended graph structure G1~G N , forming the timing diagram structure data , which will Expanded to G t The method comprises steps S51 to S52;

[0067] S51, preset the interaction threshold and time window length T, if v j v i Neighbor nodes of X, count the nodes with X t D in the image within the time window starting from ij The number N that is less than the interaction threshold ij , and calculate e according to the following formula ij Weight ;

[0068] ,

[0069] Where D ij v i 、v j The distance, α is the spatial distance weight, γ is the interaction weight;

[0070] S52, generated The extended feature vector ,use replace Medium i The initial eigenvector x i , we get X t The extended graph structure G t , According to the following formula:

[0071] ,

[0072] In the formula, For Node The set of neighbor nodes of v j Graph convolution features;

[0073] S6, use the time-series graph structure data to train the ST-GCN network until convergence, and obtain the type recognition model. During training, The input of the tth time step is G t , the expected output is , , X t The actual type and stage of guaranteed business;

[0074] S7, use the type recognition model to identify the security service type and service stage of the images in the surveillance video of the apron.

[0075] The support services include jet bridge docking services, refueling services, baggage loading and unloading services, cargo loading and unloading services and in-flight catering services.

[0076] In S21, the video frame sequence is collected by a camera covering the apron area and obtained after video compression, format conversion and image enhancement, and each video frame in the video frame sequence contains all targets in its corresponding support business.

[0077] right When labeling entities, first use the target detection network to detect the entity and obtain the target detection result, then use the image segmentation network to perform semantic segmentation on the entity and obtain the semantic segmentation result;

[0078] The target detection network and the image segmentation network are pre-trained;

[0079] The target detection result includes entity category, bounding box and detection confidence, which is used for entity labeling;

[0080] The semantic segmentation result includes a set of entity contours and segmentation confidence, which are used to generate an initial feature vector of a node.

[0081] In S4, In the example, the set of constructed edges is: i 、v j The distance D ij is less than the distance threshold, then in v i 、v j Create an edge , get the set of edges .

[0082] S7 specifically includes: obtaining the surveillance video of the apron and generating a sequence of video frames to be identified; for the image at the tth time step , S4 and S5 generate the corresponding extended graph structure, which is sent to the type recognition model to obtain The predicted output ,in , They are Forecasted assurance business types and business stages.

[0083] Example 2: See Figure 1 Regarding various types of support services: including but not limited to bridge docking, refueling, baggage loading and unloading, cargo loading and unloading, and in-flight catering. Airports can customize a set of actions as support services based on actual business operations. We give the specific definitions as follows:

[0084] The jet bridge docking service refers to the staff adjusting the jet bridge height and extension length after the aircraft is in place, so that the jet bridge is perfectly aligned with the aircraft door to ensure the safety of passengers boarding and disembarking. The entities in the jet bridge docking service identification include the jet bridge and the aircraft nose. The task of jet bridge docking service identification is to identify and record the time node when the jet bridge completes docking (bridge docking completion) and the time node when the jet bridge is evacuated (bridge withdrawal completion); that is to say: in the business stage of the jet bridge docking service, the completion of the bridge docking is the starting stage, the completion of the bridge withdrawal is the ending stage, and the stage in between is the ongoing stage.

[0085] The refueling business refers to the refueling personnel parking the refueling truck accurately at the appropriate location and connecting the oil gun to the aircraft refueling port to refuel. The entities in the refueling business include the staff, the refueling truck, the oil gun and the right wing of the aircraft. The task of refueling business identification is to identify and record the time node when the oil gun is connected to the aircraft (refueling starts) and the time node when the oil gun is separated from the aircraft (refueling ends); therefore, the node when the oil gun is connected to the aircraft is the starting node, and the node when the oil gun is separated from the aircraft is the ending stage, and the period between the two is in progress.

[0086] Baggage handling business refers to the baggage handling personnel using baggage transfer vehicles, flatbed trucks and other equipment to unload the baggage of arriving passengers and load the baggage of departing passengers. The entities in the baggage handling business include staff, baggage transfer vehicles, flatbed trucks and aircraft cargo doors. The task in the baggage handling business is to identify and record the time node when the baggage cart or flatbed truck first docks with the cargo door (the beginning of baggage loading and unloading) and the time node when the baggage cart or flatbed truck last evacuates from the cargo door (the end of baggage loading and unloading). Therefore, based on these definitions, we can divide the baggage handling business into the starting stage, the ongoing stage and the ending stage.

[0087] Cargo handling business refers to cargo handling personnel using lift trucks to unload arriving cargo and load departing cargo onto aircraft. Entities in cargo handling business include personnel, lift trucks, and aircraft cargo doors. The task in cargo handling business is to identify and record the time node when the lift truck first docks with the cargo door (cargo loading and unloading begins) and the time node when the lift truck last evacuates from the cargo door (cargo loading and unloading ends). Similarly, based on the start and end nodes, we can divide cargo handling business into the start stage, ongoing stage, and end stage.

[0088] Airline catering business refers to the catering staff driving the catering vehicle close to the aircraft to complete the docking and handing over the meals to the crew. The entities in the airline catering business include staff, food trucks and aircraft cabin doors. The task in the airline catering business is to identify and record the time node when the catering truck first docks with the cabin door (the start of catering) and the time node when it last evacuates from the cabin door (the end of catering). According to the start and end nodes, we can divide the airline catering business into the start stage, the ongoing stage and the end stage.

[0089] The above are only defined protection services. Users can choose entities to define according to their own needs.

[0090] Regarding the temporal behavior patterns of entities, such as The timing behavior pattern of We take the physical refueling truck in the refueling business as an example and decompose its actions into the following sequence: the refueling truck arrives, the refueling truck connects, the refueling truck starts refueling, the refueling truck finishes refueling, the refueling truck disconnects, and the refueling truck leaves. We can count the time intervals between these actions based on historical data and obtain the feature vector {average arrival connection time, average connection refueling time, average refueling time, …} as the temporal behavior pattern of the refueling truck in the refueling business.

[0091] Example 3: See Figure 1 , based on Example 1, further comprising step S8, The prediction output is visualized.

[0092] Example 4: See Figure 1 A method for identifying abnormalities in apron flight support services is used to identify abnormal events in support services. The abnormal events include unreasonable locations, insufficient number of people, incorrect operation sequences, etc. The method can be designed according to needs. In this embodiment, the method mainly includes the following three types.

[0093] The unreasonable position identification includes steps Sa1 to Sa3;

[0094] Sa1, for each entity of each type of security service, pre-set its specified area at each service stage;

[0095] Sa2, get The prediction output is obtained according to the type of guaranteed business and business stage. the prescribed area of ​​each entity within;

[0096] Sa3, yes Perform target detection and semantic segmentation to obtain For each entity and entity outline, determine whether each entity outline is located in the specified area. If so, skip it, otherwise alarm.

[0097] The abnormal event also includes insufficient number of people, and the insufficient number of people identification includes steps Sb1 to Sb3;

[0098] Sb1, if a type of protection business If there is a number requirement in a business phase, the business phase of the guarantee business is marked as a node to be detected, and the number threshold of the node to be detected is predicted. ;

[0099] Sb2, get The predicted output is used to determine whether it is a node to be detected. If not, skip it. , otherwise proceed to the next step;

[0100] Sb3, yes Perform target detection and obtain the number of entities whose type is person ,like , then alarm, otherwise skip .

[0101] The abnormal events also include errors in the operation sequence, identification and guarantee of business GB c The method of operating in the wrong order includes steps Sc1 to Sc3;

[0102] Sc1, will ensure business GB c Divided into S sub-actions, marked in order of precedence ~ , and set GB c Order of operations ;

[0103] Sc2, get The forecast output is: if the service GB is to be guaranteed c At the beginning stage, And subsequent images are sent to the action recognition model in sequence until The prediction output of subsequent images is to ensure the business GB c the end stage;

[0104] Sc3, the action recognition model recognizes the actions of the input images in sequence. If the sequence of recognized actions is consistent with If different, alarm will be issued.

[0105] Regarding the unreasonable location identification in abnormal event identification: This is because in the security business, each entity, such as operating personnel, refueling trucks, luggage trucks, catering trucks, etc., has a specified working area in each business stage. By judging whether the entity is located in the specified area through steps Sa1~Sa3, it is possible to automatically detect whether an "unreasonable location" event occurs.

[0106] Regarding insufficient number of people in abnormal event identification: It is mainly used in certain business nodes, such as boarding, luggage loading, etc. These nodes have clear personnel requirements. If the number of detected people is less than the expected number, an abnormal alarm is triggered.

[0107] Regarding the identification of operation sequence errors in abnormal event identification: it is mainly used to determine whether the execution sequence of each node in the security business is consistent with the preset standard process.

[0108] Example 5: Based on Example 4, the present invention can be further optimized, for example, by adding a visualization module. The visualization module is intended to provide an integrated, dynamic visualization interface so that managers can monitor the progress of apron flight support services in real time. The module combines multiple views and visualization elements such as monitoring videos, node lists, abnormal behavior markers, and dynamic Gantt charts to present business status and abnormal events through a unified interface.

[0109] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying apron flight support services, characterized in that: The steps include: S1, defines C types of security services. Each type of security service is completed by several entities in cooperation. The security service is divided into three business stages: start, ongoing, and end. If the c-type protection business GB c Corresponding to n entities, number the entities and construct GB c The entity collection , v i is the i-th entity; Decomposition i For each action, a temporal behavior pattern is generated , is the average duration from the k-1th action to the kth action; S2, for each type of security service, obtain several corresponding video frame sequences, one of which is , X t is the image at the tth time step in X, N is the total number of time steps, 1≤t≤N; S3, data annotation of video frame sequence, X t The marking method is: If X corresponds to GB c , then according to EC c X t Perform entity annotation and business stage annotation, wherein the entity annotation includes entity type and location, and the business stage annotation includes guarantee business type and business stage; S4, the entity set EC c The entity in the middle is used as a node to generate X1~X N The initial graph structure ~ , extract the graph convolution features of each node in the initial graph structure through the GCN network, and t The initial graph structure , V is a set of nodes and V=EC c , node v in V i The initial eigenvector x i =[Type i ,C i ,A i ,P i ], Type i , C i , A i , P i They are v i type, geometric center, area and polygon boundary, v i The graph convolution feature is , E is the set of edges E={e ij },e ij is the node v in V i 、v j The edge of S5, yes ~ Expand and obtain the corresponding extended graph structure G1~G N , forming the timing diagram structure data , which will Expanded to G t The method comprises steps S51 to S52; S51, preset the interaction threshold and time window length T, if v j v i Neighbor nodes of X, count the nodes with X t D in the image within the time window starting from ij The number N that is less than the interaction threshold ij , and calculate e according to the following formula ij Weight ; ， Where D ij is the v in the same image i and v j The spatial distance For X t Medium i 、v j The spatial distance, α is the spatial distance weight, and γ is the interaction weight; S52, generated The extended feature vector ,use replace Medium i The initial eigenvector x i , we get X t The extended graph structure G t , According to the following formula: ， In the formula, For Node The set of neighbor nodes of v j Graph convolution features; S6, use the time-series graph structure data to train the ST-GCN network until convergence, and obtain the type recognition model. During training, Medium G t The expected output is , , X t The actual type and stage of guaranteed business; S7, use the type recognition model to identify the security service type and service stage of the images in the surveillance video of the apron.

2. A method for identifying apron flight support services according to claim 1, characterized in that: The support services include jet bridge docking services, refueling services, baggage loading and unloading services, cargo loading and unloading services and in-flight catering services.

3. The method for identifying apron flight support services according to claim 1, characterized in that: In S21, the video frame sequence is collected by a camera covering the apron area and obtained after video compression, format conversion and image enhancement, and each video frame in the video frame sequence contains all targets in its corresponding support business.

4. The method for identifying apron flight support services according to claim 1, characterized in that: X t When labeling entities, first use the target detection network to detect the entity and obtain the target detection result, then use the image segmentation network to perform semantic segmentation on the entity and obtain the semantic segmentation result; The target detection network and the image segmentation network are pre-trained; The target detection result includes entity category, bounding box and detection confidence, which is used for entity labeling; The semantic segmentation result includes a set of entity contours and segmentation confidence, which are used to generate an initial feature vector of a node.

5. The method for identifying apron flight support services according to claim 1, characterized in that: In S4, In the example, the set of constructed edges is: i 、v j The distance D ij is less than the distance threshold, then in v i 、v j Room construction ij , we get the edge set E={e ij }.

6. The method for identifying apron flight support services according to claim 1, characterized in that: S7 specifically: Obtain surveillance video of the apron and generate a sequence of video frames to be identified; For the image at the tth time step , S4 and S5 generate the corresponding extended graph structure, which is sent to the type recognition model to obtain The predicted output ,in , They are Forecasted assurance business types and business stages.

7. A method for identifying apron flight support services according to claim 6, characterized in that: The method further comprises step S8: The prediction output is visualized.

8. A method for identifying abnormalities in apron flight support services, characterized in that: Based on the method for identifying apron flight support services as described in claim 6, abnormal events are identified for support services, wherein the abnormal events include unreasonable locations; The unreasonable position identification includes steps Sa1 to Sa3; Sa1, for each entity of each type of security service, pre-set its specified area at each service stage; Sa2, get The prediction output is obtained according to the type of guaranteed business and business stage. the prescribed area of ​​each entity within; Sa3, yes Perform target detection and semantic segmentation to obtain For each entity and entity outline, determine whether each entity outline is located in the specified area. If so, skip it, otherwise alarm.

9. The method for identifying abnormalities in apron flight support services according to claim 8, characterized in that: The abnormal event also includes insufficient number of people, and the insufficient number of people identification includes steps Sb1 to Sb3; Sb1, if a type of guarantee service GB c If there is a number requirement in a business phase, then the business phase of the guarantee business is marked as a node to be detected, and the number threshold N of the node to be detected is predicted expected ; Sb2, get The predicted output is used to determine whether it is a node to be detected. If not, skip it. , otherwise proceed to the next step; Sb3, yes Perform target detection and obtain the number of entities whose type is person ,like , then alarm, otherwise skip .

10. The method for identifying abnormalities in apron flight support services according to claim 8, characterized in that: The abnormal events also include errors in the operation sequence, identification and guarantee of business GB c The method of operating in the wrong order includes steps Sc1 to Sc3; Sc1, will ensure business GB c Divided into S sub-actions, marked in order of precedence ~ , and set GB c Order of operations ; Sc2, get The forecast output is: if the service GB is to be guaranteed c At the beginning stage, And subsequent images are sent to the action recognition model in sequence until The prediction output of subsequent images is to ensure the business GB c the end stage; Sc3, the action recognition model recognizes the actions of the input images in sequence. If the sequence of recognized actions is the same as EO c If different, alarm will be issued.

Images