A flight complex support node detection method based on space-time constraints

Abstract

The application discloses a flight complex guarantee node detection method based on space-time constraints, which comprises the following steps: S1, constructing a flight guarantee annotation data input Yolov8 target detector for guarantee node target detection training; S2, constructing a time-constrained state automaton and setting state and state transition conditions; S3, collecting the monitoring video stream of the parking position, inputting the Yolov8 target detector, using the ByteTrack algorithm for target association, and obtaining motion trajectory data; S4, extracting the trajectory information S(i) of the corresponding guarantee node N(i) of the flight in the parking position, and the state automaton performs state recognition and state transformation; S5, the state automaton generates state transition data of the guarantee node N(i) according to the time sequence of the flight in the parking position, and the state transition data comprises time and events. The application constructs a guarantee node space-time constraint state automaton to perform state recognition and state transformation, and can realize high-precision state and state transition detection and recognition of the flight guarantee node.

Description

Technical Field

[0001] The present invention relates to the field of airport flight support visual processing, and in particular to a flight complex support node detection method based on time and space constraints. Background Art

[0002] Currently, the digitalization landscape places increasing demands on the detection types and accuracy of airport flight support nodes. Node detection types have evolved from the traditional two simple types of support node detection (target entry and target departure) to four complex types of support node detection (target entry, operation start, operation end, and target departure). Examples include jet bridge support nodes (entry, operation start, operation end, and departure), catering vehicle support nodes (entry, operation start, operation end, and departure), refueling vehicle support nodes (entry, operation start, operation end, and departure), and baggage conveyor vehicle support nodes (entry, operation start, operation end, and departure). Node detection accuracy has evolved from the traditional one to three minutes to 30 seconds or even less. This context places even higher demands on support node detection technology. Support node identification solutions based on vehicle position inference (using onboard positioning devices for vehicle positioning) and support node time inference (relying on vehicle position for inference) are unable to handle the complex and changing realities of airports and cannot meet the high-precision support node identification requirements. In addition, the support node identification technology based on single-frame video image analysis cannot accurately identify the working status of the support node, such as entering the position, starting operation, ending operation, and leaving the position, because it does not take into account the historical status of the target; in addition, occasional target omissions and false detections during the support node identification process will lead to a decrease in the detection accuracy of the support node, affecting the stability of the support detection. Summary of the Invention

[0003] The purpose of the present invention is to solve the problems of existing support node detection and identification accuracy, slow response and inability to perform working state conversion, and provide a flight complex support node detection method based on spatiotemporal constraints. The parking stand monitoring video stream is collected and passed through the Yolov8 target detector for target detection and identification, and the ByteTrack algorithm target association is used to obtain target motion trajectory data. A state automaton with spatiotemporal constraints on the support node is constructed for state identification and state conversion. High-precision state and state conversion detection of flight support nodes can be achieved, and state conversion data that advances in time sequence is output, thereby improving the state detection accuracy and identification informationization level of flight support nodes.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A method for detecting complex flight support nodes based on spatiotemporal constraints, the method comprising:

[0006] S1. Construct flight support annotation data containing parking stand monitoring video stream images and support node annotations, and input them into the Yolov8 target detector to perform target detection training for support nodes. The targets of the support nodes include jet bridges, catering trucks, fuel trucks, and baggage conveyor belt trucks.

[0007] S2. Construct a time-constrained state machine and set the states of the state machine and the transition conditions between states;

[0008] S3: Collect the surveillance video stream of the parking space and input it into the Yolov8 target detector, and use the ByteTrack algorithm to associate the targets and obtain the motion trajectory data of all targets;

[0009] S4. Extract the trajectory information S(i) of the support node N(i) corresponding to the flight in the parking space, where i represents the target. The state automaton performs state recognition and state transformation.

[0010] S5. The state automaton generates state transition data for the support node N(i) according to the time sequence of the flights in the parking stand. The state transition data includes time and events. Events include entering the stand, starting the operation, ending the operation, leaving the stand, and ending the operation and leaving the stand.

[0011] In order to better implement the present invention, the states of the state machine include an empty state Me, a state where the target appears but does not enter the working area Mo, a state where the target enters the working area Mi, and a state where the target is working Mw.

[0012] Preferably, the transition conditions T of each state in the state automaton M are {condition A, condition B, condition C, condition D}, where condition A is that the target transitions from the empty state Me to the state Mo and remains in the state Mo for T1 time, condition B is that the target transitions from the state Mo to the state Mi and remains in the state Mi for T2 time, condition C is that the target transitions from the state Mi to the state Mw and remains in the state Mw for T3 time, and condition D is that the target transitions from the state Mw to the empty state Me and remains in the empty state Me for T4 time.

[0013] Preferably, the surveillance video stream of the parking space is collected by a camera combination system, which is a plurality of cameras arranged around.

[0014] Preferably, the state transition condition determination method of the state automaton is as follows:

[0015] S41. If the trajectory information S(i) of target i at the current moment is empty, target i is set to the empty state Me. The state automaton combines the previous state recognition of target i to determine whether it meets condition D. If not, condition A tracking judgment is executed;

[0016] S42. If the state of the latest K1 frames in the trajectory information S(i) of target i is the empty state Me, and the state of the K1+1 frame is changed to the state Mo and the video frames for T1 time are all in the state Mo, then set the state transition condition T(i) = [condition A];

[0017] S43. After the state transition condition T(i) = [Condition A], if the mean of the target center position of the latest K2 frames in the trajectory information S(i) of target i is within the working area corresponding to target i, the variance of the target center position of the latest K2 frames is less than the threshold Pi, and the target state labels of the latest K2 frames are all working states Mi, then set the state transition condition T(i) = [Condition B];

[0018] S44. After the state transition condition T(i) = [Condition B], if the mean of the target center position of the latest K3 frames in the trajectory information S(i) of target i is within the working area corresponding to target i, the variance of the target center position of the latest K3 frames is less than the threshold Pi, and the target state labels of the latest K3 frames are all working states Mw, then set the state transition condition T(i) = [Condition C];

[0019] S45. After the state transition condition T(i)=[Condition C], if the states of the latest K4 frames in the trajectory information S(i) of target i are all empty states Me, then set the state transition condition T(i)=[Condition D].

[0020] Preferably, the state automaton generates state transition data according to the time sequence of flights in the parking stands as follows:

[0021] S51. If the state transition condition T(i) = [condition B], the previous state of the state machine is state Mo and the current state is state Mi, the event of the state transition data is set to entry and the entry time is recorded;

[0022] S52. If the state transition condition T(i) = [condition C], the previous state of the state machine is state Mi and the current state is state Mw, the event of the state transition data is set to start operation and the start time of the operation is recorded;

[0023] S53. If the state transition condition T(i) = [condition C], the previous state of the state machine is state Mw and the current state is state Mi, the event of the state transition data is set to end the job and the end time of the job is recorded;

[0024] S54. If the state transition condition T(i) = [condition C], the previous state of the state machine is state Mi and the current state is state Mo, the event of the state transition data is set to off-position and the off-position time is recorded;

[0025] S55. If the state transition condition T(i) = [condition D], the current state of the state machine is state Mo, and the event of the state transition data is set to end the job and leave.

[0026] Preferably, the Yolov8 target detector detects and identifies target i on the security node, and the state automaton marks the state in which target i is not identified as the empty state Me; the Yolov8 target detector identifies target i, and the Yolov8 target detector has a security node work identification module inside; the processing method of the security node work identification module is as follows: the security node work identification module demarcates the working area of ​​target i based on the parking space for the parking space video frame image containing the aircraft and performs position judgment. If the target i is entirely located in the working area, the target i is marked as target i_idle, and the state automaton sets the state of target i to state Mi; otherwise, it is only marked as target i, and the state automaton sets the state of target i to state Mo.

[0027] Preferably, the working state feature extraction and recognition are performed on the target labeled as i_idle, the target i identified as being in the working state is labeled as target i_work, and the state automaton sets the state of the target i to the state Mw.

[0028] Preferably, the Yolov8 target detector performs frame-by-frame target detection on the surveillance video stream, and the ByteTrack algorithm performs target association and tracking according to target i based on the previous and next frames of the surveillance video stream to obtain motion trajectory data corresponding to target i.

[0029] Preferably, the video images in the flight support annotation data cover all weather scenes, all lighting modes, and all states of support nodes including jet bridges, catering trucks, refueling trucks, and baggage conveyor belt trucks.

[0030] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0031] (1) The present invention collects parking space monitoring video streams and performs target detection and identification through the Yolov8 target detector, uses the ByteTrack algorithm target association to obtain target motion trajectory data, and constructs a state automaton with time and space constraints for the support node to perform state identification and state transformation. It can achieve high-precision state and state transition detection of flight support nodes, output state transition data that advances in time sequence, and improve the state detection accuracy and recognition informationization level of flight support nodes.

[0032] (2) The state automatic mechanism of the present invention is equipped with the state recognition and state transformation conditions of the jet bridge, catering truck, refueling truck and luggage conveyor belt truck as the support node targets, and combines time constraints and model feature recognition to achieve high-precision detection of complex flight support nodes, solving the problems of low accuracy and large errors in traditional inference, and target omission and false detection in single-frame video analysis, solving the technical problems of low support accuracy and high cost of existing solutions, making the flight support system run more stably. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the method of the present invention;

[0034] Figure 2 Schematic diagram of the transfer condition principle of the state automaton in the embodiment;

[0035] Figure 3 Schematic diagram of the target recognition of the baggage conveyor belt vehicle security node by the Yolov8 target detector in the embodiment;

[0036] Figure 4 Schematic diagram of target recognition of a corridor bridge security node by the Yolov8 target detector in an embodiment;

[0037] Figure 5 Schematic diagram of the Yolov8 target detector identifying the working status of the corridor bridge security node in the embodiment. DETAILED DESCRIPTION

[0038] Below in conjunction with embodiment, the present invention is described in further detail:

[0039] Example

[0040] like Figure 1 As shown, a flight complex support node detection method based on time and space constraints includes:

[0041] S1. Construct flight support annotation data including parking stand monitoring video stream images (the monitoring video stream of the parking stand in this embodiment is collected by a camera joint system, and the camera joint system is a number of cameras set up around it) and support node annotations, and input them into the Yolov8 target detector to perform target detection training for support nodes. The targets of the support nodes of the present invention include four categories: jet bridges, catering trucks, refueling trucks, and luggage conveyor belt vehicles. The video images in the flight support annotation data cover all weather scenes, all lighting modes, and all states of support nodes including jet bridges, catering trucks, refueling trucks, and luggage conveyor belt vehicles. The flight support annotation data is a flight support annotation dataset D, which includes video stream images and support node annotations on the video stream images.

[0042] S2. Construct a state automaton with time constraints, and set the states of the state automaton and the transition conditions between states. The state automaton M of the present invention contains the time constraint information of the target, which can ensure that the detection of the security node is more accurate, and is not affected by occasional erroneous target detection results that lead to misidentification of the security node, thereby improving the stability of the node detection scheme. In some embodiments, the states of the state automaton include an empty state Me, a state Mo where the target appears but does not enter the working area, a state Mi where the target enters the working area, and a state Mw where the target is working. The transition conditions T of each state in the state automaton M = {condition A, condition B, condition C, condition D}, where condition A is that the target transitions from the empty state Me to the state Mo and remains in the state Mo for T1 time (which can be replaced by a continuous number of P1 video frames); Figure 2 As shown, in this embodiment, condition A is further provided that the target changes from state Mi to state Mo and lasts for T1 time. Condition B is that the target changes from state Mo to state Mi and lasts for T2 time (which can be replaced by P2 consecutive video frames) in state Mi; Figure 2 As shown, in this embodiment, Condition B also adds the requirement that the target transition from state Mw to state Mi for a duration of T1. Condition C requires that the target transition from state Mi to state Mw and remain in state Mw for a duration of T3 (which can be replaced by P3 consecutive video frames). Condition D requires that the target transition from state Mw to state Me and remain in state Me for a duration of T4 (which can be replaced by P4 consecutive video frames). For simple examples, in this embodiment, the time range of T1 to T4 is 2 to 5 seconds.

[0043] S3. Capture the parking stand surveillance video stream, input it into the Yolov8 object detector, and use the ByteTrack algorithm to perform target association and obtain the motion trajectory data of all targets. Preferably, in this embodiment, the parking stand surveillance video stream is collected by a camera linkage system, which consists of several cameras positioned around the parking stand. If multiple cameras are used to form the camera linkage system, the cameras are positioned at different monitoring locations on the parking stand and assigned security node detection tasks. Target detection is performed on the video frame image packets from all cameras at the same time. The Yolov8 object detector detects and identifies target i at the security node, and the state machine labels the state where target i is not identified as the empty state Me. The Yolov8 object detector identifies target i and includes an internal security node working identification module. The security node working identification module processes the following: The security node working identification module demarcates the working area of ​​target i based on the parking stand video frame containing the aircraft and determines its position. If target i is completely within the working area, target i is labeled as target i_idle, and the state machine sets the state of target i to state Mi; otherwise, it is simply labeled as target i. The state automaton sets the state of target i to state Mo. The feature extraction and recognition of the working state of target i_idle are performed, and the target i identified as the working state is marked as target i_work. The state automaton sets the state of target i to state Mw.

[0044] Target i includes four types of monitoring targets: corridor bridge, catering vehicle, fuel truck and luggage conveyor belt vehicle. If target i is a corridor bridge, Yolov8 target detector identifies the corridor bridge target in the video frame image and marks it as lq (lq_idle and lq_work are marked later); Figure 4 As shown, this embodiment uses the Yolov8 target detector to detect and identify the target of the corridor bridge support node. If the corridor bridge target is identified, it is first marked as lq. If the Yolov8 target detector detects that the corridor bridge is located in the corridor bridge working area (the Yolov8 target detector demarcates the corridor bridge working area based on the aircraft fuselage), but does not recognize that the corridor bridge is in a working state (the guarantee node work identification module performs feature extraction and identification of the working state, whether the corridor bridge is docked with the aircraft fuselage), it is marked as lq_idle, and the state automaton sets the state of the corridor bridge lq_ to state Mi. If the Yolov8 target detector detects that the corridor bridge is located in the corridor bridge working area and recognizes that the corridor bridge is in a working state, it is marked as lq_work, as shown in FIG. Figure 5 As shown, this embodiment uses the Yolov8 target detector to detect and identify the corridor bridge support node. If the support node work identification module detects that the corridor bridge is in the working state, it is marked as lq_work. The state automaton sets the state of the corridor bridge lq_ to state Mw.

[0045] If target i is a catering vehicle, the Yolov8 target detector identifies the catering vehicle target in the video frame image and marks it as pcc (pcc_idle and pcc_work will be marked later). If the Yolov8 target detector detects that the catering vehicle is located in the catering vehicle working area (the Yolov8 target detector uses the aircraft fuselage as the reference to define the catering vehicle working area), but does not recognize that the catering vehicle is in a working state, it is marked as pcc_idle, and the state automaton sets the state of the catering vehicle pcc_ to state Mi. If the Yolov8 target detector detects that the catering vehicle is located in the catering vehicle working area and recognizes that the catering vehicle is in a working state (the guarantee node work identification module performs feature extraction and identification of the working state, and whether the catering vehicle's catering operation is raised), it is marked as pcc_work, and the state automaton sets the state of the catering vehicle pcc_ to state Mw.

[0046] If target i is a refueling truck, the Yolov8 target detector identifies the refueling truck target in the video frame image and labels it as jyc (jyc_idle and jyc_work will be labeled later). If the Yolov8 target detector detects that the refueling truck is located in the refueling truck working area (the Yolov8 target detector uses the aircraft fuselage as a reference to define the refueling truck working area), but does not recognize that the refueling truck is in a working state, it is labeled as jyc_idle, and the state automaton sets the state of the refueling truck jyc_ to state Mi. If the Yolov8 target detector detects that the refueling truck is located in the refueling truck working area and recognizes that the refueling truck is in a working state (the guarantee node work identification module performs feature extraction and identification of the working state, whether the refueling truck is docked with the aircraft fuselage or the refueling pipe is connected), it is labeled as jyc_work, and the state automaton sets the state of the refueling truck jyc_ to state Mw.

[0047] If the target i is a luggage conveyor belt car, the Yolov8 target detector recognizes the luggage conveyor belt car target in the video frame image and marks it as xlcsd (xlcsd_idle and xlcsd_work will be marked later), such as Figure 3As shown, in this embodiment, the Yolov8 target detector is used to detect and identify the target of the baggage conveyor car support node. If the baggage conveyor car target is identified, it is first marked as xlcsd. If the Yolov8 target detector detects that the baggage conveyor car is located in the working area of ​​the baggage conveyor car (the Yolov8 target detector defines the working area of ​​the baggage conveyor car based on the aircraft fuselage), but does not recognize that the baggage conveyor car is in a working state (the working identification module of the support node performs feature extraction and identification of the working state, and whether the baggage conveyor car is docked with the aircraft fuselage), it is marked as xlcsd_idle, and the state automaton sets the state of the baggage conveyor car xlcsd_ to state Mi; as shown in FIG. Figure 3 As shown, in this embodiment, the Yolov8 target detector performs target detection and identification on the baggage conveyor vehicle support node. If the support node work identification module detects that the baggage conveyor vehicle is not docked with the aircraft fuselage, it is marked as xlcsd_idle. If the Yolov8 target detector detects that the baggage conveyor vehicle is within the baggage conveyor vehicle work area and identifies that the baggage conveyor vehicle is in the working state, it is marked as xlcsd_work, and the state automaton sets the state of the baggage conveyor vehicle xlcsd_ to the state Mw; Figure 3 As shown, this embodiment uses the Yolov8 target detector to detect and identify the baggage conveyor car support node. If the support node work identification module detects that the baggage conveyor car is docked with the aircraft fuselage, it is marked as xlcsd_work.

[0048] The Yolov8 target detector of the present invention performs frame-by-frame target detection on the monitoring video stream, and the ByteTrack algorithm performs target association and tracking according to target i based on the previous and next frames of the monitoring video stream to obtain the motion trajectory data corresponding to target i (forming a continuous motion trajectory containing different states of the same target).

[0049] S4. Extract the trajectory information S(i) of the support node N(i) corresponding to the flight in the parking stand, where i represents the target (four types of monitoring target objects: jet bridge, catering truck, fuel truck and baggage conveyor truck). The state automaton performs state recognition and state transformation.

[0050] S5. The state automaton generates state transition data for the support node N(i) according to the time sequence of the flights in the parking stand. The state transition data includes time and events. Events include entering the stand, starting the operation, ending the operation, leaving the stand, and ending the operation and leaving the stand.

[0051] In some embodiments, the states of the state automaton include an empty state Me, a target appears but does not enter the working area state Mo, a target enters the working area state Mi, and a target is working state Mw. The transition conditions T = {condition A, condition B, condition C, condition D} of each state in the state automaton M, and the state transition condition judgment method of the state automaton is as follows:

[0052] S41, if the trajectory information S(i) of the target i at the current time is empty, the target i is set to the empty state Me, and the state automaton combines the previous state of the target i to judge whether it is condition D. If it is not condition D, condition A tracking judgment is performed.

[0053] S42, if the states of the last K1 frames (in this embodiment, the time interval of adjacent video frames is 0.2 seconds, and k1 is 10) in the trajectory information S(i) of the target i are all empty state Me, the state of the K1+1 frame is transferred to the state Mo, and the video frames (which can be replaced by a continuous M1 number of video frames. If the time interval of adjacent video frames is 0.2 seconds, M1 is 10-25) in the T1 time (in this embodiment, T1 time is set to 2-5 seconds) are all in the state Mo, the state transition condition T(i) = [condition A] is set.

[0054] S43, after the state transition condition T(i) = [condition A], if the average of the target center positions of the last K2 frames in the trajectory information S(i) of the target i is within the working area corresponding to the target i, the variance of the target center positions of the last k2 frames (in this embodiment, the time interval of adjacent video frames is 0.2 seconds, and k2 is 10) is less than the threshold value Pi (the threshold value Pi is set according to the target i), and the target state labels of the last k2 frames are all working state Mi, the state transition condition T(i) = [condition B] is set.

[0055] S44, after the state transition condition T(i) = [condition B], if the average of the target center positions of the last K3 frames in the trajectory information S(i) of the target i is within the working area corresponding to the target i, the variance of the target center positions of the last k3 frames (in this embodiment, the time interval of adjacent video frames is 0.2 seconds, and k2 is 12) is less than the threshold value Pi (the threshold value Pi is set according to the target i), and the target state labels of the last k3 frames are all working state Mw, the state transition condition T(i) = [condition C] is set.

[0056] S45, after the state transition condition T(i) = [condition C], if the states of the last K4 frames in the trajectory information S(i) of the target i are all empty state Me, the state transition condition T(i) = [condition D] is set.

[0057] In some embodiments, the state automaton generates state transition data according to the time sequence of the flights in the parking position as follows:

[0058] S51. If the state transition condition T(i) = [condition B], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor truck), the last state of the state automaton (i.e., the state of target i in the previous video frame) is state Mo and the current state (i.e., the state of target i in the current video frame) is state Mi, the event of target i in the state transition data is set to entry and the entry time is recorded.

[0059] S52. If the state transition condition T(i) = [condition C], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor truck), the last state of the state machine (i.e., the state of target i in the previous video frame) is state Mi and the current state (i.e., the state of target i in the current video frame) is state Mw, the event of target i in the state transition data is set to start operation and the start time of operation is recorded.

[0060] S53. If the state transition condition T(i) = [condition C], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor truck), the last state of the state automaton (i.e., the state of target i in the previous video frame) is state Mw and the current state (i.e., the state of target i in the current video frame) is state Mi, the event of target i in the state transition data is set to end the operation and the end operation time is recorded.

[0061] S54. If the state transition condition T(i) = [condition C], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor truck), the last state of the state automaton (i.e., the state of target i in the previous video frame) is state Mi and the current state (i.e., the state of target i in the current video frame) is state Mo, the event of target i in the state transition data is set to leave and the leaving time is recorded.

[0062] S55. If the state transition condition T(i) = [condition D], i represents the target object (i.e., the four target objects of the corridor bridge, catering vehicle, refueling vehicle and luggage conveyor vehicle), the current state of the state automaton is state Mo, and the event of target i in the state transition data is set to end the operation and leave.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting complex flight support nodes based on spatiotemporal constraints, characterized by: The methods include: S1. Construct flight support annotation data containing parking stand monitoring video stream images and support node annotations, and input them into the Yolov8 target detector to perform target detection training for support nodes. The targets of the support nodes include jet bridges, catering trucks, fuel trucks, and baggage conveyor belt trucks. S2. Construct a time-constrained state machine and set the states of the state machine and the transition conditions between states; the states of the state machine include an empty state Me, a state Mo where the target appears but has not entered the work area, a state Mi where the target has entered the work area, and a state Mw where the target is working; the transition conditions T for each state in the state machine M = {condition A, condition B, condition C, condition D}, where condition A means that the target transitions from the empty state Me to the state Mo and remains in the state Mo for a period of time T1, condition B means that the target transitions from the state Mo to the state Mi and remains in the state Mi for a period of time T2, condition C means that the target transitions from the state Mi to the state Mw and remains in the state Mw for a period of time T3, and condition D means that the target transitions from the state Mw to the empty state Me and remains in the empty state Me for a period of time T4; S3: Collect the surveillance video stream of the parking space and input it into the Yolov8 target detector, and use the ByteTrack algorithm to associate the targets and obtain the motion trajectory data of all targets; S4. Extract the trajectory information S(i) of the support node N(i) corresponding to the flight in the parking space, where i represents the target. The state automaton performs state recognition and state transition. The state transition condition judgment method of the state automaton is as follows: S41. If the trajectory information S(i) of target i at the current moment is empty, target i is set to the empty state Me. The state automaton combines the previous state recognition of target i to determine whether it meets condition D. If not, condition A tracking judgment is executed; S42. If the state of the latest K1 frames in the trajectory information S(i) of target i is the empty state Me, and the state of the K1+1 frame is transferred to the state Mo and the video frames for T1 time are all in the state Mo, then set the state transition condition T(i) = [condition A]; S43. After the state transition condition T(i) = [Condition A], if the mean of the target center position of the latest K2 frames in the trajectory information S(i) of target i is within the working area corresponding to target i, the variance of the target center position of the latest K2 frames is less than the threshold Pi, and the target state labels of the latest K2 frames are all working states Mi, then set the state transition condition T(i) = [Condition B]; S44. After the state transition condition T(i) = [Condition B], if the mean of the target center position of the latest K3 frames in the trajectory information S(i) of target i is within the working area corresponding to target i, the variance of the target center position of the latest K3 frames is less than the threshold Pi, and the target state labels of the latest K3 frames are all working states Mw, then set the state transition condition T(i) = [Condition C]; S45. After the state transition condition T(i) = [Condition C], if the states of the latest K4 frames in the trajectory information S(i) of target i are all empty states Me, then set the state transition condition T(i) = [Condition D]; S5. The state automaton generates state transition data for the support node N(i) according to the time sequence of the flights in the parking stand. The state transition data includes time and events. Events include entering the stand, starting the operation, ending the operation, leaving the stand, and ending the operation and leaving the stand.

2. The method for detecting complex flight support nodes based on spatiotemporal constraints according to claim 1, characterized in that: The surveillance video stream of the parking space is collected by a camera joint system, which is a number of cameras set up around it.

3. The method for detecting complex flight support nodes based on spatiotemporal constraints according to claim 1, characterized in that: The method for the state automaton to generate state transition data according to the time sequence of flights in the parking position is as follows: S51. If the state transition condition T(i) = [condition B], the previous state of the state machine is state Mo and the current state is state Mi, the event of the state transition data is set to entry and the entry time is recorded; S52. If the state transition condition T(i) = [condition C], the previous state of the state machine is state Mi and the current state is state Mw, the event of the state transition data is set to start the job and the start time of the job is recorded; S53. If the state transition condition T(i) = [condition C], the previous state of the state machine is state Mw and the current state is state Mi, the event of the state transition data is set to end the job and the end time of the job is recorded; S54. If the state transition condition T(i) = [condition C], the previous state of the state machine is state Mi and the current state is state Mo, the event of the state transition data is set to off-position and the off-position time is recorded; S55. If the state transition condition T(i) = [condition D], the current state of the state machine is state Mo, and the event of the state transition data is set to end the job and leave.

4. The method for detecting complex flight support nodes based on spatiotemporal constraints according to claim 1, characterized in that: The Yolov8 target detector detects and identifies target i on the support node, and the state automaton marks the state of unidentified target i as empty state Me; the Yolov8 target detector identifies target i, and the Yolov8 target detector has a support node work identification module inside; the processing method of the support node work identification module is as follows: the support node work identification module demarcates the working area of ​​target i based on the parking space for the parking space video frame image containing the aircraft and performs position judgment. If the target i is entirely located in the working area, the target i is marked as target i_idle, and the state automaton sets the state of target i to state Mi; otherwise, it is only marked as target i, and the state automaton sets the state of target i to state Mo.

5. The method for detecting complex flight support nodes based on spatiotemporal constraints according to claim 4, characterized in that: The working state feature extraction and recognition are performed on the target labeled as i_idle, and the target i identified as being in the working state is labeled as target i_work. The state automaton sets the state of the target i to the state Mw.

6. The method for detecting complex flight support nodes based on spatiotemporal constraints according to claim 1, characterized in that: The Yolov8 target detector performs frame-by-frame target detection on the surveillance video stream, and the ByteTrack algorithm performs target association and tracking according to target i based on the previous and next frames of the surveillance video stream to obtain the motion trajectory data corresponding to target i.

7. The method for detecting complex flight support nodes based on spatiotemporal constraints according to claim 1, characterized in that: The video images in the flight support annotation data cover all weather scenes, all lighting modes, and all states of support nodes including jet bridges, catering trucks, refueling trucks, and baggage conveyor belt trucks.

Images