Flight complex guarantee node detection method based on space-time constraint

Abstract

The invention discloses a time-space constraint-based flight complex guarantee node detection method. The method comprises the following steps of S1, constructing flight guarantee label data, and inputting the flight guarantee label data into a Yolov8 target detector to carry out target detection training of guarantee nodes; s2, constructing a time-constrained state automaton, and setting states and transfer conditions between the states; s3, collecting a monitoring video stream of a gate position, inputting the monitoring video stream into a Yolov8 target detector, carrying out target association by using a ByteTrack algorithm, and obtaining motion trail data; s4, extracting track information S (i) of a guarantee node N (i) corresponding to a flight in a gate position, and performing state recognition and state conversion by the state automaton; and S5, the state automaton generates state conversion data of the guarantee node N (i) according to time sequence propulsion of flights in the gate positions, wherein the state conversion data comprises time and events. According to the method, the state automaton of the space-time constraint of the guarantee node is constructed for state recognition and state conversion, and the high-precision state and state conversion detection and recognition of the flight guarantee node can be realized.

Description

Technical Field

[0001] The present invention relates to the field of airport flight support visual processing, and in particular to a flight complex support node detection method based on time and space constraints. Background Art

[0002] At present, the requirements for the detection type and detection accuracy of airport flight support nodes in the context of digitalization are getting higher and higher. In terms of node detection types, the traditional two types of simple support node detection of target entry and target departure have gradually changed to four types of complex support node detection of target entry, start operation, end operation, and target departure, such as the bridge support node (entry, start operation, end operation, departure), catering vehicle support node (entry, start operation, end operation, departure), refueling vehicle support node (entry, start operation, end operation, departure), and baggage conveyor vehicle support node (entry, start operation, end operation, departure); in terms of node detection accuracy, it has evolved from the traditional 1 minute to 3 minutes to 30 seconds or even shorter time requirements. In this context, higher requirements are put forward for support node detection technology. Support node identification solutions based on vehicle position inference (loading positioning devices on vehicles for vehicle positioning) and support node time inference (relying on vehicle position to infer) cannot handle the complex and changeable actual situation of airports and cannot meet the high-precision identification requirements of support nodes. In addition, the support node recognition technology based on single-frame video image analysis cannot accurately identify the working states of the support node, such as entering, starting, ending, and leaving the position, because it does not take into account the historical status of the target; in addition, occasional target omissions and false detections during the support node recognition process will lead to a decrease in the detection accuracy of the support node, affecting the stability of the support detection. Summary of the invention

[0003] The purpose of the present invention is to solve the problems of poor accuracy of detection and identification of existing support nodes, slow response and inability to perform working state conversion, and provide a flight complex support node detection method based on spatiotemporal constraints, collect parking stand monitoring video streams and perform target detection and identification through Yolov8 target detector, use ByteTrack algorithm target association to obtain target motion trajectory data, build a state automaton with spatiotemporal constraints of support nodes for state identification and state conversion, which can realize high-precision state and state conversion detection of flight support nodes, output state conversion data that advances in time sequence, and improve the flight support node state detection accuracy and recognition informationization level.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A flight complex support node detection method based on time and space constraints, the method comprising:

[0006] S1. Construct flight support annotation data including parking stand monitoring video stream images and support node annotations, and input them into Yolov8 target detector to perform target detection training for support nodes. The targets of the support nodes include jet bridges, catering trucks, refueling trucks and baggage conveyor belt trucks.

[0007] S2. Construct a state machine with time constraints, and set the states of the state machine and the transition conditions between states;

[0008] S3, collect the monitoring video stream of the parking space, input it into the Yolov8 target detector, and use the ByteTrack algorithm to associate the targets and obtain the motion trajectory data of all targets;

[0009] S4, extracting the trajectory information S(i) of the flight support node N(i) corresponding to the parking space, where i represents the target, and the state automaton performs state recognition and state transformation;

[0010] S5. The state automaton generates state transition data of the support node N(i) according to the time sequence of the flights in the parking position. The state transition data includes time and events. Events include entering the parking position, starting operation, ending operation, leaving the parking position, and ending operation and leaving the parking position.

[0011] In order to better implement the present invention, the states of the state machine include an empty state Me, a state where the target appears but does not enter the working area Mo, a state where the target enters the working area Mi, and a state where the target is working Mw.

[0012] Preferably, the transition conditions T of each state in the state automaton M are {condition A, condition B, condition C, condition D}, condition A is that the target is transferred from the empty state Me to the state Mo and remains in the state Mo for T1 time, condition B is that the target is transferred from the state Mo to the state Mi and remains in the state Mi for T2 time, condition C is that the target is transferred from the state Mi to the state Mw and remains in the state Mw for T3 time, and condition D is that the target is transferred from the state Mw to the empty state Me and remains in the empty state Me for T4 time.

[0013] Preferably, the surveillance video stream of the parking space is collected by a camera combination system, which is a plurality of cameras arranged around.

[0014] Preferably, the state transition condition determination method of the state automaton is as follows:

[0015] S41, if the trajectory information S(i) of target i at the current moment is empty, then target i is set to the empty state Me, and the state automaton combines the previous state recognition of target i to determine whether it is condition D. If it is not condition D, condition A tracking judgment is executed;

[0016] S42, if the states of the latest K1 frames in the trajectory information S(i) of target i are all empty states Me, and the state of the K1+1 frame is transferred to the state Mo and the video frames for T1 time are all in the state Mo, then set the state transfer condition T(i) = [condition A];

[0017] S43, after the state transition condition T(i) = [condition A], if the mean of the target center position of the latest K2 frames in the trajectory information S(i) of target i is within the operating area corresponding to target i, the variance of the target center position of the latest k2 frames is less than the threshold Pi, and the target state labels of the latest k2 frames are all operating states Mi, then set the state transition condition T(i) = [condition B];

[0018] S44, after the state transition condition T(i) = [condition B], if the mean of the target center position of the latest K3 frames in the trajectory information S(i) of target i is within the working area corresponding to target i, the variance of the target center position of the latest k3 frames is less than the threshold Pi, and the target state labels of the latest k3 frames are all working states Mw, then set the state transition condition T(i) = [condition C];

[0019] S45. After the state transition condition T(i)=[Condition C], if the states of the latest K4 frames in the trajectory information S(i) of target i are all empty states Me, then the state transition condition T(i)=[Condition D] is set.

[0020] Preferably, the state automaton generates state transition data according to the time sequence of flights in the parking positions as follows:

[0021] S51, if the state transition condition T(i) = [condition B], the last state of the state machine is state Mo and the current state is state Mi, the event of the state transition data is set to entry and the entry time is recorded;

[0022] S52, if the state transition condition T(i) = [condition C], the last state of the state machine is state Mi and the current state is state Mw, the event of the state transition data is set to start operation and the start operation time is recorded;

[0023] S53, if the state transition condition T(i) = [condition C], the last state of the state machine is state Mw and the current state is state Mi, the event of the state transition data is set to end the job and the end time of the job is recorded;

[0024] S54, if the state transition condition T(i) = [condition C], the last state of the state machine is state Mi and the current state is state Mo, the event of the state transition data is set to leave and the leave time is recorded;

[0025] S55. If the state transfer condition T(i) = [condition D], the current state of the state machine is state Mo, and the event of the state transition data is set to end the job and leave.

[0026] Preferably, the Yolov8 target detector detects and identifies target i on the security node, and the state automaton marks the state of unidentified target i as empty state Me; the Yolov8 target detector identifies target i, and the Yolov8 target detector has a security node work identification module inside; the processing method of the security node work identification module is as follows: the security node work identification module demarcates the working area of ​​target i based on the parking space for the parking space video frame image containing the aircraft and makes a position judgment. If the target i is entirely located in the working area, the target i is marked as target i_idle, and the state automaton sets the state of target i to state Mi; otherwise, it is only marked as target i, and the state automaton sets the state of target i to state Mo.

[0027] Preferably, the working state feature extraction and recognition are performed on the target labeled as i_idle, the target i identified as being in the working state is labeled as target i_work, and the state automaton sets the state of the target i to the state Mw.

[0028] Preferably, the Yolov8 target detector performs frame-by-frame target detection on the monitoring video stream, and the ByteTrack algorithm performs target association and tracking according to target i based on the previous and next frames of the monitoring video stream to obtain the motion trajectory data corresponding to target i.

[0029] Preferably, the video images in the flight support annotation data cover all weather scenes, all lighting modes, and all states of support nodes including jet bridges, catering trucks, refueling trucks, and baggage conveyor belt trucks.

[0030] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0031] (1) The present invention collects parking stand monitoring video streams and performs target detection and identification through Yolov8 target detector, uses ByteTrack algorithm target association to obtain target motion trajectory data, and constructs a state automaton with time and space constraints for the support node to perform state identification and state transformation. It can realize high-precision state and state transition detection of flight support nodes, output state transition data that advances in time sequence, and improve the state detection accuracy and recognition informationization level of flight support nodes.

[0032] (2) The state automation mechanism of the present invention is equipped with the state recognition and state change conditions of the support node targets such as the jet bridge, catering truck, refueling truck and luggage conveyor belt truck, and combines time constraints and model feature recognition to achieve high-precision detection of complex flight support nodes, solving the problems of low accuracy and large errors in traditional estimation, and target omission and false detection in single-frame video analysis. It solves the technical problems of low support accuracy and high cost in existing solutions, making the flight support system run more stably. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the method of the present invention;

[0034] Figure 2 Schematic diagram of the transfer condition principle of the state automaton in the embodiment;

[0035] Figure 3 It is a schematic diagram of the target recognition of the baggage conveyor belt vehicle security node by the Yolov8 target detector in the embodiment;

[0036] Figure 4 It is a schematic diagram of the target recognition of the corridor bridge security node by the Yolov8 target detector in the embodiment;

[0037] Figure 5 It is a schematic diagram of the Yolov8 target detector identifying the working status of the corridor bridge security node in the embodiment. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with embodiments:

[0039] Example

[0040] like Figure 1 As shown, a flight complex support node detection method based on time and space constraints includes:

[0041] S1. Construct flight support annotation data including parking stand monitoring video stream images (in this embodiment, the monitoring video stream of the parking stand is collected by a camera joint system, and the camera joint system is a number of cameras set up around) and support node annotations, and input them into the Yolov8 target detector to perform target detection training for support nodes. The targets of the support nodes of the present invention include four categories: jet bridges, catering trucks, refueling trucks, and luggage conveyor belt trucks. The video images in the flight support annotation data cover all weather scenes, all lighting modes, and all states of support nodes including jet bridges, catering trucks, refueling trucks, and luggage conveyor belt trucks. The flight support annotation data is a flight support annotation dataset D. The flight support annotation dataset D includes video stream images and support node annotations on the video stream images.

[0042] S2. Construct a state automaton with time constraints, and set the states of the state automaton and the transition conditions between states. The state automaton M of the present invention contains the time constraint information of the target, which can ensure that the detection of the security node is more accurate, and is not affected by occasional erroneous target detection results that lead to misidentification of the security node, thereby improving the stability of the node detection scheme. In some embodiments, the states of the state automaton include an empty state Me, a state Mo where the target appears but has not entered the working area, a state Mi where the target enters the working area, and a state Mw where the target is working. The transition conditions T of each state in the state automaton M = {condition A, condition B, condition C, condition D}, where condition A is that the target transitions from the empty state Me to the state Mo and remains in the state Mo for T1 time (which can be replaced by a continuous number of P1 video frames); Figure 2 As shown, in this embodiment, condition A is further provided that the target changes from state Mi to state Mo and lasts for T1 time. Condition B is that the target changes from state Mo to state Mi and lasts for T2 time (which can be replaced by P2 consecutive video frames) in state Mi; Figure 2 As shown, condition B of this embodiment is further set to have the target change from state Mw to state Mi for T1 time. Condition C is that the target changes from state Mi to state Mw and stays in state Mw for T3 time (which can be replaced by P3 consecutive video frames), and condition D is that the target changes from state Mw to empty state Me and stays in empty state Me for T4 time (which can be replaced by P4 consecutive video frames). For simple example, the time setting range of T1 to T4 in this embodiment is 2 to 5 seconds.

[0043] S3, collect the monitoring video stream of the parking position, input it into the Yolov8 target detector, and use the ByteTrack algorithm to associate the targets and obtain the motion trajectory data of all targets. Preferably, the monitoring video stream of the parking position in this embodiment is collected by a camera joint system, and the camera joint system is a number of cameras arranged around; if multiple cameras are used to form a camera joint system, the multiple cameras are respectively arranged at different monitoring positions of the parking position and assigned with the task of ensuring node detection, and the video frame image packages of all cameras at the same time are subjected to target detection. The Yolov8 target detector detects and identifies the target i on the guarantee node, and the state automaton marks the state of the target i not being identified as the empty state Me. The Yolov8 target detector identifies the target i, and the Yolov8 target detector has a guarantee node work identification module inside. The processing method of the guarantee node work identification module is as follows: the guarantee node work identification module demarcates the working area of ​​the target i based on the parking position for the parking position video frame image containing the aircraft and performs position judgment. If the target i is all located in the working area, the target i is marked as target i_idle, and the state automaton sets the state of the target i to state Mi; otherwise, it is only marked as target i. The state automaton sets the state of target i to state Mo. The feature extraction and recognition of the working state are performed on the target i_idle, and the target i identified as the working state is marked as target i_work. The state automaton sets the state of target i to state Mw.

[0044] Target i includes four types of monitoring targets: corridor bridge, catering vehicle, refueling vehicle and luggage conveyor belt vehicle. If target i is a corridor bridge, Yolov8 target detector identifies the corridor bridge target in the video frame image and marks it as lq (lq_idle and lq_work will be marked later); Figure 4 As shown, this embodiment uses the Yolov8 target detector to perform target detection and identification on the corridor bridge support node. If the corridor bridge target is identified, it is first marked as lq. If the Yolov8 target detector detects that the corridor bridge is located in the corridor bridge working area (the Yolov8 target detector demarcates the corridor bridge working area based on the aircraft fuselage), but does not recognize that the corridor bridge is in a working state (the support node work identification module performs feature extraction and identification of the working state, whether the corridor bridge is docked with the aircraft fuselage), it is marked as lq_idle, and the state automaton sets the state of the corridor bridge lq_ to state Mi. If the Yolov8 target detector detects that the corridor bridge is located in the corridor bridge working area and recognizes that the corridor bridge is in a working state, it is marked as lq_work. Figure 5 As shown, in this embodiment, Yolov8 target detector is used to detect and identify the corridor bridge security node. If the security node work identification module detects that the corridor bridge is already in the working state, it is marked as lq_work. The state automaton sets the state of the corridor bridge lq_ to the state Mw.

[0045] If target i is a catering vehicle, the Yolov8 target detector identifies the catering vehicle target in the video frame image and marks it as pcc (pcc_idle and pcc_work will be marked later). If the Yolov8 target detector detects that the catering vehicle is located in the catering vehicle working area (the Yolov8 target detector uses the aircraft fuselage as a reference to define the catering vehicle working area), but does not recognize that the catering vehicle is in working condition, it is marked as pcc_idle, and the state automaton sets the state of the catering vehicle pcc_ to state Mi. If the Yolov8 target detector detects that the catering vehicle is located in the catering vehicle working area and recognizes that the catering vehicle is in working condition (the guarantee node work identification module performs feature extraction and identification of the working condition, and whether the catering vehicle's catering operation is raised), it is marked as pcc_work, and the state automaton sets the state of the catering vehicle pcc_ to state Mw.

[0046] If target i is a refueling truck, the Yolov8 target detector identifies the refueling truck target in the video frame image and marks it as jyc (jyc_idle and jyc_work will be marked later). If the Yolov8 target detector detects that the refueling truck is located in the refueling truck working area (Yolov8 target detector demarcates the refueling truck working area based on the aircraft fuselage), but does not recognize that the refueling truck is in working state, it is marked as jyc_idle, and the state automaton sets the state of the refueling truck jyc_ to state Mi. If the Yolov8 target detector detects that the refueling truck is located in the refueling truck working area and recognizes that the refueling truck is in working state (the node work identification module performs feature extraction and identification of the working state, whether the refueling truck is docked with the aircraft fuselage or the refueling pipe is connected), it is marked as jyc_work, and the state automaton sets the state of the refueling truck jyc_ to state Mw.

[0047] If target i is a baggage conveyor belt vehicle, the Yolov8 target detector identifies the baggage conveyor belt vehicle target in the video frame image and marks it as xlcsd (xlcsd_idle and xlcsd_work will be marked later), such as Figure 3As shown, this embodiment uses the Yolov8 target detector to detect and identify the baggage conveyor belt vehicle support node. If the baggage conveyor belt vehicle target is identified, it is first marked as xlcsd. If the Yolov8 target detector detects that the baggage conveyor belt vehicle is located in the working area of ​​the baggage conveyor belt vehicle (the Yolov8 target detector uses the aircraft fuselage as a reference to define the working area of ​​the baggage conveyor belt vehicle), but does not recognize that the baggage conveyor belt vehicle is in a working state (the support node working identification module performs feature extraction and identification of the working state, and whether the baggage conveyor belt vehicle is docked with the aircraft fuselage), it is marked as xlcsd_idle, and the state automaton sets the state of the baggage conveyor belt vehicle xlcsd_ to state Mi; Figure 3 As shown, in this embodiment, Yolov8 target detector is used to detect and identify the baggage conveyor belt vehicle support node. If the support node work identification module detects that the baggage conveyor belt vehicle is not docked with the aircraft fuselage, it is marked as xlcsd_idle. If Yolov8 target detector detects that the baggage conveyor belt vehicle is located in the working area of ​​the baggage conveyor belt vehicle and identifies that the baggage conveyor belt vehicle is in working state, it is marked as xlcsd_work, and the state automaton sets the state of the baggage conveyor belt vehicle xlcsd_ to state Mw; Figure 3 As shown, this embodiment uses the Yolov8 target detector to detect and identify the baggage conveyor belt vehicle support node. If the support node work identification module detects that the baggage conveyor belt vehicle is docked with the aircraft fuselage, it is marked as xlcsd_work.

[0048] The Yolov8 target detector of the present invention performs frame-by-frame target detection on the monitoring video stream, and the ByteTrack algorithm performs target association and tracking according to target i based on the previous and next frames of the monitoring video stream to obtain the motion trajectory data corresponding to target i (forming a continuous motion trajectory containing different states of the same target).

[0049] S4. Extract the trajectory information S(i) of the flight support node N(i) corresponding to the parking space, where i represents the target (four types of monitoring target objects: jet bridge, catering truck, refueling truck and baggage conveyor belt truck). The state automaton performs state recognition and state transformation.

[0050] S5. The state automaton generates state transition data of the support node N(i) according to the time sequence of the flights in the parking position. The state transition data includes time and events. Events include entering the parking position, starting operation, ending operation, leaving the parking position, and ending operation and leaving the parking position.

[0051] In some embodiments, the states of the state machine include an empty state Me, a state Mo where the target appears but does not enter the working area, a state Mi where the target enters the working area, and a state Mw where the target is working. The transition conditions T of each state in the state machine M = {condition A, condition B, condition C, condition D}, and the state transition condition determination method of the state machine is as follows:

[0052] S41. If the trajectory information S(i) of target i at the current moment is empty, target i is set to the empty state Me. The state automaton combines the previous state recognition of target i to determine whether it is condition D. If it is not condition D, condition A tracking judgment is executed.

[0053] S42. If the states of the most recent K1 frames (the time interval between adjacent video frames in this embodiment is 0.2 seconds, and k1 is 10) in the trajectory information S(i) of target i are all empty states Me, and the state of the K1+1 frame is transferred to state Mo and the video frames (which can be replaced by M1 consecutive video frames, if the time interval between adjacent video frames is 0.2 seconds, then M1 is 10 to 25) that last for T1 time (in this embodiment, T1 time is set to 2 to 5 seconds) are all in state Mo, then set the state transition condition T(i) = [condition A].

[0054] S43. After the state transition condition T(i) = [condition A], if the mean of the target center position of the latest K2 frames in the trajectory information S(i) of target i is within the working area corresponding to target i, the variance of the target center position of the latest k2 frames (the time interval between adjacent video frames in this embodiment is 0.2 seconds, and the value of k2 is 10) is less than the threshold Pi (the threshold Pi is set according to target i), and the target state labels of the latest k2 frames are all working states Mi, then the state transition condition T(i) = [condition B] is set.

[0055] S44. After the state transition condition T(i) = [condition B], if the mean of the target center position of the latest K3 frames in the trajectory information S(i) of target i is within the working area corresponding to target i, the variance of the target center position of the latest k3 frames (the time interval between adjacent video frames in this embodiment is 0.2 seconds, and the value of k2 is 12) is less than the threshold Pi (the threshold Pi is set according to target i), and the target state labels of the latest k3 frames are all working states Mw, then the state transition condition T(i) = [condition C] is set.

[0056] S45. After the state transition condition T(i)=[Condition C], if the states of the latest K4 frames in the trajectory information S(i) of target i are all empty states Me, then the state transition condition T(i)=[Condition D] is set.

[0057] In some embodiments, the state automaton generates state transition data according to the time sequence of flights in the parking stands as follows:

[0058] S51. If the state transfer condition T(i) = [condition B], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor belt truck), the last state of the state machine (i.e., the state of target i in the previous video frame) is state Mo and the current state (i.e., the state of target i in the current video frame) is state Mi, the event of target i in the state transition data is set to entry and the entry time is recorded.

[0059] S52. If the state transfer condition T(i) = [condition C], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor truck), the last state of the state machine (i.e., the state of target i in the previous video frame) is state Mi and the current state (i.e., the state of target i in the current video frame) is state Mw, and the event of target i in the state transition data is set to start the operation and the start time of the operation is recorded.

[0060] S53. If the state transfer condition T(i) = [condition C], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor belt truck), the last state of the state machine (i.e., the state of target i in the previous video frame) is state Mw and the current state (i.e., the state of target i in the current video frame) is state Mi, the event of target i in the state transition data is set to end the operation and the end time of the operation is recorded.

[0061] S54. If the state transfer condition T(i) = [condition C], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor belt truck), the last state of the state machine (i.e., the state of target i in the previous video frame) is state Mi and the current state (i.e., the state of target i in the current video frame) is state Mo, the event of target i in the state transition data is set to leave and the leaving time is recorded.

[0062] S55. If the state transfer condition T(i) = [condition D], i represents the target object (i.e., the four target objects of the corridor bridge, catering truck, refueling truck and luggage conveyor belt truck), the current state of the state automaton is state Mo, and the event of target i in the state transition data is set to end the operation and leave.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A flight complex support node detection method based on time and space constraints, characterized by: The methods include: S1. Construct flight support annotation data including parking stand monitoring video stream images and support node annotations, and input them into Yolov8 target detector to perform target detection training for support nodes. The targets of the support nodes include jet bridges, catering trucks, refueling trucks and baggage conveyor belt trucks. S2. Construct a state machine with time constraints, and set the states of the state machine and the transition conditions between states; S3, collect the monitoring video stream of the parking space, input it into the Yolov8 target detector, and use the ByteTrack algorithm to associate the targets and obtain the motion trajectory data of all targets; S4, extracting the trajectory information S(i) of the flight support node N(i) corresponding to the parking space, where i represents the target, and the state automaton performs state recognition and state transformation; S5. The state automaton generates state transition data of the support node N(i) according to the time sequence of the flights in the parking position. The state transition data includes time and events. Events include entering the parking position, starting operation, ending operation, leaving the parking position, and ending operation and leaving the parking position.

2. The method for detecting complex flight support nodes based on time and space constraints according to claim 1, characterized in that: The states of the state automaton include an empty state Me, a state Mo in which the target appears but does not enter the working area, a state Mi in which the target enters the working area, and a state Mw in which the target is working.

3. The method for detecting complex flight support nodes based on time and space constraints according to claim 2, characterized in that: The transition conditions T of each state in the state automaton M are {condition A, condition B, condition C, condition D}, where condition A is that the target is transferred from the empty state Me to the state Mo and remains in the state Mo for T1 time, condition B is that the target is transferred from the state Mo to the state Mi and remains in the state Mi for T2 time, condition C is that the target is transferred from the state Mi to the state Mw and remains in the state Mw for T3 time, and condition D is that the target is transferred from the state Mw to the empty state Me and remains in the empty state Me for T4 time.

4. The method for detecting complex flight support nodes based on time and space constraints according to claim 1, characterized in that: The surveillance video stream of the parking space is collected through a camera joint system, which is a number of cameras installed around.

5. The method for detecting complex flight support nodes based on time and space constraints according to claim 2, characterized in that: The state transition condition determination method of the state automaton is as follows: S41, if the trajectory information S(i) of target i at the current moment is empty, then target i is set to the empty state Me, and the state automaton combines the previous state recognition of target i to determine whether it is condition D. If it is not condition D, condition A tracking judgment is executed; S42, if the states of the latest K1 frames in the trajectory information S(i) of target i are all empty states Me, and the state of the K1+I frame is transferred to the state Mo and the video frames for T1 time are all in the state Mo, then set the state transfer condition T(i) = [condition A]; S43, after the state transition condition T(i) = [condition A], if the mean of the target center position of the latest K2 frames in the trajectory information S(i) of target i is within the operating area corresponding to target i, the variance of the target center position of the latest k2 frames is less than the threshold Pi, and the target state labels of the latest k2 frames are all operating states Mi, then set the state transition condition T(i) = [condition B]; S44, after the state transition condition T(i) = [condition B], if the mean of the target center position of the latest K3 frames in the trajectory information S(i) of target i is within the working area corresponding to target i, the variance of the target center position of the latest k3 frames is less than the threshold Pi, and the target state labels of the latest k3 frames are all working states Mw, then set the state transition condition T(i) = [condition C]; S45. After the state transition condition T(i)=[Condition C], if the states of the latest K4 frames in the trajectory information S(i) of target i are all empty states Me, then the state transition condition T(i)=[Condition D] is set.

6. The method for detecting complex flight support nodes based on time and space constraints according to claim 5, characterized in that: The method for the state automaton to generate state transition data according to the time sequence of flights in the parking positions is as follows: S51, if the state transition condition T(i) = [condition B], the last state of the state machine is state Mo and the current state is state Mi, the event of the state transition data is set to entry and the entry time is recorded; S52, if the state transition condition T(i) = [condition C], the last state of the state machine is state Mi and the current state is state Mw, the event of the state transition data is set to start operation and the start operation time is recorded; S53, if the state transition condition T(i) = [condition C], the last state of the state machine is state Mw and the current state is state Mi, the event of the state transition data is set to end the job and the end time of the job is recorded; S54, if the state transition condition T(i) = [condition C], the last state of the state machine is state Mi and the current state is state Mo, the event of the state transition data is set to leave and the leave time is recorded; S55. If the state transfer condition T(i) = [condition D], the current state of the state machine is state Mo, and the event of the state transition data is set to end the job and leave.

7. The method for detecting complex flight support nodes based on time and space constraints according to claim 1, characterized in that: The Yolov8 target detector detects and identifies target i on the support node, and the state automaton marks the state of unidentified target i as empty state Me; the Yolov8 target detector identifies target i, and the Yolov8 target detector has a support node work identification module inside; the processing method of the support node work identification module is as follows: the support node work identification module demarcates the working area of ​​target i based on the parking space for the parking space video frame image containing the aircraft and makes a position judgment. If the target i is entirely located in the working area, the target i is marked as target i_idle, and the state automaton sets the state of target i to state Mi; otherwise, it is only marked as target i, and the state automaton sets the state of target i to state Mo.

8. The method for detecting complex flight support nodes based on time and space constraints according to claim 7 is characterized in that: The working state feature extraction and recognition are performed on the target i_idle, the target i identified as the working state is marked as target i_work, and the state automaton sets the state of the target i to the state Mw.

9. The method for detecting complex flight support nodes based on time and space constraints according to claim 1, characterized in that: The Yolov8 target detector performs frame-by-frame target detection on the monitoring video stream, and the ByteTrack algorithm performs target association and tracking according to target i based on the previous and next frames of the monitoring video stream to obtain the motion trajectory data corresponding to target i.

10. The method for detecting complex flight support nodes based on time and space constraints according to claim 1, characterized in that: The video images in the flight support annotation data cover all weather scenes, all lighting modes, and all states of support nodes including jet bridges, catering trucks, refueling trucks, and baggage conveyor belt trucks.

Images