Airport parking space safety supervision method and system

Abstract

The invention discloses an airport gate position safety supervision method and system. The method comprises the following steps: acquiring multi-code stream video data acquired in an aircraft engine area; performing decoding, gray level conversion and caching on video data of each code stream in the multi-code-stream video data to generate a corresponding time sequence gray level image queue; all grayscale images are extracted from the time sequence grayscale image queue every time T, and a three-dimensional time sequence array P with the dimension of M * N * K is generated through superposition according to the time sequence; performing autocorrelation calculation on each pixel position in the three-dimensional time-order array P to generate a normalized autocorrelation coefficient matrix; performing spatial convolution accumulation on the autocorrelation coefficient matrix to obtain an accumulation matrix; if elements higher than the preset threshold exist in the cumulative matrix, it is judged that the engine is in a rotating state. According to the airport parking space safety supervision method and system, the problem of misjudgment caused by low illumination, a stroboscopic effect and local interference is solved, and the accuracy and reliability of parking space safety supervision are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of aviation safety, and particularly to an airport apron parking position safety supervision method and system. Background Art

[0002] In airport apron operations, after an aircraft taxis into a parking position and comes to a stop, if the engine is not completely shut down, there is a major safety hazard that maintenance personnel or vehicles entering the red line area may be sucked into the engine. The existing technologies mainly have the following defects: strong light dependence, traditional image detection methods cannot clearly capture the movement of engine fan blades under low illuminance or limited dynamic range, resulting in difficult manual marking. False judgment due to stroboscopic effect, when the engine speed is synchronized with the camera frame rate, the picture appears static (for example, a rotation speed of 60 revolutions per second with a frame rate of 60 frames per second), which is likely to misjudge the engine as being in the off state. Insufficient anti-interference ability, local light flicker or camera noise may be misidentified as a periodic motion signal. Summary of the Invention

[0003] To solve the technical problems in the background art, the present invention proposes an airport apron parking position safety supervision method and system.

[0004] An airport apron parking position safety supervision method proposed by the present invention includes the following steps:

[0005] S1. Obtain multi-stream video data collected in the aircraft engine area;

[0006] S2. Decode, perform grayscale conversion, and cache the video data of each stream in the multi-stream video data to generate a corresponding time-sequence grayscale image queue;

[0007] S3. Extract all grayscale images from the time-sequence grayscale image queue every time interval T, and stack them in chronological order to generate a three-dimensional time-sequence array P with dimensions M×N×K;

[0008] S4. Perform autocorrelation calculation on each pixel position in the three-dimensional time-sequence array P to generate a normalized autocorrelation coefficient matrix;

[0009] S5. Perform spatial convolution accumulation on the autocorrelation coefficient matrix to obtain an accumulation matrix;

[0010] S6. If there are elements in the accumulation matrix higher than a preset threshold, determine that the engine is in a rotating state;

[0011] S7. Repeat steps S2 to S6 for the video data of each stream. When the engine state corresponding to the video data of any stream is in the rotating state, finally output the engine state as the rotating state.

[0012] Preferably, step S2 specifically includes:

[0013] Decode the video data of each bitstream, convert the decoded RGB image into a grayscale image, and cache it into the sequential grayscale image queue. Each bitstream's video data corresponds to a sequential grayscale image queue. Among them, the conversion formula is as follows:

[0014] Gray = 0.299×R + 0.587×G + 0.114×B;

[0015] Among them, R, G, and B are the red, green, and blue components of the RGB color image respectively; Gray is the grayscale image;

[0016] The length of each sequential grayscale image queue satisfies:

[0017]

[0018] Among them, t is the caching time; i = 1 or 2; L i is the length of the sequential grayscale image queue.

[0019] Preferably, step S1 specifically includes:

[0020] Synchronously collect multi-bitstream video data of the aircraft engine area through at least two camera bitstreams. The resolution of each bitstream is M×N, and the frame rates are f1 and f2 respectively. f1 and f2 are integers and are relatively prime numbers, and the shutter time does not exceed 1 / 50 second.

[0021] Preferably, step S4 specifically includes:

[0022] Take out all the channel data at each pixel position (x, y) in the sequential array P, and denote it as the sequential data vector V = [v1, v2,... v K ;

[0023] Calculate the normalized autocorrelation coefficient of the sequential data vector V. The calculation formula is:

[0024]

[0025] Among them, R(x, y, τ) is the autocorrelation coefficient; σ 2 is the variance; is the mean value of each channel data; τ is the sequential delay;

[0026] Repeat the above steps to obtain the normalized autocorrelation coefficient matrix R with dimensions M×N×K.

[0027] Preferably, step S5 specifically includes:

[0028] Perform spatial convolution accumulation on the autocorrelation coefficient matrix using a circular convolution kernel with a size of m×m. The convolution kernel definition satisfies: (u - m / 2) 2+(v - m / 2) 2 ≤r 2 Set the position where it is 2

[0029] Traverse the two-dimensional array R of each channel i For each element in the two-dimensional array R i Assume the current element position is (i, j);

[0030] Calculate the sum of element products C of the overlapping part between the current element position (i, j) and the circular convolution kernel H(u, v) i,j The calculation formula is as follows:

[0031]

[0032] Store the convolution result of each channel in the corresponding cumulative matrix C

[0033] Preferably, step S6 specifically includes:

[0034] If there are elements with channel i > 1 in the cumulative matrix C and satisfy:

[0035] C(x, y, i) > T1;

[0036] Then it is determined that the engine is in a rotating state, where T1 is a preset threshold; C(x, y, i) is the element with channel i > 1 in the cumulative matrix C

[0037] Preferably, it further includes:

[0038] S8. Real-time detect the number of people in the aircraft engine area. When the engine is in a rotating state and the detected number of people is greater than the preset number of people, trigger real-time alarm

[0039] An airport parking space safety supervision system proposed by the present invention includes:

[0040] A data acquisition module for acquiring multi-stream video data collected in the aircraft engine area

[0041] An image caching module for decoding, grayscale conversion and caching the video data of each stream in the multi-stream video data to generate a corresponding time-series grayscale image queue

[0042] A time-series array generation module for extracting all grayscale images from the time-series grayscale image queue every time T and stacking them in chronological order to generate a three-dimensional time-series array P with dimensions M×N×K

[0043] A picture time-series period analysis module for performing autocorrelation calculation on each pixel position in the three-dimensional time-series array P to generate a normalized autocorrelation coefficient matrix

[0044] A convolution module, which is used to perform spatial convolution accumulation on the autocorrelation coefficient matrix to obtain an accumulation matrix;

[0045] A determination module, which is used to determine that the engine is in a rotating state if there are elements in the accumulation matrix higher than a preset threshold;

[0046] An output module, which is used to repeat the determination operation on the video data of each code stream. When the engine state corresponding to the video data of any code stream is in a rotating state, the engine state is finally output as the rotating state.

[0047] Preferably, it further includes:

[0048] An aircraft maintenance personnel quantity detection module, which is used to detect the number of personnel in the aircraft engine area in real time. When the engine is in a rotating state and the detected number of personnel is greater than the preset number of personnel, a real-time alarm is triggered.

[0049] In the present invention, the proposed airport apron safety supervision method and system accurately detect the start-stop state of the engine and monitor the entry of personnel in real time through dual co-prime frame rate video acquisition, time-series autocorrelation analysis, and spatial convolution filtering. It solves the misjudgment problems caused by low illuminance, stroboscopic effect, and local interference, and significantly improves the accuracy and reliability of apron safety supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the working process structure of an airport apron safety supervision method proposed by the present invention;

[0051] Figure 2 It is a schematic diagram of the implementation process structure of an airport apron safety supervision method proposed by the present invention;

[0052] Figure 3 It is a schematic diagram of the system architecture of an airport apron safety supervision system proposed by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0053] Referring to Figures 1-3 , an airport apron safety supervision method proposed by the present invention includes the following steps:

[0054] S1. Obtain multi-code stream video data collected in the aircraft engine area.

[0055] In this embodiment, step S1 specifically includes:

[0056] Collect multi-stream video data of the aircraft engine area through at least two camera streams synchronously. The resolution of each stream is M×N, and the frame rates are f1 and f2 respectively. f1 and f2 are integers and are relatively prime numbers, and the shutter time does not exceed 1 / 50 second. Typical setting 1: frame rate f1 = 22 frames / second, f2 = 25 frames / second; Typical setting 2: frame rate f1 = 59 frames / second, f2 = 60 frames / second.

[0057] Specifically, in the present invention, technologies such as video dual-stream, temporal autocorrelation coefficient calculation, and spatial accumulation are adopted, which can effectively improve the accuracy of engine rotation detection.

[0058] The video dual-stream avoids the stroboscopic effect. When the engine speed is relatively high and exceeds the camera frame rate f, image undersampling occurs, resulting in the stroboscopic effect. When the engine speed is an integer multiple of f, it may cause the image of the engine to appear stationary, leading to misjudgment as stationary.

[0059] For example, when the engine speed is 3600 revolutions per minute (60 revolutions per second) and the camera frame rate is 60 frames per second, the image of the engine appears stationary.

[0060] When the engine is taxiing, the speed is maintained at 3000 - 9000 revolutions per minute (50 - 150 revolutions per second). To ensure that when the engine speed is 50 - 150 revolutions per second, it is still possible to determine whether the engine is rotating, a dual-stream with different frame rates is adopted. And it is ensured that the two sampling rates are relatively prime numbers, such as f1 = 22, f2 = 25, or f1 = 59, f2 = 60. In this way, when the engine speed is f1, the stream with frame rate f2 can still accurately determine whether the engine is rotating.

[0061] When the engine speed is the least common multiple LCM(f1,f2) = f1*f2 of f1 and f2, the stroboscopic effect will occur. At this time, the typical engine speed is 22*25*60 = 33000 revolutions per minute (f1 = 22, f2 = 25) or 59*60*60 = 212,400 revolutions per minute (f1 = 59, f2 = 60), which exceeds the normal taxiing speed of the engine. Therefore, during ground taxiing, misjudgment of the engine rotation state due to the stroboscopic effect is avoided.

[0062] S2. Decode, convert to grayscale, and cache the video data of each stream in the multi-stream video data to generate a corresponding queue of temporal grayscale images.

[0063] In this embodiment, step S2 specifically includes:

[0064] Decode the video data of each bitstream, convert the decoded RGB image into a grayscale image, and cache it into the time-series grayscale image queue. Each bitstream's video data corresponds to a time-series grayscale image queue. The conversion formula is as follows:

[0065] Gray = 0.299×R + 0.587×G + 0.114×B;

[0066] where R, G, and B are the red, green, and blue components of the RGB color image respectively; Gray is the grayscale image;

[0067] The length of each time-series grayscale image queue satisfies:

[0068]

[0069] where t is the caching time; i = 1 or 2; L i is the length of the time-series grayscale image queue.

[0070] Specifically, obtain the compressed video data through protocols such as RTSP / GB28181, and decode the H.264 or H.265 video data into video frames. Regularly delete the expired pictures in the time-series grayscale image queue.

[0071] S3. Extract all the grayscale images from the time-series grayscale image queue every time interval T, and stack them in chronological order to generate a three-dimensional time-series array P with dimensions M×N×K.

[0072] S4. Perform autocorrelation calculation on each pixel position in the three-dimensional time-series array P to generate a normalized autocorrelation coefficient matrix.

[0073] In this embodiment, step S4 specifically includes:

[0074] Take out all the channel data at each pixel position (x, y) in the time-series array P, and denote it as the time-series data vector V = [v1, v2,... v K ;

[0075] Calculate the normalized autocorrelation coefficient of the time-series data vector V. The calculation formula is:

[0076]

[0077] where R(x, y, τ) is the autocorrelation coefficient; v 2 is the variance; is the mean value of each channel data; τ is the time-series delay;

[0078] Repeat the above steps to obtain a normalized autocorrelation coefficient matrix R with dimensions M×N×K.

[0079] Specifically, calculate the mean value of the data for each channel The calculation formula is as follows:

[0080]

[0081] Calculate the variance σ 2 , and the calculation formula is as follows:

[0082]

[0083] Specifically, the calculation of the autocorrelation coefficient of the time series can accurately detect the periodic signal in the dark part when the dynamic range is limited or the illumination is low. When the dynamic range of the surveillance camera is limited or the imaging quality is poor under low illumination, the fan part cannot be clearly seen, and it is impossible to see whether the fan is in a rotating state. By performing time series superposition on the camera image and calculating the autocorrelation coefficient, it is possible to ensure that the periodicity of the fan rotation picture data can still be detected under low illumination.

[0084] S5. Perform spatial convolution accumulation on the autocorrelation coefficient matrix to obtain an accumulation matrix.

[0085] In this embodiment, step S5 specifically includes:

[0086] Perform spatial convolution accumulation on the autocorrelation coefficient matrix using a circular convolution kernel of size m×m. The convolution kernel is defined to satisfy: (u - m / 2) 2 +(v - m / 2) 2 ≤r 2 Set the positions to 1 and the remaining positions to zero;

[0087] Traverse the two-dimensional array R of each channel i , for each element in the two-dimensional array R i , assume the current element position is (i, j);

[0088] Calculate the sum of the element products C of the overlapping part between the current element position (i, j) and the circular convolution kernel H(u, v) i,j , and the calculation formula is as follows:

[0089]

[0090] Store the convolution result of each channel in the corresponding accumulation matrix C.

[0091] Specifically, in step S5, the convolution kernel radius r = 3 pixels and m = 7, which is used to filter out interference signals with an area smaller than πr 2 .

[0092] S6. If there are elements in the accumulation matrix that are higher than the preset threshold, it is determined that the engine is in a rotating state.

[0093] In this embodiment, step S6 specifically includes:

[0094] If there are elements with channel i > 1 in the cumulative matrix C and satisfy:

[0095] C(x, y, i) > T1;

[0096] Then it is determined that the engine is in a rotating state, where T1 is a preset threshold; C(x, y, i) is an element with channel i > 1 in the cumulative matrix C.

[0097] Specifically, in step S6, the preset threshold T1 = 0.8, and it is required to meet the condition for 3 consecutive frames during determination.

[0098] Specifically, spatial accumulation can exclude the interference caused by accidental factors. The brightness of small - area points in the video shows periodicity due to light flickering, resulting in misjudgment as the engine rotating. Cumulating the autocorrelation coefficient matrix by convolution can exclude the interference caused by small - area light flickering.

[0099] S7. Repeat steps S2 to S6 for the video data of each code stream. When the engine state corresponding to the video data of any code stream is in a rotating state, the engine state is finally output as the rotating state.

[0100] Specifically, if each code stream is judged as the engine being stationary according to steps S2 to S6, it is determined that the engine is in a stationary state.

[0101] In this embodiment, it further includes:

[0102] S8. Real - time detect the number of people in the aircraft engine area. When the engine is in a rotating state and the detected number of people is greater than the preset number of people, trigger real - time alarm.

[0103] Specifically, from the camera image, crop the picture of the aircraft engine range, use a deep - learning classification model or detection model to analyze the picture, infer whether there are people near the engine, and compare the extracted number of people with the preset number of people. When the obtained number of people is greater than the preset number of people and at this time the engine is determined to be in a rotating state through the above steps, trigger real - time alarm. Preferably, use the EfficientnetV2 or Yolov12 model to perform transfer learning for this scenario and infer whether there are people near the engine.

[0104] Refer to Figures 1-3 , a kind of airport parking space safety supervision system proposed by the present invention includes:

[0105] A data acquisition module, used to acquire multi - code - stream video data collected in the aircraft engine area;

[0106] An image cache module, configured to decode, perform grayscale conversion, and cache the video data of each code stream in the multi-code stream video data, and generate a corresponding sequential grayscale image queue;

[0107] A sequential array generation module, configured to extract all grayscale images from the sequential grayscale image queue every time T, and stack them in chronological order to generate a three-dimensional sequential array P with dimensions of M×N×K;

[0108] A picture sequential period analysis module, configured to perform autocorrelation calculation on each pixel position in the three-dimensional sequential array P to generate a normalized autocorrelation coefficient matrix;

[0109] A convolution module, configured to perform spatial convolution accumulation on the autocorrelation coefficient matrix to obtain an accumulation matrix;

[0110] A determination module, configured to determine that the engine is in a rotating state if there is an element in the accumulation matrix higher than a preset threshold;

[0111] An output module, configured to repeat the determination operation on the video data of each code stream. When the engine state corresponding to the video data of any one code stream is in a rotating state, the engine state is finally output as a rotating state.

[0112] In this embodiment, it further includes:

[0113] An aircraft maintenance personnel quantity detection module, configured to detect the number of personnel in the aircraft engine area in real time. When the engine is in a rotating state and the detected number of personnel is greater than a preset number of personnel, trigger a real-time alarm.

[0114] As mentioned above, the above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered within the protection scope of the present invention.

Claims

1. An airport parking bay safety supervision method, characterized in that, It includes the following steps: S1. Obtain multi-stream video data collected in the aircraft engine area; S2. Decode, perform grayscale conversion, and cache the video data of each stream in the multi-stream video data to generate a corresponding time-series grayscale image queue; S3. Extract all grayscale images from the time-series grayscale image queue every time T, and stack them in chronological order to generate a three-dimensional time-series array P with dimensions of M×N×K; S4. Perform autocorrelation calculation on each pixel position in the three-dimensional time-series array P to generate a normalized autocorrelation coefficient matrix; S5. Perform spatial convolution accumulation on the autocorrelation coefficient matrix to obtain an accumulation matrix; S6. If there are elements in the accumulation matrix higher than a preset threshold, determine that the engine is in a rotating state; S7. Repeat steps S2 to S6 for the video data of each stream. When the engine state corresponding to the video data of any one stream is in a rotating state, finally output the engine state as the rotating state.

2. The airport apron parking space safety supervision method according to claim 1, characterized in that Step S2 specifically includes: Decode the video data of each stream, convert the decoded RGB image into a grayscale image, and cache it into the time-series grayscale image queue. Each stream of video data corresponds to a time-series grayscale image queue; among them, the conversion formula is as follows: Gray = 0.299×R + 0.587×G + 0.114×B; where R, G, and B are the red, green, and blue components of the RGB color image respectively; Gray is the grayscale image; The length of each time-series grayscale image queue satisfies: where t is the caching time; i = 1 or 2; L i is the length of the sequential grayscale image queue.

3. The airport apron parking space safety supervision method according to claim 1, characterized in that Step S1 specifically includes: Collect multi-stream video data in the aircraft engine area through at least two camera streams synchronized. The resolution of each stream is M×N, and the frame rates are f1 and f2 respectively. f1 and f2 are integers and are relatively prime numbers, and the shutter time does not exceed 1 / 50 second.

4. The airport apron safety supervision method according to claim 1, characterized in that Step S4 specifically includes: Fetch all channel data at each pixel position (x, y) in the time series array P, and denote it as the time series data vector V = [v1, v2, … v K ; Calculate the normalized autocorrelation coefficient of the time-series data vector V. The calculation formula is: Among them, R(x, y, τ) is the autocorrelation coefficient; σ 2 is the variance; is the mean of the data for each channel; τ is the time series delay; Repeat the above steps to obtain a normalized autocorrelation coefficient matrix R with dimensions of M×N×K.

5. The airport apron parking space safety supervision method according to claim 1, wherein, Step S5 specifically includes: The autocorrelation coefficient matrix is spatially convolutionally accumulated using a circular convolution kernel of size m×m, and the convolution kernel is defined to satisfy: (u - m / 2) 2 +(v - m / 2) 2 ≤r 2 The positions are set to 1, and the remaining positions are set to zero; Traverse the two-dimensional array R of each channel i , for each element in the two-dimensional array R i , assume the current element position is (i, j); Calculate the sum of element products C of the overlapping part between the current element position (i, j) and the circular convolution kernel H(u, v) i,j , and the calculation formula is as follows: Store the convolution result of each channel in the corresponding accumulation matrix C.

6. The airport apron parking space safety supervision method according to claim 1, wherein, Step S6 specifically includes: If there are elements in the accumulation matrix C where the channel i > 1 and satisfy: C(x, y, i) > T1; Then determine that the engine is in a rotating state, where T1 is the preset threshold; C(x, y, i) is the element in the accumulation matrix C where the channel i > 1.

7. The airport apron parking space safety supervision method according to claim 1, wherein It also includes: S8. Real-time detect the number of people in the aircraft engine area. When the engine is in a rotating state and the detected number of people is greater than the preset number of people, trigger a real-time alarm.

8. An airport apron parking space safety supervision system, characterized in that, It includes: A data acquisition module for obtaining multi-stream video data collected in the aircraft engine area; An image caching module for decoding, performing grayscale conversion, and caching the video data of each stream in the multi-stream video data to generate a corresponding time-series grayscale image queue; A time-series array generation module for extracting all grayscale images from the time-series grayscale image queue every time T, and stacking them in chronological order to generate a three-dimensional time-series array P with dimensions of M×N×K; An image timing cycle analysis module is used to perform autocorrelation calculations on each pixel position in the three-dimensional timing array P to generate a normalized autocorrelation coefficient matrix; A convolution module is used to perform spatial convolution accumulation on the autocorrelation coefficient matrix to obtain an accumulation matrix; A determination module is used to determine that the engine is in a rotating state if there are elements in the accumulation matrix that are higher than a preset threshold; An output module is used to repeat the determination operation on the video data of each code stream. When the engine state corresponding to the video data of any code stream is in a rotating state, the engine state is finally output as a rotating state.

9. The airport apron stand safety supervision system according to claim 8, characterized in that It further includes: An aircraft maintenance personnel quantity detection module is used to detect the number of personnel in the aircraft engine area in real time. When the engine is in a rotating state and the detected number of personnel is greater than the preset number of personnel, a real-time alarm is triggered.

Images