A multi-dimensional perimeter security intrusion event identification method and identification system

Abstract

The application discloses a multi-dimensional perimeter security invasion event recognition method and recognition system, comprising: acquiring a phase-modulated interference signal by using a double Mach-Zehnder interferometer to construct a one-dimensional time sequence signal of an invasion event; converting the one-dimensional time sequence signal into a GPS signal and a two-dimensional time-frequency signal; a UAV shoots a video according to the GPS signal to form a three-dimensional video signal; an industrial computer performs synchronous recognition of the invasion event on the two-dimensional time-frequency signal and the three-dimensional video signal in a multi-dimensional feature extraction deep learning network which has been trained, multiplies the confidence of each type of event obtained and normalizes the confidence, and outputs an event with the maximum confidence as a final recognized event. The application not only extracts features of an invasion event by using an optical signal, but also extracts features by using a video signal shot by a UAV, multi-dimensionally describes the invasion event, widens the number of detectable invasion events, increases the pattern recognition accuracy, and increases the reliability of the system.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of sensing and intelligent detection, and particularly relates to a distributed optical fiber vibration sensing signal and a multi-dimensional signal recognition method and system for a UAV system platform. BACKGROUND

[0002] The distributed optical fiber vibration sensing system uses optical fibers to position and sense external vibration events, and can achieve accurate positioning within a long distance range. Compared with traditional perimeter security technology, the distributed optical fiber vibration sensing has a larger coverage area, better anti-electromagnetic interference capability, and higher sensitivity, and its cost per kilometer in engineering application is lower, and it can achieve complete unmanned monitoring in a large range of key areas. In the field of perimeter security, the behavior pattern of intrusion will directly determine the processing method of the monitoring system. At the same time, accurate pattern classification and recognition can reduce the false alarm rate of the distributed optical fiber vibration sensing system, and can accurately intervene in the intrusion event under adverse environmental conditions. Existing pattern classification and recognition research focuses on single optical signal classification and one-dimensional time sequence signal discrimination, and the implemented pattern classification types are less, and the intrusion process cannot be directly recorded. However, the best record of the intrusion behavior is to store and identify the intrusion process in the form of video, and to continuously track the subsequent intrusion behavior of the intruder. The traditional monitoring mode uses fixed cameras to monitor the defense area, and sets a fixed camera every certain distance to ensure coverage of the entire defense area. This type of method is suitable for small-scale perimeter security in small areas such as research institutes, prisons, and mental hospitals, and can achieve all-around real-time monitoring. However, for large-scale and long-distance defense areas, such as railway perimeter security, border perimeter security, and large infrastructure perimeter security, the number of camera installations will be extremely large, and the maintenance and use costs will also increase.

[0003] As a rapidly developing industry in recent years, the UAV has many advantages such as small system, high flexibility, rapid response, real-time tracking, and has a good application prospect in many fields such as military, agriculture, and industrial pipeline inspection. It has become possible for the UAV to replace manual patrol and intervene in hazardous events. The existing UAV is still in the initial stage of application in the field of perimeter security, and the UAV system alone can only patrol in a fixed area, and the intrusion behavior at a specific position still needs to be manually operated to record and identify abnormal events, and fundamentally, it still cannot realize true unmanned operation. In addition, the existing UAV can only realize positioning and tracking of intruders, and cannot identify the behavior of the intrusion event, thus limiting the application of the UAV in the field of perimeter security.

[0004] CN208459615A discloses a perimeter security system based on a distributed optical fiber sensor, using sensing optical fiber as sensing medium, connecting each sensing module with a monitoring center, but the security system is suitable for short distance security monitoring and cannot overcome the influence of positioning error on system performance caused by long distance.

[0005] CN109326070A discloses a perimeter security system and a monitoring method thereof, which provides a passive monitoring system by utilizing the conversion of stress and vibration effect and the detection of vibration sensing of optical fiber. However, the system cannot accurately judge the intrusion behavior.

[0006] Although a series of measures such as polarization control and improved positioning algorithm can be taken to improve the positioning accuracy of the system, the problem caused by positioning error in long distance monitoring scene cannot be perfectly solved, and the distributed optical fiber disturbance sensing system can only locate the intrusion point and identify the mode of the intrusion behavior, and cannot realize real-time and all-around video monitoring of the area around the intrusion point.

[0007] In summary, in the field of perimeter security, the advantages of high accuracy and wide coverage of the distributed optical fiber vibration sensing system are combined with the advantages of fast maneuvering and video recording of the unmanned aerial vehicle, the optical signal is combined with the video signal of the unmanned aerial vehicle, a multi-dimensional intelligent full unmanned monitoring system is realized, and the stability, reliability and fast response of the unmanned perimeter security system are improved, which has urgent practical significance and important value. SUMMARY

[0008] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a multi-dimensional perimeter security intrusion event identification method and system to solve the problem of being unable to quickly and accurately identify intrusion events in long distance and large range perimeter security field. At the same time, the optical signal of the distributed optical fiber vibration sensing system and the video signal of the unmanned aerial vehicle are used for intrusion behavior mode identification, the short-time Fourier transform is used for frequency domain analysis of the optical signal, the one-dimensional signal is converted into a two-dimensional signal, the optical two-dimensional signal and the video three-dimensional signal are combined, the event with the maximum confidence is selected as the final intrusion event, so as to improve the identification accuracy of the intrusion mode and reduce the system response time.

[0009] A multi-dimensional perimeter security intrusion event identification method, which utilizes a double Mach-Zehnder optical fiber interferometer, a data acquisition card, an unmanned aerial vehicle and an industrial computer, specifically comprising:

[0010] Step 1: Real-time detection of optical signal, when an abnormal intrusion event occurs, a double Mach-Zehnder interferometer is used to obtain a phase-modulated interference signal, a data acquisition card is used to collect data and send it to an industrial computer, then the two-way interference signal is correlated and demodulated to obtain the disturbance position of the intrusion event, and a one-dimensional time sequence signal of the intrusion event is constructed;

[0011] Step two: the industrial computer converts the disturbance position obtained in step one into a GPS signal, and instructs the unmanned aerial vehicle to automatically go to the position coordinates of the intrusion site, and after arriving, video shooting of the intrusion event is performed, and the video signal of the intrusion behavior is transmitted back to the industrial computer in real time, and the video signal shot is taken as the three-dimensional video signal of the intrusion event;

[0012] Step three: the industrial computer synchronously identifies the intrusion event by using the one-dimensional time sequence signal and the three-dimensional video signal obtained in the multi-dimensional feature extraction deep learning network which has been trained; including: performing median filtering, intrusion position endpoint detection and short-time Fourier transform on the one-dimensional time sequence signal obtained in step one to obtain a two-dimensional time-frequency signal, and then sending the transformed two-dimensional time-frequency signal into a pre-trained resnet152 network for identification to extract events with preset confidence degrees in the front n; at the same time, the three-dimensional video signal shot by the unmanned aerial vehicle obtained in step two is processed by using a slowfast algorithm for video recognition, and a 3Dresnet50 network is used for feature extraction and identification, and events with confidence degrees in the front n are also extracted.

[0013] The confidence degrees of various events identified from the two-dimensional time-frequency signal and the confidence degrees of various events identified from the three-dimensional video signal are multiplied and normalized, and the event with the maximum confidence degree is output as the final identified event, and the intrusion event type, intrusion event picture, intrusion event position and intrusion event time are output.

[0014] In the prior art, because the vibration signals generated by shaking, kicking and smashing in the intrusion event are similar, the two-dimensional time-frequency signal after phase demodulation in the double Mach-Zehnder fiber interferometer cannot be accurately identified by using a convolutional neural network; and because the climbing, shaking and kicking actions in the intrusion event are similar, the convolutional neural network cannot accurately distinguish the climbing, shaking and kicking in the three-dimensional video signal. Therefore, the two-dimensional time-frequency signal and the three-dimensional video signal are identified respectively in the present application, and the event confidence degrees output by the two are multiplied and normalized, so as to comprehensively consider the two and output the intrusion event with the maximum confidence degree as the final identified event.

[0015] Further, after the unmanned aerial vehicle arrives at the position coordinates in step two, the real-time position of the intruder is located by running a yolo algorithm on the on-board computer of the unmanned aerial vehicle while collecting the video signal, so as to realize the following shooting of the intruder.

[0016] Further, the pre-set intrusion events of the multi-dimensional feature extraction deep learning network include: no intrusion, climbing, smashing, kicking, cutting, shaking, heavy knocking, light knocking and pulling.

[0017] Further, the step three "performing median filtering and short-time Fourier transform on the one-dimensional time sequence signal" specifically includes:

[0018] The median filter slides from the left side of the original signal to the right side with a length of 2N+1, and calculates the median output at each step of the sliding process; assuming that the length of the median filter input original signal is L, the output value at i is x(i), 1≤i≤L. If y(i) is the corresponding output of the median filter at i, then y(i) can be expressed as:

[0019]

[0020] Where j≠i and 1≤j≤L; N is a positive integer; after the end points are truncated, the truncated data is subjected to short-time Fourier transform:

[0021]

[0022] Where τ is the time shift, f is the frequency, and ω(t-τ) is the Hamming window function.

[0023] A multi-dimensional perimeter security intrusion event recognition system, comprising a dual Mach-Zehnder optical fiber interferometer, a data acquisition card, a UAV and an industrial computer;

[0024] The dual Mach-Zehnder optical fiber interferometer is used to extract the optical signal corresponding to the phase-modulated intrusion event;

[0025] The data acquisition card is used to collect data of the dual Mach-Zehnder optical fiber interferometer and send it to the industrial computer;

[0026] The UAV goes to the intrusion site to take pictures according to the GPS signal sent by the industrial computer;

[0027] The industrial computer receives data from the data acquisition card, and performs cross-correlation operation on the two interference signals to demodulate the disturbance position of the intrusion event, as a one-dimensional time sequence signal of the intrusion event. The industrial computer performs two operations on the one-dimensional time sequence signal: one is to convert the disturbance position into a GPS signal and send it to the UAV; the other is to pre-process the signal and convert it into a two-dimensional time-frequency signal; the two-dimensional time-frequency signal and the three-dimensional video signal taken by the UAV in real time are respectively subjected to intrusion event recognition in the multi-dimensional feature extraction deep learning network which has been trained. The optical signal is first recognized, and the recognition event confidence is generated; the video signal is recognized later, and the recognition result is directly multiplied by the confidence generated by the optical signal recognition, and finally the intrusion event type, intrusion event picture, intrusion event position and occurrence time are output.

[0028] Further, the one-dimensional time sequence signal and the three-dimensional video signal are input into the trained multi-dimensional feature extraction deep learning network for synchronous identification of the intrusion event, specifically including: performing median filtering, intrusion position endpoint detection and Fourier transform on the one-dimensional time sequence signal to obtain a two-dimensional time-frequency signal; inputting the two-dimensional time-frequency signal into a resnet152 convolutional neural network for identification; inputting the three-dimensional video signal into a video behavior analysis model based on a slowfast algorithm for identification, and using a 3Dresnet50 network for feature extraction.

[0029] Compared with the prior art, the technical scheme of the present application has the following beneficial effects:

[0030] Unlike the pattern recognition method of the traditional distributed optical fiber vibration sensing system, the present application not only uses optical signals to extract features of intrusion events, but also uses video signals captured by unmanned aerial vehicles to extract features, thus describing intrusion events in multiple dimensions. The present application breaks through the limitations of traditional recognition methods on the types of intrusion events, widens the number of detectable intrusion events, increases the accuracy of pattern recognition, further reduces the false alarm rate of the system, and increases the reliability of the system.

[0031] Moreover, in the case of inaccurate disturbance position extraction by the dual Mach-Zehnder optical fiber interferometer, no additional optical devices or extensive calculations are required, but a pre-learned convolutional neural network is used in combination with the distributed optical fiber laying route to identify the nine common intrusion events in the perimeter security field, and finally display the type, picture, occurrence position and occurrence time of the identified intrusion event, and remind relevant personnel to intervene in time to ensure perimeter safety. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 A flowchart showing the multi-dimensional perimeter security intrusion event recognition method of the present application is shown.

[0033] Figure 2 An extraction schematic diagram of the positioning information of the dual Mach-Zehnder optical fiber interferometer in step one of the present application is shown.

[0034] Figure 3 A flowchart of the multi-dimensional feature extraction deep learning network structure of step three of the present application is shown.

[0035] Figure 4 A structure diagram of the multi-dimensional feature extraction deep learning network structure of step three of the present application is shown.

[0036] In the figure:

[0037] 1: light source 2: isolator 3: first optical fiber coupler

[0038] 4: first circulator 5: second circulator 6: second optical fiber coupler

[0039] 7: third optical fiber coupler 8: first photodetector 9: second photodetector

[0040] 10: data acquisition card 11: industrial computer DETAILED DESCRIPTION

[0041] The technical solutions of the present application will be described in further detail below in combination with the drawings and specific embodiments, and the specific embodiments described are only used to explain and illustrate the present application and do not limit the present application.

[0042] The multi-dimensional perimeter security intrusion event recognition method described in the present application combines an intrusion behavior position acquisition algorithm, a distributed vibration optical fiber sensing system, communication software design for a unmanned aerial vehicle system, an optical signal and video signal recognition processing algorithm, and a distributed optical fiber laying route to construct a multi-dimensional intrusion event recognition model. The model is used to recognize nine common intrusion events (no intrusion, climbing, smashing, kicking, cutting, shaking, heavy knocking, light knocking, and pulling) in the perimeter security field, and simultaneously displays and records the intrusion event type, intrusion event picture, intrusion event position, and intrusion event time on the industrial computer. According to the different degrees of harm of the intrusion events, relevant personnel are reminded to intervene in time to ensure perimeter safety.

[0043] As shown in Figure 1 A multi-dimensional perimeter security intrusion event recognition method, which utilizes a dual Mach-Zehnder optical fiber interferometer, a data acquisition card, a unmanned aerial vehicle, and an industrial computer.

[0044] Step one: real-time detection of optical signals, when an abnormal intrusion event occurs on the sensing optical fiber, the vibration caused will cause changes in the refractive index and other parameters at the corresponding sensing optical fiber, thereby causing corresponding changes in the phase of the sensing signal. The sensing event signal is processed and analyzed by detecting the amplitude signal transformation caused by the phase change. In the dual Mach-Zehnder interferometer, there are two routes of interference light in clockwise and counterclockwise directions. Due to the different relative positions of the vibration in different directions, the time of the phase-modulated optical signal reaching the two end detectors is also different; thus, the dual Mach-Zehnder interferometer is used to obtain the phase-modulated optical signal, which is used to realize abnormal event positioning and subsequent optical signal feature extraction; after the data acquisition card collects data and sends it to the industrial computer, the mutual correlation operation is performed on the two interference signals to demodulate the occurrence position of the intrusion event. Different intrusion events have different phase modulations on the optical fiber, and the corresponding interference light signals are also different. Therefore, the intrusion event-modulated optical signal is extracted to construct a one-dimensional time sequence signal of the intrusion event. The optical signal then completes two functions in the industrial computer, one is to convert the disturbance position into a GPS signal to let the unmanned aerial vehicle go to shoot, and the other is to preprocess the optical signal and then enter the Resnet152 network.

[0045] Figure 2 The schematic diagram of the dual Mach-Zehnder fiber interferometer positioning information extraction is shown, including a light source 1, an isolator 2, a first fiber coupler 3, a first circulator 4, a second circulator 5, a dual Mach-Zehnder interferometer, a data acquisition card 10 and an industrial computer 11. The light source 1 uses a narrow-band continuous light laser with a wavelength of 1550 nm and a maximum output power of 5 mW as the light source of the system. The data acquisition card 10 is used to collect the sensing electrical signals transmitted back by the photodetector, and the industrial computer 11 is used to analyze and demodulate the received signals, thereby demodulating the amplitude-frequency information of the vibration signal applied to the sensing optical fiber. The main implementation process includes:

[0046] The laser emitted by the light source 1 is transmitted to the first fiber coupler 3 through the isolator 2, and the first fiber coupler 3 divides the signal light into two beams, which are transmitted to the first circulator 4 and the second circulator 5, respectively. Then the light enters the dual Mach-Zehnder interferometer for interference. According to the sensing principle of the dual Mach-Zehnder interferometer, the light emitted by the same light source propagates in two opposite directions. The light propagating in the clockwise direction first passes through the first circulator 4, then is split by the second fiber coupler 6, passes through the sensing optical fiber, and then interferes at the third fiber coupler 7. The interfered signal light is received by the first photodetector 8 through the second circulator 5, and then enters the industrial computer 11 through the data acquisition card 10. The light propagating in the counterclockwise direction first passes through the second circulator 5, then is split by the third fiber coupler 7, passes through the sensing optical fiber, and then interferes at the second fiber coupler 6. The interfered signal light is received by the second photodetector 9 through the second circulator 5, and then the optical signal is sent to the industrial computer 11 through the data acquisition card 10.

[0047] The industrial computer 11 removes the direct current component from the two-way interference signal to obtain the interference intensity I CW and I CCW which are respectively represented by trigonometric functions as follows:

[0048]

[0049] A0 represents the amplitude value of the continuous laser signal, φ(t) represents the phase change information introduced by the external vibration event, τ1 and τ2 respectively represent the time delay of the signal reaching the first photodetector 8 and the second photodetector 9, and the difference τ between the two can be obtained according to the cross-correlation operation, and then the corresponding disturbance position x is obtained:

[0050]

[0051] where n, c and l respectively represent the effective refractive index of the optical fiber, the propagation constant of light in vacuum and the length of the optical fiber link.

[0052] Because different external intrusion events, the phase modulation φ(t) generated is also different, so the optical signal received by the industrial computer 11 also has different characteristics.

[0053] Step two: obtaining the video signal of the intrusion behavior

[0054] According to the disturbance position x obtained in step one, the industrial computer 11 converts the disturbance position into a GPS signal containing the position coordinates and transmits it to the UAV ground station. After the UAV ground station receives the intrusion signal, it transmits the intrusion position to the UAV, and the UAV ground station controls the UAV to automatically go to the position coordinates of the intrusion site. The UAV switches to the mission mode and goes to the intrusion position. When the ground station detects that the UAV has arrived at the intrusion site, it automatically switches the UAV mode to the offboard mode, i.e. the on-board computer calculation mode, and the tracking function is started. After the UAV arrives at the intrusion site, it monitors the intrusion event with video. At the same time of collecting the video signal, the on-board computer of the UAV runs the yolo algorithm to locate the real-time position of the intruder, realizing the follow-up shooting of the intruder. Since the UAV is in normal cruising mode in the air, it can quickly go to the intrusion site, obtain the video signal of the intrusion behavior, and transmit the video signal of the intrusion behavior back to the industrial computer 11 in real time, taking the video signal as the three-dimensional video signal of the intrusion event.

[0055] Step three: the industrial computer 11 sends the obtained one-dimensional time sequence signal and third-dimensional video signal (i.e. optical signal and video signal) into the multi-dimensional feature extraction deep learning network structure which has been trained. As shown in Figure 4 the multi-dimensional feature extraction deep learning network structure includes Resnet152 network, Slowfast algorithm and 3Dresnet50 network.

[0056] S301: training of convolutional neural network

[0057] Pre-select 9 kinds of intrusion events (no intrusion, climbing, smashing, kicking, cutting, shaking, heavy knocking, light knocking, pulling) each with 1500 groups of optical signals, and 200 groups of 20s video clips for each type of video signal. The training set, validation set and test set are divided according to the ratio of 8:1:1 for training; among them, the optical signal is trained by Resnet152 network, and the three-dimensional video signal is trained by Slowfast algorithm model. The Slowfast algorithm model is a dual-path SlowFast model for video recognition, including a slow channel running at low frame rate and slow refresh speed, and a fast channel for capturing fast changing motion. After the features of the two channels are fused in time scale and space scale, 3Dresnet50 network is used for training to obtain the training weight of three-dimensional video stream.

[0058] The video signal needs to be converted into a data set in the ava format before training, and the training set information and the verification set information of the video signal are respectively made. The video signal mode is labeled using the Via tool. In the embodiment of the application, the parameters of the model (including the initial learning rate, the batch size, the loss function constraint term, the number of residual dense modules, and the like) are adjusted according to the type and characteristics of the intrusion mode to train the convolutional neural network model. The input image is flipped and rotated to realize data expansion during the training process. Specifically, in this embodiment, the minimum batch sample number is 16, the learning rate is initialized to 0.0001, the exponential decay learning rate method is used, the decay rate is set to 0.5, the training period is 100, the learning rate is decayed once every 20 rounds, the cross-entropy loss function is selected as the loss function, the Adam algorithm is used to optimize the loss function, and the number of residual dense modules is 50 and 152 respectively. In the test process, the test result can reach an accuracy of 99.6%, and the model has high recognition accuracy and classification recognition stability.

[0059] The trained Resnet152 network, Slowfast algorithm and 3Dresnet50 network model are saved as a multi-dimensional feature extraction deep learning network, and the one-dimensional time sequence signal and the third-dimensional video signal obtained in steps one and two are respectively trained and recognized together in the multi-dimensional feature extraction deep learning network structure.

[0060] S302: processing the optical signal

[0061] As shown in Figure 3 The one-dimensional time sequence signal obtained in step one is affected by light source noise, circuit noise and background environmental noise, and contains a large amount of noise redundant information, and does not contain frequency spectrum information. Therefore, the optical signal needs to be preprocessed. The optical signal is first subjected to median filtering to effectively reduce signal noise. The median filter slides from the left side of the original signal to the right side with a length of 2N+1 (N is a positive integer), and calculates the median output in each step of the sliding process. Assuming that the length of the median filter input original signal is L, the output value at i is x(i), 1≤i≤L. If y(i) is the corresponding output of the median filter at i, then y(i) can be expressed as:

[0062]

[0063] In the formula, j≠i and 1≤j≤L. In the filtering process, N=200 is selected. After the endpoints are intercepted, data with a length of 6.67k after the endpoints are selected. In this embodiment, the optical signal is subjected to intrusion position endpoint detection, the optical signal at the time of intrusion is intercepted, the data amount is reduced, and the operation speed is accelerated. The intercepted data is subjected to short-time Fourier transform:

[0064]

[0065] wherein, tau is time shift, f is frequency, omega(t-tau) is Hamming window function. The transformed two-dimensional time-frequency signal is sent into a pre-trained resnet152 network for recognition, and the top 5 events in confidence are extracted.

[0066] S303: processing video signal

[0067] The three-dimensional video signal obtained by the unmanned aerial vehicle in step two is processed by a pre-trained slowfast algorithm for video recognition, a 3Dresnet50 network is used for feature extraction and recognition, and the top 5 events in confidence are also extracted.

[0068] S304: multiplying the confidence obtained by recognizing the two-dimensional time-frequency signal in S302 and the confidence obtained by recognizing the three-dimensional video signal in S303, and normalizing, outputting the event with the maximum confidence as the final recognized event, that is, determining the intrusion event to which the current disturbance belongs.

[0069] Finally, the type of intrusion event, the picture of the intrusion event, the location of the intrusion event and the time of the intrusion event are displayed and recorded on the industrial computer. For the nine kinds of intrusion events set in advance, i.e. no intrusion, climbing, smashing, kicking, cutting, shaking, heavy knocking, light knocking and pulling, according to the different harm degrees of the intrusion events, the relevant personnel are reminded to intervene in time to ensure the safety of the perimeter.

[0070] Although the preferred embodiments of the present application are described above in combination with the drawings, the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which all belong to the protection scope of the present application.

Claims

1. A multi-dimensional perimeter security intrusion event identification method, the identification method utilizes a dual Mach-Zehnder fiber interferometer, a data acquisition card (10), a UAV and an industrial computer (11), characterized in that, Specifically comprising: Step one: real-time detection of optical signals, when an abnormal intrusion event occurs, a double Mach-Zehnder interferometer is used to obtain a phase-modulated interference signal, and after the data collected by the data acquisition card is sent to the industrial computer (11), the two-way interference signal is correlated and demodulated to obtain the disturbance position of the intrusion event, and a one-dimensional time sequence signal of the intrusion event is constructed; Step two: the industrial computer (11) converts the disturbance position obtained in step one into a GPS signal, and instructs the unmanned aerial vehicle to automatically go to the position coordinates of the intrusion site, and after arriving, the video of the intrusion event is shot, and the video signal of the intrusion behavior is transmitted back to the industrial computer (11) in real time, and the video signal shot is taken as a three-dimensional video signal of the intrusion event; Step three: the industrial computer obtains the one-dimensional time sequence signal and the three-dimensional video signal in the multi-dimensional feature extraction deep learning network which has been trained to identify the intrusion event; including: performing median filtering, intrusion position endpoint detection, and short-time Fourier transform on the one-dimensional time sequence signal obtained in step one to obtain a two-dimensional time-frequency signal, and then sending the transformed two-dimensional time-frequency signal to the pre-trained resnet152 network for identification, and extracting the preset top-n events according to confidence; At the same time, the three-dimensional video signal shot by the unmanned aerial vehicle obtained in step two is processed by a slowfast algorithm for video recognition, and a 3Dresnet50 network is used for feature extraction and identification, and the top-n events according to confidence are also extracted. Multiply and normalize the confidence of each type of event obtained by the two-dimensional time-frequency signal recognition and the confidence of each type of event obtained by the three-dimensional video signal recognition, and output the event with the maximum confidence as the final recognized event, and output the intrusion event type, intrusion event picture, intrusion event position and occurrence time.

2. The method of claim 1, wherein the method further comprises: In step two, after the unmanned aerial vehicle arrives at the position coordinates, the onboard computer of the unmanned aerial vehicle runs the yolo algorithm to locate the real-time position of the intruder while collecting the video signal, realizing the follow-up shooting of the intruder.

3. The method of claim 1, wherein the method further comprises: The pre-set intrusion events of the multi-dimensional feature extraction deep learning network include: no intrusion, climbing, smashing, kicking, cutting, shaking, heavy knocking, light knocking, and pulling.

4. The method of claim 1, wherein the method further comprises: In step three, "performing median filtering and short-time Fourier transform on the one-dimensional time sequence signal" specifically includes: The median filter slides from the left side of the original signal to the right side with a length of 2N+1, and calculates the median output in each step of the sliding process; Assuming that the length of the median filter input original signal is L, the output value at i is x(i), 1≤i≤L; If y(i) is the corresponding output of the median filter at i, then y(i) can be expressed as: Where j≠i and 1≤j≤L; N is a positive integer; after intercepting the endpoints, short-time Fourier transform is performed on the intercepted data: Where τ is the time shift, f is the frequency, and ω(t-τ) is the Hamming window function.

5. A multi-dimensional perimeter security intrusion event identification system, comprising a double Mach-Zehnder fiber optic interferometer, a data acquisition card, an unmanned aerial vehicle, and an industrial computer; The double Mach-Zehnder fiber optic interferometer is used to extract the phase-modulated optical signal corresponding to the intrusion event; The data acquisition card is used for collecting data of the double Mach-Zehnder fiber interferometer and sending the data into the industrial computer; The unmanned aerial vehicle goes to the intrusion site to take pictures according to the GPS signal sent by the industrial computer; The industrial computer receives data from the data acquisition card, performs cross-correlation operation on two-way interference signals, demodulates the disturbance position of the intrusion event, and takes the one-dimensional time sequence signal as the intrusion event; the industrial computer performs two operations on the one-dimensional time sequence signal: one is to convert the disturbance position into a GPS signal and send it to the unmanned aerial vehicle; the second is to pre-process the signal to convert it into a two-dimensional time-frequency signal; the two-dimensional time-frequency signal and the three-dimensional video signal taken by the unmanned aerial vehicle in real time are respectively subjected to intrusion event recognition in the multi-dimensional feature extraction deep learning network which has been trained; the optical signal is first recognized and the recognition event confidence is generated; the video signal is recognized later, and the recognition result is directly multiplied by the confidence generated by the optical signal recognition, and finally the intrusion event type, intrusion event picture, intrusion event position and occurrence time are output.

6. The multi-dimensional perimeter security intrusion event recognition system of claim 5, wherein, The "pre-processing the signal to convert it into a two-dimensional time-frequency signal; the two-dimensional time-frequency signal and the three-dimensional video signal taken by the unmanned aerial vehicle in real time are respectively subjected to intrusion event recognition in the multi-dimensional feature extraction deep learning network which has been trained" specifically includes: The one-dimensional time sequence signal is subjected to median filtering, intrusion position endpoint detection and Fourier transform to obtain a two-dimensional time-frequency signal; the two-dimensional time-frequency signal is sent to a resnet152 convolutional neural network for recognition; the three-dimensional video signal is sent to a video behavior analysis model based on the slowfast algorithm for recognition, and a 3Dresnet50 network is used for feature extraction.

Images