A perimeter intrusion identification method based on multi-source information fusion

Abstract

The application discloses a kind of perimeter intrusion identification method based on multi-source information fusion, the method includes the following steps: step S1: build the data acquisition system based on multi-sensor;Step S2: through the data acquisition system of step S1 build, environment signal and intrusion target signal are collected, and the sample database of intrusion target and environmental noise is established by the mode of wavelet threshold denoising, image enhancement;Step S3: the sample database of step S2 is constructed, and the characteristic value of the environmental data and target data of sample database is extracted by multi-domain feature extraction algorithm;Step S4: different types of characteristic values of step S3 construction are fused by using fusion network, and finally sent into target recognition algorithm, whether the current is judged to be invaded or not.The method of the application combines image, sound and vibration sensor to construct target recognition technology, makes up the problem that system cannot work normally when image sensor is shielded and damaged, improves the fault tolerance of perimeter early warning system.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of perimeter security and the field of signal processing and recognition, and relates to a target recognition method based on multi-source information fusion. BACKGROUND

[0002] The current perimeter early warning system has a single intrusion prevention method, and the detection and recognition accuracy is greatly affected by the environment, so the reliability is low in actual application, and it is difficult to achieve the effect of early prevention.

[0003] Using a single sensor to detect whether an intrusion is not completely accurate, and using multi-sensor detection through multi-source information fusion can avoid the problem of inaccurate detection results. Multi-source information fusion is generally divided into data level fusion, feature level fusion and decision level fusion. The data level fusion is to fuse the original data before inputting the classifier, but this will inevitably lose part of the unique features of the intrusion target. Decision level fusion is to fuse the classification results, and decision level fusion has good real-time performance and fault tolerance, but the preprocessing cost is high. The feature level fusion method can effectively extract the independent features of the sensor, achieve good information compression, and also achieve high recognition accuracy. SUMMARY

[0004] In order to solve the problems of false alarm rate, inaccurate identification of intrusion objects and lack of intelligence in the existing perimeter intrusion detection system, the present application provides a perimeter intrusion recognition method based on multi-source information fusion. The method combines vibration sensors, sound sensors and image sensors to collect different data, uses the complementarity between them, adopts a multi-collaborative representation learning method, improves the feature fusion capability of multi-target at the feature level, and realizes effective detection and recognition of intrusion behavior.

[0005] The purpose of the present application is achieved by the following technical solutions:

[0006] A perimeter intrusion recognition method based on multi-source information fusion, comprising the following steps:

[0007] Step S1: building a data acquisition system based on multi-sensor, wherein:

[0008] The data acquisition system includes an array of vibration sensing units, an acoustic array sensing unit, an image sensing unit and a remote communication unit;

[0009] Step S2: collecting environment signals and intrusion target signals through the data acquisition system built in step S1, and establishing a sample database of intrusion targets and environmental noise through wavelet threshold denoising and image enhancement, the specific steps are as follows:

[0010] Step S21: Place the data acquisition system in a test site without moving targets around, and collect environmental data at different times and under different weather conditions;

[0011] Step S22: Collect personnel motion signals of different motion modes and distances, and collect vehicle motion signals of different speeds and distances;

[0012] Step S23: According to the environmental data collected in step S21 and the motion signals collected in step S22, establish an intrusion target motion data and environmental data sample database;

[0013] Step S24: For the one-dimensional vibration signal and sound signal in the sample database established in step S23, use the wavelet denoising method to realize signal preprocessing, and for the two-dimensional image signal, use the image enhancement method to realize signal preprocessing;

[0014] Step S3: For the sample database constructed in step S2, extract the characteristic values of the environmental data and target data of the sample database through a multi-domain feature extraction algorithm, and the specific steps are as follows:

[0015] Step S31: Extract the vibration signal characteristic value of the intrusion target

[0016] Step S311: By extracting a large number of pedestrian vibration signal and a variety of vehicle vibration signal characteristic values, analyze a variety of personnel and vehicle vibration signal data, extract the rhythm and amplitude characteristics of target motion, and establish a difference and correlation evaluation between characteristics;

[0017] Step S312: Use the linear model feature selection method based on L1 regularization to screen effective classification features for human and vehicle classification;

[0018] Step S313: Through the feature vector based on time domain, frequency domain and time-frequency domain characteristics, design a lightweight neural network, use neural network training input signal, take a certain hidden layer as a feature extraction layer, construct a vibration signal feature extraction model based on neural network, and use the model to extract effective characteristic values of the intrusion target vibration signal;

[0019] Step S32: Extract the sound signal characteristic value of the intrusion target

[0020] Step S321: Perform effective feature extraction on the sound signal, and extract information that can distinguish people, vehicles, and environment from it;

[0021] Step S322: Combine the lightweight neural network to train the effective features extracted in S321, and construct a sound signal feature extraction model based on neural network;

[0022] Step S33: Extract the image signal characteristic value of the intrusion target

[0023] Step S331: Input an image, transform its size and dimension, and then input it into a pre-trained extraction network to obtain feature maps of different sizes;

[0024] Step S332: Extract feature maps from Conv4_3, Conv7_2, Conv8_2, Conv9_2, Conv10_2, and Conv11_2 layers. Construct 6 edge boxes of different scales for each point on these feature maps and perform detection to generate multiple edge boxes that initially meet the conditions.

[0025] Step S333: Combine the bounding boxes obtained from different feature maps and use the non-maximum suppression method to obtain the target location result;

[0026] Step S4: The different types of feature values ​​constructed in step S3 are fused using a fusion network, and finally fed into the target recognition algorithm to determine whether an intrusion has occurred. The specific steps are as follows:

[0027] Step S41: Construct a fusion network Net using a deep neural network to convert the feature values ​​F of the vibration signal. 振 F 图 F 声 Input into the fusion network of the first round of alternating optimization:

[0028] [Net,T]=(σ;h1,h2...h n ;F 振 F 图 F 声 );

[0029] σ = {W, b};

[0030] In the formula, Net is the trained fusion network, T is the trained network parameters, σ is the neural network parameters, h is the number of hidden layers, n is the number of hidden layers, and F is the number of hidden layers. 振 F 图 F 声 These are the feature values ​​extracted from vibration signals, image signals, and sound signals, respectively. Vibration and sound feature values ​​are one-dimensional features, while image feature values ​​are two-dimensional features.

[0031] Step S42: Calculate the total error of the fusion network:

[0032] L i =L N1 +L N2 +L N3 ;

[0033] In the formula, L i Let L be the total error of the i-th round of alternating optimization. Ni(i = 1, 2, 3) are errors of the sub-networks of the fusion network; when L i <ε, the i-th round of alternating optimization is qualified, the training is terminated, the fusion network model Net, the network parameter T and the fusion feature F are saved, otherwise, the step S41 is returned;

[0034] Step S43: the fusion feature extracted by the fusion network in step S42 is sent into a target recognition algorithm to realize class classification of the intrusion target.

[0035] Compared with the prior art, the present application has the following advantages:

[0036] 1. The method of the present application combines image, sound and vibration sensors to construct a target recognition technology, which makes up for the problem that the system cannot work normally when the image sensor is damaged by shielding, and improves the fault tolerance of the perimeter warning system.

[0037] 2. The method of the present application realizes multi-modal target recognition based on multi-source information fusion, uses the complementarity between multi-modalities to eliminate the redundancy between modalities, and uses a multi-modal collaborative representation learning method to improve the fusion ability of multi-target at the feature level.

[0038] 3. The method of the present application can improve the poor weak signal detection ability of the existing system, improve the single detection means, and effectively solve various problems of the perimeter intrusion detection system by combining the multi-domain characteristics of the vibration signal and the existing deep learning algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 It is a whole flow chart of the perimeter intrusion recognition method based on multi-source information fusion;

[0040] Figure 2 It is a data acquisition system framework diagram;

[0041] Figure 3 It is a sample database establishment flow chart;

[0042] Figure 4 It is a vibration signal feature extraction schematic diagram;

[0043] Figure 5 It is an image signal feature extraction schematic diagram;

[0044] Figure 6 It is a feature level fusion collaborative representation schematic diagram;

[0045] Figure 7 It is a whole intrusion recognition schematic diagram;

[0046] Figure 8 It is a personnel intrusion detection recognition result;

[0047] Figure 9The vehicle intrusion detection recognition result. DETAILED DESCRIPTION

[0048] The technical solutions of the present application are further described below in conjunction with the drawings, but are not limited thereto, and any modification or equivalent replacement of the technical solutions of the present application without departing from the spirit and scope of the technical solutions of the present application shall be covered in the protection scope of the present application.

[0049] A perimeter intrusion recognition method based on multi-source information fusion, comprising the following steps:

[0050] Step S1: building a data acquisition system based on multiple sensors.

[0051] In this step, the basic architecture of the data acquisition system is as shown in Figure 2 , which includes a vibration perception unit array, a sound array perception unit, an image perception unit, and a remote communication unit. The specific building steps are as follows: the vibration perception unit array, the sound array perception unit, and the image perception unit are concentrated together, and the acquired environmental signals and intrusion target signals are transmitted to the remote communication unit, wherein:

[0052] The vibration perception unit array is composed of vibration sensors, and the vibration sensors adopt buried passive moving coil sensors;

[0053] The sound array perception unit is composed of sound sensors;

[0054] The image perception unit is composed of image sensors, and the image sensors adopt infrared cameras.

[0055] Step S2: collecting environmental signals and intrusion target signals through the data acquisition system built in step S1, and establishing a sample database of intrusion targets and environmental noise through wavelet threshold denoising and image enhancement.

[0056] In this step, the flow chart of establishing the sample database is as shown in Figure 3 , which specifically includes the following steps:

[0057] Step S21: arranging the data acquisition system in a test site without moving targets around, and collecting environmental data under different time and weather conditions;

[0058] Step S22: collecting personnel motion signals of different motion modes and different distances, and collecting vehicle motion signals of different speeds and different distances;

[0059] Step S23: establishing an intrusion target motion data and environmental data sample database according to the environmental data collected in step S21 and the motion signals collected in step S22;

[0060] Step S24: The one-dimensional vibration signal and sound signal in the sample database established in step S23 are preprocessed by using the wavelet denoising method, and the two-dimensional image signal is preprocessed by using the image enhancement method.

[0061] Step S3: The feature values of the environment data and target data of the sample database constructed in step S2 are extracted by using a multi-domain feature extraction algorithm.

[0062] In this step, the feature extraction technology uses the time domain, frequency domain and time-frequency domain features of the signal, and extracts the features of the input signal by combining a lightweight neural network, to complete the recognition of different types of ground moving targets. The specific steps are as follows:

[0063] Step S31: Extracting the vibration signal feature value of the intrusion target

[0064] As shown in the figure, the specific steps include the following steps: Figure 4

[0065] Step S311: By extracting the feature values of a large number of pedestrian vibration signals and a variety of vehicle vibration signals, analyzing a variety of personnel and vehicle vibration signal data, extracting the rhythm and amplitude of target motion and other features, and establishing the difference and correlation evaluation between the features;

[0066] Step S312: Using a linear model feature selection method based on L1 regularization, screening effective classification features for human-vehicle classification;

[0067] Step S313: By using a feature vector based on time domain, frequency domain and time-frequency domain features, a lightweight neural network is designed, the input signal is trained by using the neural network, a certain hidden layer is used as a feature extraction layer, a vibration signal feature extraction model based on a neural network is constructed, and the effective feature values of the intrusion target vibration signal are extracted by using the model;

[0068] Step S32: Extracting the sound signal feature value of the intrusion target

[0069] Step S321: Extracting the features of the sound signal, and extracting the information capable of distinguishing people, vehicles and environment from the sound signal.

[0070] The feature extraction method includes energy, amplitude and zero-crossing rate features based on time domain feature extraction, classical power spectrum features and parameter model power spectrum estimation features based on frequency domain feature extraction, and wavelet packet-based features, etc.

[0071] Since the wireless sensor network node has limited resources and cannot process very complex algorithms, simple time domain features such as short-time energy, short-time average amplitude and zero-crossing rate are selected.

[0072] ​Step S322: Combine the effective features selected in step S321 with the lightweight neural network training to construct a sound signal feature extraction model based on the neural network.

[0073] Step S33: Extracting the feature values of the intrusion target image signal

[0074] As shown in Figure 5 , it specifically includes the following steps:

[0075] Step S331: Input an image, convert its size latitude, adapt to the network input channel size, and then input it into the pre-trained extraction network to obtain different size feature maps, wherein:

[0076] The basic network structure of the extraction network adopts the VCG-16 network architecture, replaces the fully connected layer with a convolutional layer, and is used to extract low-scale feature maps; the auxiliary convolutional layer connects the last feature map of the basic network, and outputs 4 high-scale feature maps through the convolutional neural network; the auxiliary convolutional layer and the basic network feature map work together to complete the extraction of the target image feature information; the prediction convolutional layer predicts the rectangular frame information of each point of the feature map;

[0077] Step S332: Extract the feature maps of Conv4_3, Conv7_2, Conv8_2, Conv9_2, Conv10_2, and Conv11_2 layers, and construct 6 edge boxes of different scales on each point on these feature maps to generate multiple preliminary edge boxes that meet the conditions;

[0078] Step S333: Combine the edge boxes obtained from different feature maps, and obtain the target position result through the non-maximum suppression method.

[0079] Step S4: Use the fusion network to perform multi-feature fusion on the different types of feature values constructed in step S3, and finally input them into the target recognition algorithm to determine whether it is an intrusion.

[0080] In this step, the feature fusion technology uses a collaborative representation method to complete the feature-level fusion, constructs a fusion network model to realize the fusion of multi-source features, and inputs them into the target recognition algorithm to obtain the classification result and determine whether it is an intrusion. If it is an intrusion, it is a personnel intrusion or a vehicle intrusion. The schematic diagram is shown in Figure 6 and Figure 7 As shown in

[0081] [Net,T]=(σ;h1,h2...h n ;F振 , F 图 , F 声 )

[0082] σ={W, b}

[0083] Wherein, Net is the trained fusion network, T is the trained network parameter, σ is the neural network parameter, h is the hidden layer, n is the number of hidden layers, F 振 , F 图 , F 声 Respectively, the characteristic value of the vibration signal extraction, the characteristic value of the image signal extraction, the characteristic value of the sound signal extraction, wherein the vibration characteristic value, the sound characteristic value is one-dimensional feature, the image characteristic value is two-dimensional feature. The obtained characteristic value is input into the fusion network Net, and the alternating optimization is carried out, and finally the fusion feature is obtained. The specific steps are as follows:

[0084] Step S41: the characteristic value F 振 The vibration signal extraction is input into the first round alternating optimization N1 network, the generated fusion network parameter is initialized, sent into the fusion network layer (the number of hidden layers can be set according to actual situation) The parameters T1 of the fusion network and the fusion feature F1 extracted from the vibration signal are saved;

[0085] Step S42: taking the extracted fusion network parameter T1 as the initial value of N2 network parameter, the characteristic value F 图 The image signal extraction is input into the fusion network N2, and the parameters T2 of the fusion network and the fusion feature F2 extracted from the image signal are saved.

[0086] Step S43: taking the extracted fusion network parameter T2 as the initial value of N3 network parameter, the characteristic value F 声 The sound signal extraction is input into the fusion network N3, and the parameters T3 of the fusion network and the fusion feature F3 extracted from the sound signal are saved.

[0087] Step S44: the total error of the three sub-networks is calculated:

[0088] The error of the first round alternating optimization is:

[0089] L1=L N1 +L N2 +L N3

[0090] If the first round optimization error L1 is greater than the set error ε (set according to actual situation, generally set ε=0.05), it is indicated that the extracted fusion feature does not meet the requirements, and the fusion network is sent back for training until the optimization error is less than the set error value, then the training is terminated, and the fusion network model Net, the network parameter T and the fusion feature are saved, that is, the i round alternating optimization: Li

[0091] Step S45: After the feature value is fused through the fusion network, the fused feature is sent into the target recognition algorithm, and the improved SVM network recognition algorithm is used as the target recognition algorithm of the present application to realize the class classification of the human and vehicle targets.

[0092] Figure 8 And Figure 9 The feature fusion technology is used to realize the target recognition of personnel and vehicle intrusion. Figure 8 And Figure 9 It can be seen that when the red light corresponding to the person is on, it indicates that there is personnel intrusion; when the red light corresponding to the vehicle is on, it indicates that there is vehicle intrusion, thereby realizing the recognition of different intrusion targets.​

Claims

1.A perimeter intrusion identification method based on multi-source information fusion, characterized in that The method comprises the following steps: Step S1: building a multi-sensor-based data acquisition system, wherein: The data acquisition system comprises an array of vibration sensing units, an acoustic array sensing unit, an image sensing unit, and a remote communication unit; Step S2: collecting environmental signals and intrusion target signals through the data acquisition system built in step S1, and establishing a sample database of intrusion targets and environmental noise through wavelet threshold denoising and image enhancement; Step S3: extracting feature values of environmental data and target data of the sample database constructed in step S2 through a multi-domain feature extraction algorithm, the specific steps being as follows: Step S31: extracting vibration signal feature values of the intrusion target Step S311: analyzing a plurality of personnel and vehicle vibration signal data by extracting feature values of a large number of pedestrian vibration signals and a plurality of vehicle vibration signals, extracting rhythm and amplitude characteristics of target motion, and establishing difference and correlation evaluation between the characteristics; Step S312: screening effective classification features for human and vehicle classification by using a linear model feature selection method based on L1 regularization; Step S313: designing a lightweight neural network based on time domain, frequency domain, and time-frequency domain features, using a neural network to train input signals, taking a certain hidden layer as a feature extraction layer, constructing a vibration signal feature extraction model based on a neural network, and using the model to extract effective feature values of the intrusion target vibration signal; Step S32: extracting sound signal feature values of the intrusion target Step S321: performing effective feature extraction on the sound signal to extract information that can distinguish between people, vehicles, and the environment; Step S322: combining the effective features extracted in S321 with a lightweight neural network to construct a sound signal feature extraction model based on a neural network; Step S33: extracting image signal feature values of the intrusion target Step S331: inputting an image, performing size latitude conversion on the image, and then inputting the image into a pre-trained extraction network to obtain feature maps of different sizes; Step S332: extracting feature maps of Conv4_3, Conv7_2, Conv8_2, Conv9_2, Conv10_2, and Conv11_2 layers, constructing 6 edge boxes of different sizes on each point on the feature maps to detect, and generating a plurality of edge boxes that preliminarily meet the conditions; Step S333: combining the edge boxes obtained from different feature maps, and obtaining target position results through a non-maximum suppression method; Step S4: using a fusion network to fuse different types of feature values constructed in step S3, and finally inputting the feature values into a target recognition algorithm to determine whether there is an intrusion, the specific steps being as follows: Step S41: using a deep neural network to construct a fusion network Net, inputting the characteristic values F 振 , F 图 , F 声 of the vibration signal into the fusion network of the first round of alternating optimization: [Net, T] = (σ; h1, h2... h n ; F 振 , F 图 , F 声 ); σ={W, b}; In the formula, Net is a trained fusion network, T is a trained network parameter, sigma is a neural network parameter, h is a hidden layer, n is a number of hidden layers, F 振 , F 图 , F 声 are respectively a characteristic value extracted from a vibration signal, a characteristic value extracted from an image signal, and a characteristic value extracted from a sound signal, wherein the vibration characteristic value and the sound characteristic value are one-dimensional characteristics, and the image characteristic value is a two-dimensional characteristic. Step S42: calculating the total error of the fusion network: L i = L N1 +L N2 +L N3 ; In the formula, L i is the total error of the i-th round of alternating optimization, L Ni is the error of the sub-network of the fusion network, i = 1, 2, 3; when L i When < ε, the i-th round of alternating optimization is qualified, the training is terminated, the fusion network model Net, the network parameters T and the fusion feature F are saved, otherwise, return to step S41; Step S43: inputting the fusion features extracted by the fusion network in step S42 into the target recognition algorithm to realize classification of the intrusion target. 2.The perimeter intrusion identification method based on multi-source information fusion according to claim 1, characterized in that In step S1, the array of vibration sensing units is composed of vibration sensors, the acoustic array sensing unit is composed of acoustic sensors, and the image sensing unit is composed of image sensors. 3.The perimeter intrusion identification method based on multi-source information fusion according to claim 2, characterized in that The vibration sensor adopts a buried passive moving coil sensor, and the image sensor adopts an infrared camera. 4.The perimeter intrusion identification method based on multi-source information fusion according to claim 1, characterized in that The specific steps of the step S2 are as follows: Step S21: The data acquisition system is arranged in a test site without moving targets around, and environmental data in different time and different weather conditions are collected; Step S22: Personnel motion signals in different motion modes and different distances are collected, and vehicle motion signals in different speeds and different distances are collected; Step S23: According to the environmental data collected in the step S21 and the motion signals collected in the step S22, an invasion target motion data and environmental data sample database is established; Step S24: The one-dimensional vibration signals and sound signals in the sample database established in the step S23 are preprocessed by using a wavelet denoising method, and the two-dimensional image signals are preprocessed by using an image enhancement method. 5.The perimeter intrusion identification method based on multi-source information fusion according to claim 1, characterized in that The specific steps of the step S41 are as follows: Step S411: extracting the feature value F of the vibration signal 振 The generated fusion network parameters are initialized and input into the fusion network layer to store the parameters T1 of the fusion network and the extracted fusion features F1 of the vibration signal. Step S412: taking the extracted fusion network parameter T1 as the initial value of the N2 network parameter, and inputting the feature value F 图 of the image signal into the fusion network N2, saving the fusion network parameter T2 and the extracted fusion feature F2 of the image signal; Step S413: taking the extracted fusion network parameter T2 as the initial value of the N3 network parameter, and taking the feature value F 声 into the fusion network N3, and saving the fusion network parameter T3 and the extracted fusion feature F3 of the sound signal. 6.The perimeter intrusion identification method based on multi-source information fusion according to claim 1, characterized in that The target recognition algorithm is an improved SVM network recognition algorithm.

Images