Infrared target detection model flash invisibility method based on recurrent neural network

Abstract

The present application discloses a stroboscopic stealth method of an infrared target detection model based on a recurrent neural network, which comprises the following steps: 1) generating 2n groups of data required for training the model; 2) pre-training an LSTM prediction model; and 3) combining the LSTM prediction model with an infrared target detection model to form an integrated model and performing main training on the integrated model. The stroboscopic stealth method of the infrared target detection model has small environmental influence, is suitable for various environments, and can realize self-adjustment and feedback.

Description

TECHNICAL FIELD

[0001] The present application relates to an infrared target detection model stroboscopic stealth method, which adds a recurrent neural network to the pre-training part of the model and the feedback training part of the model to the current infrared target detection model stroboscopic stealth method, and obtains an infrared target detection model stroboscopic stealth method and a specific device. BACKGROUND

[0002] Infrared rays are a kind of electromagnetic waves, which have similar properties to radio waves and visible light. Infrared radiation is the most widespread electromagnetic radiation in nature, which is based on the irregular motion of molecules and atoms of any object in a conventional environment, and continuously radiates thermal infrared energy. The most commonly used infrared detection method is passive infrared detection. Passive infrared detection is mainly used for hot spot detection of industrial equipment and components, military infrared night vision devices, infrared sighting scopes, and human body temperature measurement in places with dense human flow such as airports and stations.

[0003] The existing model for evading infrared detection has a small application range, and the physical attack effect does not have good transferability between different samples. A research team has proposed a physical attack method based on a small bulb, which can make a person holding a small bulb plate successfully evade the detection of an infrared detector, reducing the confidence of the YOLOv3 detector by 64.12%. However, the generated adversarial samples by this method are single-acting and do not have wide target feasibility, and cannot well achieve the stealth effect of the infrared target detection model.

[0004] The existing technical solution makes the stealth effect of the sample related to the surrounding environment, which is greatly affected by the environment and does not have self-adjustment and feedback. The existing technology mainly uses camouflage to keep the temperature inside the object consistent with the environment temperature, so as to evade infrared detection. The above methods are designed to destroy a certain link in the detection according to the principle of infrared target detection, to achieve the stealth effect of the infrared detection model. However, the success rate of the method itself is not high, and there is still a large error. The model has few training samples before being put into use, and the sample stealth effect is not ideal, and the confidence of the infrared detection model is still in a high range.

[0005] Physical attack on the infrared target detection model can achieve the effect of stealth. The "stroboscopic effect" can be used to achieve physical attack on the infrared target detection model. The "stroboscopic effect" is caused by the flicker of the light source, and the flicker is an unstable visual phenomenon caused by the fluctuation of light intensity or spectral distribution over time.

[0006] By controlling the electric pulse, the LED lamp can generate light of different frequencies and brightness, and the sampling frequency of the camera is higher than the flicker frequency of the LED lamp, so that the strobe phenomenon of the LED lamp can be captured, and different width and brightness of the strobe light band will be formed on the camera imaging photo. When the light band of a specific width and brightness is added to the target object as a mask, the attack effect formed after being received by the camera will make the YOLO model fail to detect the target. SUMMARY

[0007] The present application overcomes the above-mentioned shortcomings of the prior art and provides a frequency flash invisibility method for an infrared target detection model based on a recurrent neural network.

[0008] In order to realize a more efficient and feasible infrared target detection model frequency flash invisibility effect with a wider application range, the present application designs a pre-training process of the model in combination with the characteristics of the recurrent neural network under the premise of having a sufficient amount of sample data, and introduces a feedback variable in the main training process of the model, which is used as a target function to modify the parameters in the model, so as to obtain a frequency flash invisibility device that can avoid detection by the infrared target detection model.

[0009] Based on the understanding of the YOLO model, the present application constructs a target detection model in an infrared imaging mode, and establishes an integrated model and performs training in combination with the prediction function of the LSTM model, so as to realize the frequency flash invisibility of the infrared target detection model based on the recurrent neural network.

[0010] The specific technical scheme of the present application is as follows: the frequency flash invisibility method for an infrared target detection model based on a recurrent neural network comprises the following steps:

[0011] 1) As shown in Figure 1 , 2n groups of data required for training the model are generated, and the specific steps are as follows:

[0012] 1.1) input n groups of samples without light bands as input data into the infrared target detection model, the infrared target detection model is established on the basis of the YOLOv3 target detection model and has a 53-layer convolutional network, and the logical regression is used to predict the score of each bounding box, the model outputs the position (x, y, w, h) of the detected target frame and the confidence Pr(object) of the frame in the case of successful detection, and n groups of output data are obtained. Wherein, x and y refer to the center coordinates of the frame, and w and h are the length and width of the bounding box. When the frame does not contain the target, Pr(object) = 0; when the frame contains the target, Pr(object) = 1.

[0013] 1.2) By setting the frequency, power of the electric signal, get different width, brightness of the stroboscopic light belt, and add it to the sample as a mask. Combine n groups of different frequencies and powers, and add the corresponding stroboscopic light belt with different brightness and width as a mask to the sample. The resulting sample is used as input data for the infrared target detection model, and the corresponding output result is a 6-dimensional matrix array including the position (coordinates + length and width) of the frame and the confidence. Record the n groups of output data obtained by output.

[0014] 2) According to the pre-training process in Figure 2 , pre-train the LSTM prediction model, and the specific steps are as follows:

[0015] 2.1) The n group output results of each sample without light belt in 1) are arranged as a 6-dimensional matrix [sample, x, y, w, h, Pr (object)], which is used as input data for the LSTM prediction model. The output data of the LSTM prediction model is the frequency and power of the electric signal that can make the sample invisible, which is a 2-dimensional data.

[0016] 2.2) Calculate the MSE value between the true value and the predicted value of the frequency and power corresponding to the sample, the coordinates of the frame, the length and width, and the confidence. The formula is as follows:

[0017]

[0018] Where, y i is the true data, is the predicted data, and n represents the number of frequency and power. The MSE is used as a feedback index to feedback to the LSTM prediction model. Through the gradient descent algorithm, the changes of the specific parameters in the model are continuously adjusted, so that the size of MSE tends to 0, and the pre-training of the model is completed.

[0019] 3) According to the process in Figure 2 , combine the LSTM prediction model with the infrared target detection model to form an integrated model, and perform main training of the integrated model, and the specific steps are as follows:

[0020] 3.1) The position (x, y, w, h) of the frame and the confidence Pr (object) of the n group output data obtained by the infrared target detection model in step 1) are used as input data for the LSTM prediction model. The input data is first arranged into a 6-dimensional matrix [sample, x, y, w, h, Pr (object)], and the n group predicted frequency and power of the electric signal are obtained by the LSTM model prediction output. According to the frequency and power of the electric signal, different stroboscopic light belts can be obtained, and the brightness and width of each stroboscopic light belt are known.

[0021] The relationship between the power of the electrical signal and the brightness of the stroboscopic light strip is known:

[0022]

[0023] wherein, is the luminous flux, which refers to the brightness of the stroboscopic light strip; p is the electrical power, which refers to the energy consumed by the electrical signal per unit time; and δ is the luminous efficiency, which refers to the efficiency of the stroboscopic light in converting electrical energy into energy in the visible light band. The luminous efficiency is related to the principle of the stroboscopic light and the light-emitting material.

[0024] At the same time, the relationship between the frequency of the electrical signal and the width of the stroboscopic light strip received by the camera can also be obtained:

[0025]

[0026] wherein, f is the lowest frequency of the electrical signal, v is the propagation speed of the electrical signal, K is the relative conversion speed ratio of the electrical signal converted into the optical signal, i.e., the ratio of the actual conversion speed to the ideal conversion speed when the electrical signal is converted into the optical signal, and H is the width of the stroboscopic light strip received by the camera. To prevent aliasing of the stroboscopic light strip received by the camera, the sampling frequency f s of the camera should be greater than twice the highest frequency f max of the electrical signal.

[0027] 3.2) Add the stroboscopic light strip with n groups of different frequency and power combinations of known frequency and width as a mask to the same sample as the input data of the infrared target detection model. At this time, the output of the infrared target detection model has two possibilities, one is that the model fails to detect, i.e., the sample with the added mask achieves stroboscopic invisibility; the other is that the infrared target detection model still detects the position (x, y, w, h) and confidence Pr(object) of the sample frame.

[0028] 3.3) If the infrared target detection model still detects successfully, the LSTM prediction model needs to be trained. The area of the frame is calculated using the sample frame position (x, y, w, h) detected by the infrared target detection model in the ensemble model:

[0029] S = wh (4)

[0030] S is used as a feedback result to affect the LSTM prediction model. Through the gradient descent algorithm, the internal parameters of the LSTM prediction model are modified, so that when the light strip corresponding to the predicted electrical signal is added as a mask to the sample, the output S of the infrared target detection model tends to 0, i.e., the stroboscopic invisibility of the infrared target detection model is achieved.

[0031] 3.4) In the case that the infrared target detection model fails to detect, the predicted stroboscopic light band is added as a mask to different samples, and whether the detection result of the infrared target detection model in the integrated model fails is observed. If it still fails, the integrated model has been successfully trained. If the infrared target detection model detects successfully, different samples are input to the infrared target detection model, the output data of the model is used, and the gradient descent algorithm is used to continue to correct the internal parameters of the LSTM prediction model until the detection failure rate of the infrared target detection model for different samples is higher than 95%.

[0032] The technical concept of the present application is that, for the task requirement of improving the accuracy of the LSTM prediction model, a pre-training process of adding the LSTM prediction model is designed. The collected samples are first input as input data into the infrared target detection model, and the output data of the infrared target detection model is used to pre-train the LSTM prediction model, adjust the parameters in the LSTM prediction model, improve the accuracy of the result of the LSTM prediction model, and be used for subsequent data prediction. For the task requirement of combining the output of the LSTM prediction model with the input of the infrared target detection model, the structure of the integrated model is designed. The electrical signal is converted into a stroboscopic light band according to a certain relationship, which is added as a mask to the input sample of the infrared target detection model, so that the relationship between the LSTM prediction model and the infrared target detection model is established, and is used for subsequent data prediction and model correction. For the task requirement of determining the frequency and power of the electrical signal that can be stroboscopic stealth, the main training process of the integrated model is designed, the feedback is added, the objective function S is determined, the gradient descent algorithm is used to continuously correct the internal parameters of the LSTM prediction model, S tends to 0, and the purpose of improving the accuracy of the output result of the prediction is achieved. Through the pre-training and main training of the LSTM prediction model, the light band that can attack the infrared target detection model can be predicted and added to the sample to achieve the stealth effect.

[0033] The advantage of the present application is that, by adding the pre-training process of the LSTM prediction model, the internal parameters of the LSTM prediction model can be adjusted to a certain extent before the main training of the integrated model, and the accuracy of the result of the LSTM prediction model is improved. The number of training samples of the model before being put into use is large, the sample stealth effect is ideal, and the confidence of the infrared target detection model can reach a lower range as a whole. The stroboscopic stealth method of the infrared target detection model of the present application is less affected by the environment, is suitable for use in various environments, and can realize self-adjustment and feedback. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a schematic diagram of the training data generation process of the present application.

[0035] Figure 2is a pre-training and main training flowchart of the integrated model of the present application. DETAILED DESCRIPTION

[0036] The technical solutions of the present application are further described below in combination with the drawings.

[0037] The stroboscopic stealth method based on the infrared target detection model of the recurrent neural network includes the following steps:

[0038] 1) Preprocess the collected data, here the images in the INRIA dataset are selected as samples, here 1600 samples in the INRIA training set are taken. Add different frequency and power light strips to the 1600 groups of samples to form another 1600 groups of samples with light strips added. As shown in Figure 1 , the 1600 groups of samples without light strips are input as input data into the infrared target detection model, and the model outputs the position (x, y, w, h) of the detected target frame and the confidence Pr(object) of the frame in the case of successful detection, obtaining 1600 groups of output data. Wherein, x and y refer to the center coordinates of the frame, and w and h are the length and width of the bounding box. When the frame does not contain the target, Pr(object) = 0; when the frame contains the target, Pr(object) = 1. The following table is the output confidence statistics of the infrared target detection model for 1600 groups of samples:

[0039] Table 1 Confidence statistics of infrared target detection model

[0040] Pr(object) = 0 Pr(object) = 1 Sample size 16 1584

[0041] Through the confidence statistics of the infrared target detection model, it can be known that the infrared target detection model has high detection accuracy.

[0042] To find the light strip that can make the infrared target detection model fail to detect, the selection range of the electrical signal can be changed to obtain different light strips. Here the frequency and power of the electrical signal are taken to obtain different width and brightness of the stroboscopic light strip. The frequency range of the electrical signal is 50Hz to 60Hz, and the frequency is taken every 0.25Hz; the power range is 20w to 200w, and the power is taken every 4.5w. Different frequencies and different powers can be combined in pairs, a total of 1600 groups of different combinations of frequency and power. Different frequencies and powers of electrical signals will correspond to different brightness and width of stroboscopic light strips using photoelectric converters. The stroboscopic light strip is added as a mask to the original 1600 groups of samples without light strips as input data for the infrared target detection model to detect, and the output results of the infrared target detection model are recorded. The output results are a six-dimensional matrix, including sample number, detected frame position (coordinates + length and width), confidence, obtaining 1600 groups of output data.

[0043] 2) According to the pre-training process in Figure 2 , the LSTM prediction model is pre-trained, and the specific steps are as follows:

[0044] 2.1) The original input signal size of the LSTM prediction model is 6*1, and after using LSTM, a feature layer with a dimension of 6*16 is obtained, and a new feature layer with a dimension of 6*16 is obtained by using dropout operation, and a feature layer with a dimension of 6 is obtained by using LSTM, and a feature layer with a dimension of 2 is obtained by using full connection, which is the output 2-dimensional data at this time. The 1600 group output results of each sample without adding the light band in 1) are arranged as a matrix with a dimension of 6 [sample, x, y, w, h, Pr (object)], which is used as the input data of the LSTM prediction model. The output data of the LSTM prediction model is the frequency and power of the predicted electrical signal that can make the sample invisible, which is a data with a dimension of 2.

[0045] 2.2) Set the pre-training objective function MSE. Calculate the MSE value between the true value and the predicted value of the corresponding frequency and power under the sample, the coordinates and length and width of the frame, and the confidence. The true value here is the specific value of the 1600 group data with added light band collected in advance. The specific calculation formula of the objective function is as follows:

[0046]

[0047] Where, y i is the true data, is the predicted data, and n represents the number of frequencies and powers. The MSE is fed back to the LSTM prediction model as a feedback index. By using the gradient descent algorithm, iterative solution is performed, and the change of the specific parameters in the model is continuously adjusted, so that the size of MSE tends to 0, and the minimized loss function and model parameter value are obtained, and the pre-training of the model is completed.

[0048] 3) According to the process in Figure 2 , the LSTM prediction model is combined with the infrared target detection model to form an integrated model, and the main training of the integrated model is performed, and the specific steps are as follows:

[0049] 3.1) The position (x, y, w, h) and confidence Pr (object) of the 1600 group output data of the frame obtained by the infrared target detection model in step 1) are arranged as a 6-dimensional input data matrix [sample, x, y, w, h, Pr (object)] of the LSTM prediction model, and the predicted frequency and power of the electrical signal are obtained by the LSTM model prediction output.

[0050] The relationship between the power of the electrical signal and the brightness of the stroboscopic light strip is known:

[0051]

[0052] wherein, is the luminous flux, which refers to the brightness of the stroboscopic light strip; p is the electrical power, which refers to the energy consumed by the electrical signal per unit time; and δ is the luminous efficiency, which refers to the efficiency of the stroboscopic light in converting electrical energy into energy in the visible light band. The luminous efficiency is related to the principle of the stroboscopic light and the light-emitting material.

[0053] The relationship between the frequency of the electrical signal and the width of the stroboscopic light strip received by the camera is known:

[0054]

[0055] wherein f is the lowest frequency of the electrical signal, v is the propagation speed of the electrical signal, K is the relative conversion speed ratio of the electrical signal converted into the optical signal, i.e., the ratio of the actual conversion speed to the ideal conversion speed when the electrical signal is converted into the optical signal, and H is the width of the stroboscopic light strip received by the camera. To prevent aliasing of the stroboscopic light strip received by the camera, the sampling frequency f s of the camera should be greater than twice the highest frequency f max of the electrical signal.

[0056] According to the above relationship, the different stroboscopic light strips can be combined in terms of the frequency and power of the electrical signal, and the corresponding brightness and width of each stroboscopic light strip are known.

[0057] 3.2) 1600 groups of different frequency and power combinations of stroboscopic light strips with known frequency and width are added as masks to the same sample as the input data of the infrared target detection model. At this time, the output of the infrared target detection model has two possibilities, one is that the model fails to detect, i.e., the sample added with the mask achieves the effect of stroboscopic stealth; the other is that the infrared target detection model still detects the position (x, y, w, h) and confidence Pr(object) of the sample frame.

[0058] 3.3) In the case where the infrared target detection model still detects successfully, the LSTM prediction model is then subjected to main training. The area of the frame is calculated using the sample frame position (x, y, w, h) detected by the infrared target detection model in the ensemble model:

[0059] S = wh (4)

[0060] The S is fed back to the LSTM prediction model as a target function. Through the gradient descent algorithm, the internal parameters of the LSTM prediction model are corrected, so that when the predicted electric signal is added to the sample as a mask, the output S of the infrared target detection model tends to 0, that is, the stroboscopic invisibility of the infrared target detection model can be realized.

[0061] 3.4) In the case of failure of the infrared target detection model, the corresponding six-dimensional array information of different samples is input, and the stroboscopic light belt predicted by the LSTM is added to the corresponding different samples as a mask. Observe whether the detection result of the infrared target detection model in the integrated model fails. If it still fails, the integrated model has been successfully trained. If the infrared target detection model detects successfully, different samples are input to the infrared target detection model, the output data of the model is used, and the gradient descent algorithm is used to continue to correct the internal parameters of the LSTM prediction model.

[0062] After completing the main training process of the integrated model, the test set in the INRIA data set is called as test data input to the LSTM model, and the obtained light belt is added to the test set data sample. The sample added with the mask is input to the infrared target detection model as input data, and the output result of the model is obtained. The failure rate of the model output result is counted to obtain the following table:

[0063] Table 2 Success rate of infrared target detection after training

[0064]

[0065] The failure rate of the infrared target detection model is higher than 95%, and the surface model training achievement can achieve the effect of stroboscopic invisibility.

Claims

1. A stroboscopic stealth method for infrared target detection based on a recurrent neural network, comprising the following steps: 1) Generate the 2n sets of data needed to train the model; 2) Pre-train the LSTM prediction model; 3) Combine the LSTM prediction model with the infrared target detection model to form an ensemble model, and perform master training on the ensemble model; specifically including: 3.1) The positions (x, y, w, h) and confidence scores Pr(object) of the boxes obtained from the n sets of samples without light strips in step 1) through the infrared target detection model are used as the input data of the LSTM prediction model. The input data is first organized into a 6-dimensional matrix [sample, x, y, w, h, Pr(object)]. The LSTM model then predicts and outputs n sets of predicted electrical signal frequencies and power in a 2-dimensional matrix. Different strobe light strips can be selected based on the frequency and power of the electrical signal, and the brightness and width of each strobe light strip are known. The relationship between the power of the electrical signal and the brightness of the strobe light strip is known: in, δ is luminous flux, which refers to the brightness of the strobe light strip; p is electrical power, which refers to the energy consumed by the electrical signal per unit time; δ is luminous efficiency, which refers to the efficiency of the strobe light in converting electrical energy into visible light energy; luminous efficiency is related to the principle of the strobe light and the light-emitting materials. At the same time, the relationship between the frequency of the electrical signal and the width of the strobe light strip received by the camera can also be obtained: Where f is the lowest frequency of the electrical signal, v is the propagation speed of the electrical signal, K is the ratio of the relative conversion speeds of the electrical signal to the optical signal (i.e., the ratio of the actual conversion speed to the ideal conversion speed), and H is the width of the strobe light strip received by the camera. To prevent aliasing of the strobe light strip received by the camera, the camera's sampling frequency f is... s It must be greater than the highest frequency f of the electrical signal. max Twice as much; 3.2) Add n sets of strobe light strips with known frequencies and widths, different combinations of frequencies and power, as masks to the same sample, as input data for the infrared target detection model; at this time, the output of the infrared target detection model has two possibilities: one is that the model fails to detect, that is, the sample with added masks achieves the effect of strobe invisibility; the other is that the infrared target detection model still detects the position (x,y,w,h) and confidence Pr(object) of the sample box; 3.3) If the infrared target detection model still succeeds, then the LSTM prediction model needs to be trained; the area of ​​the bounding box is calculated using the position (x, y, w, h) of the sample box detected by the infrared target detection model in the ensemble model. S=wh (4) The S is used as a feedback result and applied to the LSTM prediction model. By correcting the internal parameters of the LSTM prediction model through the gradient descent algorithm, when the light strip corresponding to the predicted electrical signal is added to the sample as a mask, the output S of the infrared target detection model tends to 0, thus achieving the stroboscopic stealth of the infrared target detection model. 3.4) If the infrared target detection model fails, the predicted strobe light strip is added as a mask to different samples. The detection result of the infrared target detection model in the ensemble model is observed. If it still fails, the ensemble model has been successfully trained. If the infrared target detection model succeeds, different samples are input into the infrared target detection model. The gradient descent algorithm is used to continue to correct the internal parameters of the LSTM prediction model using the model's output data until the detection failure rate of the infrared target detection model is higher than 95% for different samples.

2. The stroboscopic stealth method for infrared target detection based on recurrent neural networks as described in claim 1, characterized in that: Step 1) specifically includes: 1.1) The n sets of samples without LED strips are input into the infrared target detection model. The infrared target detection model is based on the YOLOv3 target detection model and has a 53-layer convolutional network. Logistic regression is used to predict the score of each bounding box. When the detection is successful, the model outputs the position (x, y, w, h) of the detected target box and the confidence score Pr(object), resulting in n sets of output data. Here, x and y refer to the center coordinates of the box, and w and h are the length and width of the bounding box. When the box does not contain the target, Pr(object) = 0; when the box contains the target, Pr(object) = 1. 1.2) By setting the frequency and power of the electrical signal, strobe light strips of different widths and brightness are obtained and added to the samples as masks; n different frequencies and powers are combined to generate strobe light strips of different brightness and width as masks and added to the samples. The resulting samples are used as input data for the infrared target detection model. The corresponding output results are matrix arrays with a dimension of 6, including the position (x, y, w, h) and confidence level of the bounding box. The n sets of output data are recorded.

3. The stroboscopic stealth method for infrared target detection based on recurrent neural networks as described in claim 1, characterized in that: Step 2) specifically includes: 2.1) Organize the n output results of the model for each sample without light strips in 1) into a matrix of dimension 6 [sample, x, t, w, h, Pr(object)], and use it as the input data of the LSTM prediction model; the output data of the LSTM prediction model is the frequency and power of the predicted electrical signal that can make the sample invisible, which is data of dimension 2. 2.2) Calculate the MSE values ​​between the true and predicted values ​​of frequency and power at the given sample, frame coordinates, length, width, and confidence level, using the following formula: Among them, y i These are real data. The predicted data is denoted by n, which represents the number of frequencies and powers. The MSE is fed back to the LSTM prediction model as a feedback index. The model's internal parameters are continuously adjusted through the gradient descent algorithm until the MSE approaches zero, thus completing the model's pre-training.

Images