Cross-network traffic anomaly detection method, system, terminal and storage medium

Abstract

The present invention discloses a detection method, system, terminal and storage medium for cross-network traffic anomalies, which belongs to the field of video anomaly detection technology, and solves the problem that video samples cannot use algorithms to detect whether they are abnormal in the future. Resnet residual neural network is used to extract image features from each frame of the initial video sample obtained by the camera, and the CSGAN periodic synthesis generative adversarial network is trained through image feature samples, and the trained network generator is stored; Resnet residual neural network is used to extract real-time features from each frame of the video sample obtained by the camera, and labels of whether anomalies occur in the feature samples are obtained based on the network generator, and actions and state values ​​required for reinforcement learning are generated according to the obtained labels; based on the generated actions and state values, the DQN deep reinforcement learning network is trained in real time to obtain labels of whether the future time period is abnormal. The above scheme can detect video anomalies and ensure driving safety.

Description

Technical Field

[0001] The present invention relates to the technical field of video anomaly detection, and in particular to a method, system, terminal and storage medium for detecting cross-network traffic anomaly. Background Art

[0002] In today's society, rail transit such as subways, light rails, and high-speed rails has gradually become the first choice for citizens' transportation. With the rapid development of rail transit, it also brings great safety hazards. For example, people or other foreign objects entering the unsafe area of ​​rail transit safe driving will seriously affect the normal operation of rail transit and even cause safety accidents. Therefore, it is necessary to quickly and accurately provide rail locomotives with future safety status information to reduce accidents and avoid economic damage. The existing video detection method uses the method of extracting image features from the video for comparison. When detecting the video, it is impossible to add a binary classification label to calibrate the normal or abnormal detection video, and it cannot provide effective learning data for the learning network to predict whether there will be abnormalities in the future. Summary of the invention

[0003] The purpose of the present invention is to provide a method, system, terminal and storage medium for detecting cross-network traffic anomalies, so as to solve the problem that the existing video image processing method cannot judge whether anomalies will occur in the future time period.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A method for detecting cross-network traffic anomalies is provided, comprising the following steps:

[0006] Step S10: obtaining an initial image sample captured by a camera on the locomotive, using a Resnet residual neural network to extract image features of each frame of the initial video sample, using the extracted image features as input features to train a CSGAN periodic synthesis generative adversarial network to obtain a network generator, and storing the obtained network generator;

[0007] Step S20: obtaining a real-time video sample collected by a camera on the locomotive, using a Resnet residual neural network to extract real-time features of each frame of the real-time video sample, obtaining a label indicating whether an abnormality occurs in the real-time feature based on the network generator, and generating actions and state values ​​required for reinforcement learning according to the obtained label;

[0008] Step S30: Based on the generated action and state value, the DQN deep reinforcement learning network is trained in real time to obtain a label indicating whether the future time period is abnormal, and an early warning is issued when an abnormal label appears.

[0009] Compared with the prior art, the present invention has achieved the following technical effects:

[0010] The method for detecting cross-network traffic anomalies provided by the present invention trains a CSGAN periodic synthesis generative adversarial network through initial video samples obtained by a camera on a locomotive to generate a judgment standard, and can set a video feature without a preset label to indicate whether it is abnormal by extracting features of the real-time video samples obtained by the camera and comparing them with the judgment standard; generates actions and state values ​​required for reinforcement learning according to the obtained labels, and uses the actions and state values ​​to perform real-time training on a reinforcement learning network DQN, and predicts whether an abnormal situation will occur in an external video. When it is predicted that an abnormal situation will occur, an early warning is issued, and it can effectively judge whether an abnormal situation will occur in the locomotive in a future time period, thereby ensuring the driving safety of the locomotive.

[0011] A cross-network traffic anomaly detection system is also provided, comprising:

[0012] The autonomous learning module obtains the initial image samples collected by the camera on the locomotive, uses the Resnet residual neural network to extract the image features of each frame of the initial video sample, uses the extracted image features as input features to train the CSGAN periodic synthesis generative adversarial network, obtains the network generator, and stores the obtained network generator;

[0013] It is also used to obtain real-time video samples collected by the camera on the locomotive, use the Resnet residual neural network to extract the real-time features of each frame image in the real-time video samples, obtain the label of whether an abnormality occurs in the real-time features based on the network generator, and generate the action and state value required for reinforcement learning according to the obtained label;

[0014] The deep learning module performs real-time training on the DQN deep reinforcement learning network based on the generated actions and state values, obtains labels indicating whether a future time period is abnormal, and issues an early warning when an abnormal label appears.

[0015] The present invention also provides a terminal, comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes a method for detecting abnormal traffic in a strongly cross-network.

[0016] The present invention also provides a storage medium storing a computer program that can be loaded by a processor and executes a method for detecting abnormal traffic in a strongly cross-network.

[0017] Compared with the prior art, the beneficial effects of the cross-network traffic anomaly detection system, terminal and storage medium provided by the present invention are the same as the beneficial effects of the cross-network traffic anomaly detection method described in the above technical solution, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other and further objects and advantages will become apparent from the following description. The accompanying drawings are intended to illustrate examples of the invention in its various forms. The accompanying drawings should not be construed as limiting the invention to all the ways in which it can be made and adopted. Of course, changes and substitutions can be made to the various components of the invention. The invention also lies in sub-combinations and sub-systems of the elements described, and in methods of using them.

[0019] In the attached picture:

[0020] Figure 1 A method flow chart of a method for detecting cross-network traffic anomalies in an embodiment of the present invention;

[0021] Figure 2 Schematic diagram of the flow of a method for detecting cross-network traffic anomaly in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of the execution operation flow of a periodic synthesis generative adversarial network in an embodiment of the present invention;

[0023] Figure 4 Schematic diagram of the execution operation flow of the deep reinforcement learning network in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0026] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. The meaning of "several" is one or more, unless otherwise clearly and specifically defined.

[0027] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by the terms "up", "down", "front", "back", "left", "right", etc. are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.

[0028] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0029] See Figures 1 to 4 As shown, the method for detecting cross-network traffic anomaly provided by the present invention comprises the following steps:

[0030] Step S10: obtaining an initial image sample captured by a camera on the locomotive, using a Resnet residual neural network to extract image features of each frame of the initial video sample, using the extracted image features as input features to train a CSGAN periodic synthesis generative adversarial network to obtain a network generator, and storing the obtained network generator;

[0031] Step S20: obtaining a real-time video sample collected by a camera on the locomotive, using a Resnet residual neural network to extract real-time features of each frame of the real-time video sample, obtaining a label indicating whether an abnormality occurs in the real-time feature based on a network generator, and generating actions and state values ​​required for reinforcement learning according to the obtained label;

[0032] Step S30: Based on the generated action and state values, the DQN deep reinforcement learning network is trained in real time to obtain a label indicating whether the future time period is abnormal, and an early warning is issued when an abnormal label appears.

[0033] The specific implementation will be carried out according to the following steps:

[0034] Step 1: Use the camera installed on the rail transit equipment to obtain an initial video sample, then take 25 frames of images every 1 second, and perform preliminary feature extraction on each frame through the pre-trained Resnet50 residual neural network as the input feature of the subsequent network.

[0035] Step 2: Train the CSGAN cycle synthesis generative adversarial network through the feature samples extracted in step 1. CSGAN mainly consists of two branches, such as Figure 2 As shown. The CSGAN cycle synthesis generative adversarial network is divided into two branches. One branch is that the network generator uses an autoencoder module to reconstruct the input batch features, and then uses the reconstruction error. When the error is higher than a certain threshold we set, it is marked as abnormal, otherwise it is normal and marked as abnormal, which is equivalent to creating a pseudo label. Then the pseudo label can be used for the network generator. The training error calculation process is as follows:

[0036]

[0037] In the formula For the The generator reconstruction error, is the total number of batches of samples, and are the input and output values ​​of the network generator, is the image feature generated by the Resnet residual neural network. Generates error threshold for the network generator, and are the pseudo labels generated by the generator and the discriminator respectively, and are the mean and standard deviation of the reconstruction error of each batch, respectively.

[0038] The other branch is that the discriminator uses a fully connected network. The discriminator gives an estimated probability value of an instance being abnormal, and the probability value can be used to give a pseudo label. For example, if there are two features in the batch feature with high probability values ​​(higher than a certain threshold we set), these two features are marked as abnormal, which is equivalent to creating pseudo labels. These pseudo labels are then used to improve the network generator. The training error calculation process is as follows:

[0039]

[0040] In the formula, is the cross entropy error generated by the discriminator, is the discriminator output value, Generates error thresholds for the discriminator (and is calculated in the same way).

[0041] Step 3: Use the real-time video samples obtained by the cameras installed on the rail transit equipment, and then use the same Resnet residual neural network as step 1 to extract features and obtain real-time feature samples.

[0042] Step 4: Obtain labels of whether real-time feature samples are abnormal through the network generator in CSGAN, and generate actions and states required for reinforcement learning based on these labels. The label generated by the network generator in CSGAN for each frame is taken as an action, and the features extracted by the Resnet residual neural network of 15 consecutive frames and the actions corresponding to each frame are connected and regarded as a state value. The state value is put into the reinforcement network to obtain the action of the next frame.

[0043] Step 5: Train the DQN deep reinforcement learning network in real time and obtain the label of whether the future time period is abnormal, that is, select the strategy with the largest Q value as the optimal action. There are two networks in the DQN deep reinforcement learning network: the evaluation network and the target network, such as Figure 3 As shown in the figure, the two networks are LSTM long short-term memory networks with the same parameters. The TD-target time difference objective function is calculated through experience playback and fixed Q target parameter mechanism: And TD-error time difference error: , as shown below:

[0044]

[0045] In the formula for Rewards at all times, is the discount factor, and They are The state and action of each moment, The parameters of all networks in the DQN deep reinforcement learning network are then updated using the online gradient descent method:

[0046]

[0047] In the formula is the gradient, is the learning rate, In order to seek partial guidance.

[0048] Step 6: Store the real-time video. When the length of the stored video is the same as the initial video for training CSGAN, replace the previous video with the latest video to continue training CSGAN. After training, replace the previous CSGAN model.

[0049] The generator in CSGAN and the LSTM in DQN use mean square error as the loss function, the discriminator in CSGAN uses binary cross entropy as the loss function, and all networks use Adam (optimization algorithm) as the optimizer during training.

[0050] By training the generative adversarial network using cross-supervision, the video features without preset labels can be set with abnormal labels; using real-time training to train the reinforcement learning network DQN and obtain information on whether there are abnormalities in the future, the real-time collected data can be fully utilized to extract features, and the continuous updating of samples can make the network more robust. After the system is trained, it can provide abnormal information to rail transit equipment in advance without preset labels, and provide guidance data for the safe operation of rail transit.

[0051] The method for detecting cross-network traffic anomalies provided by the present invention trains a CSGAN periodic synthesis generative adversarial network through initial video samples obtained by a camera on a locomotive to generate a judgment standard, and can set a video feature without a preset label to indicate whether it is abnormal by extracting features of the real-time video samples obtained by the camera and comparing them with the judgment standard; generates actions and state values ​​required for reinforcement learning according to the obtained labels, and uses the actions and state values ​​to perform real-time training on a reinforcement learning network DQN, and predicts whether an abnormal situation will occur in an external video. When it is predicted that an abnormal situation will occur, an early warning is issued, and it can effectively judge whether an abnormal situation will occur in the locomotive in a future time period, thereby ensuring the driving safety of the locomotive.

[0052] As an implementable method, before using the Resnet residual neural network to extract the image features of each frame of the initial video sample, the following steps are also included:

[0053] Use ImageNet to train the network model for video feature extraction in the Resnet residual neural network.

[0054] By training the network model in the Resnet residual neural network, the network parameters trained in the large sample data set are mainly transferred, which saves the training cost of the model and makes the network model more targeted when extracting features from the initial video samples and real-time video samples, and also ensures the effectiveness of the trained network generator.

[0055] As an implementable method, generating the action and state values ​​required for reinforcement learning according to the acquired labels includes the following steps:

[0056] The value of the label generated by the network generator corresponding to each frame of the image is used as the action, and the image features of 15 consecutive frames of images extracted by the Resnet residual neural network are connected with the action corresponding to each frame of the image as a state value.

[0057] The actions and states required for reinforcement learning provide data support for the judgment of actions in subsequent videos by extracting image features from a continuous number of images, ensuring the accuracy of judgment on obtaining real-time feature samples.

[0058] As an implementable method, the anomaly detection method further includes the following steps:

[0059] The video captured by the camera on the locomotive in real time is stored. When the length of the stored video is the same as the length of the initial video sample for training the CSGAN cyclic synthetic generative adversarial network, the latest video is used to replace the previous video to continue training the CSGAN cyclic synthetic generative adversarial network, and the newly trained network generator is used to replace the previous network generator.

[0060] By continuously updating the prior videos and the prior network generator, the timeliness of the learning videos and the generated network generator is guaranteed. By updating the videos and the network generator, the regularity of the later data can be intuitively reflected.

[0061] The present invention also provides a cross-network traffic anomaly detection system, comprising:

[0062] The autonomous learning module obtains the initial image samples collected by the camera on the locomotive, uses the Resnet residual neural network to extract the image features of each frame of the initial video sample, uses the extracted image features as input features to train the CSGAN periodic synthesis generative adversarial network, obtains the network generator, and stores the obtained network generator;

[0063] It is also used to obtain real-time video samples collected by the camera on the locomotive, use the Resnet residual neural network to extract the real-time features of each frame of the real-time video samples, obtain the label of whether an abnormality occurs in the real-time features based on the network generator, and generate the action and state value required for reinforcement learning according to the obtained label;

[0064] The deep learning module trains the DQN deep reinforcement learning network in real time based on the generated actions and state values, obtains labels for whether the future time period is abnormal, and issues an early warning when an abnormal label appears.

[0065] The detection system for cross-network traffic anomalies provided by the present invention trains a CSGAN periodic synthesis generative adversarial network through initial video samples obtained by a camera on a locomotive to generate a judgment standard. By extracting features of real-time video samples obtained by the camera and comparing them with the judgment standard, video features without preset labels can be set to indicate whether they are abnormal. Actions and state values ​​required for reinforcement learning are generated according to the obtained labels, and the reinforcement learning network DQN is trained in real time using the actions and state values ​​to predict whether an abnormal situation will occur in an external video. When it is predicted that an abnormal situation will occur, an early warning is issued, and it can effectively judge whether an abnormal situation will occur in the locomotive in a future time period, thereby ensuring the driving safety of the locomotive.

[0066] As an implementable method, the autonomous learning module is also used to train a network model for video feature extraction in a Resnet residual neural network using ImageNet.

[0067] By training the network model in the Resnet residual neural network, the network model is more targeted when extracting features from initial video samples and real-time video samples, and the effectiveness of the trained network generator is also guaranteed.

[0068] As an implementable method, the autonomous learning module is also used to use the value of the label generated by the network generator corresponding to each frame of the image as an action, and connect the image features of 15 consecutive frames of images extracted by the Resnet residual neural network with the action corresponding to each frame of the image as a state value.

[0069] By training the network model in the Resnet residual neural network, the network parameters trained in the large sample data set are mainly transferred, which saves the training cost of the model and makes the network model more targeted when extracting features from the initial video samples and real-time video samples, and also ensures the effectiveness of the trained network generator.

[0070] As an implementable method, the autonomous learning module is also used to store the video collected in real time by the camera on the locomotive. When the length of the stored video is the same as the length of the initial video sample for training the CSGAN cyclic synthetic generative adversarial network, the latest video is used to replace the previous video to continue training the CSGAN cyclic synthetic generative adversarial network, and the newly trained network generator is used to replace the previous network generator.

[0071] By continuously updating the prior videos and prior network generators, the timeliness of learning videos and generating network generators is guaranteed. By updating the videos and network generators, the regularity of later data can be intuitively reflected.

[0072] The present invention also provides a terminal, comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes a method for detecting cross-network traffic anomalies.

[0073] The present invention also provides a storage medium storing a computer program that can be loaded by a processor and execute the method for detecting cross-network traffic anomalies.

[0074] In the description of the above embodiments, specific features, structures, materials or characteristics may be combined in a suitable manner in any one or more embodiments or examples.

[0075] Of course, various changes and substitutions can be made to the above description, and all of these changes and substitutions are within the spirit and scope of the present invention. Therefore, the present invention should not be limited except by the attached claims and their equivalents.

Claims

1. A method for detecting cross-network traffic anomalies, characterized in that: The following steps are involved: step S10: obtaining an initial video sample captured by a camera on a locomotive, using a Resnet residual neural network to extract image features of each frame of the initial video sample, using the extracted image features as input features to train a CSGAN periodic synthesis generative adversarial network to obtain a network generator, and storing the obtained network generator; Step S20: obtaining a real-time video sample collected by a camera on the locomotive, using a Resnet residual neural network to extract real-time features of each frame of the real-time video sample, obtaining a label indicating whether an abnormality occurs in the real-time feature based on the network generator, and generating actions and state values ​​required for reinforcement learning according to the obtained label; Step S30: Based on the generated action and state value, the DQN deep reinforcement learning network is trained in real time to obtain a label indicating whether the future time period is abnormal, and when an abnormal label appears, an early warning is issued; The label generated by the network generator in CSGAN corresponding to each frame is regarded as an action, and the features extracted by the Resnet residual neural network of 15 consecutive frames and the action corresponding to each frame are connected and regarded as a state value; The CSGAN cycle synthesis generative adversarial network is divided into two branches. One branch is that the network generator uses an autoencoder module to reconstruct the input batch features, and then uses the reconstruction error. When the reconstruction error is higher than the set threshold, it is marked as abnormal, otherwise it is normal and marked as abnormal, which is equivalent to creating a pseudo label, and then the pseudo label is used for the network generator; The other branch is that the discriminator uses a fully connected network. The discriminator gives a probability estimate of an instance abnormality, and uses the probability estimate to add pseudo labels. When there are two feature probability values ​​in the batch features that are higher than the set threshold, these two features are marked as abnormal, which is equivalent to creating pseudo labels, and then these pseudo labels are used to improve the network generator.

2. The method for detecting cross-network traffic anomalies according to claim 1, characterized in that: Before extracting the image features of each frame of the initial video sample using the Resnet residual neural network, the following steps are also included: Use ImageNet to train the network model for video feature extraction in the Resnet residual neural network.

3. The method for detecting cross-network traffic anomalies according to claim 1, characterized in that: Generating the action and state values ​​required for reinforcement learning according to the acquired labels includes the following steps: The value of the label generated by the network generator corresponding to each frame of the image is used as an action, and the image features of 15 consecutive frames of images extracted by the Resnet residual neural network and the action corresponding to each frame of the image are connected as a state value.

4. The method for detecting cross-network traffic anomalies according to claim 1, characterized in that: The abnormality detection method further comprises the following steps: The video captured by the camera on the locomotive in real time is stored. When the length of the stored video is the same as the length of the initial video sample for training the CSGAN cyclic synthetic generative adversarial network, the latest video is used to replace the previous video to continue training the CSGAN cyclic synthetic generative adversarial network, and the newly trained network generator is used to replace the previous network generator.

5. A cross-network traffic anomaly detection system, characterized in that: include: The autonomous learning module obtains the initial video sample collected by the camera on the locomotive, uses the Resnet residual neural network to extract the image features of each frame image in the initial video sample, uses the extracted image features as input features to train the CSGAN periodic synthesis generative adversarial network to obtain a network generator, and stores the obtained network generator; It is also used to obtain real-time video samples collected by the camera on the locomotive, use the Resnet residual neural network to extract the real-time features of each frame image in the real-time video samples, obtain the label of whether an abnormality occurs in the real-time features based on the network generator, and generate the action and state value required for reinforcement learning according to the obtained label; A deep learning module performs real-time training on a DQN deep reinforcement learning network based on the generated actions and state values, obtains a label indicating whether a future time period is abnormal, and issues an early warning when an abnormal label appears; The label generated by the network generator in CSGAN corresponding to each frame is regarded as an action, and the features extracted by the Resnet residual neural network of 15 consecutive frames and the action corresponding to each frame are connected and regarded as a state value; The CSGAN cycle synthesis generative adversarial network is divided into two branches. One branch is that the network generator uses an autoencoder module to reconstruct the input batch features, and then uses the reconstruction error. When the reconstruction error is higher than the set threshold, it is marked as abnormal, otherwise it is normal and marked as abnormal, which is equivalent to creating a pseudo label, and then the pseudo label is used for the network generator; The other branch is that the discriminator uses a fully connected network. The discriminator gives a probability estimate of an instance abnormality, and uses the probability estimate to add pseudo labels. When there are two feature probability values ​​in the batch features that are higher than the set threshold, these two features are marked as abnormal, which is equivalent to creating pseudo labels, and then these pseudo labels are used to improve the network generator.

6. The cross-network traffic anomaly detection system according to claim 5, characterized in that: The autonomous learning module is also used to train the network model for video feature extraction in the Resnet residual neural network using ImageNet.

7. The cross-network traffic anomaly detection system according to claim 5, characterized in that: The autonomous learning module is also used to use the value of the label generated by the network generator corresponding to each frame of the image as an action, and connect the image features of 15 consecutive frames of images extracted by the Resnet residual neural network and the actions corresponding to each frame of the image as a state value.

8. The cross-network traffic anomaly detection system according to claim 5, characterized in that: The autonomous learning module is also used to store the video collected in real time by the camera on the locomotive. When the length of the stored video is the same as the length of the initial video sample for training the CSGAN periodic synthesis generative adversarial network, the latest video is used to replace the previous video to continue training the CSGAN periodic synthesis generative adversarial network, and the newly trained network generator is used to replace the previous network generator.

9. A terminal, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes the method according to any one of claims 1 to 4.

10. A storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 4.

Images