Monitoring video stream privacy protection method based on deep learning and chaotic encryption

Abstract

The invention provides a monitoring video stream privacy protection method based on deep learning and chaotic encryption, and belongs to the field of monitoring video privacy protection, and the method comprises a video stream privacy extraction module, a privacy information encryption module and a video stream privacy restoration module. The video stream privacy extraction module is used for detecting privacy information in a video stream picture by using a deep learning model so as to extract image data for encryption, and the privacy information encryption module is used for processing the extracted privacy information by using a chaos algorithm so as to obtain an encrypted privacy image; the video stream privacy restoration module is used for carrying out inverse operation on the encrypted privacy image by using a chaos algorithm to carry out decryption so as to obtain restored privacy information for evidence collection; according to the method, on the basis of an existing method, the detection precision of the privacy information is improved, the security of the privacy information is enhanced, the reversibility of an encryption algorithm used by the method is benefited, and the method can provide an evidence obtaining way for the privacy information for managers.

Description

Technical Field

[0001] The present invention relates to the field of surveillance video privacy protection, and in particular to a surveillance video stream privacy protection method based on deep learning and chaotic encryption. Background Art

[0002] Privacy protection of surveillance video streams is a technical method that aims to protect the privacy of individuals in surveillance videos. Its core goal is to prevent personal identity information or sensitive data from being improperly exposed and abused by processing video data without affecting the normal use of surveillance videos. These technologies usually involve image processing, computer vision, and privacy enhancement technologies, and are important tools for ensuring a balance between public safety and privacy rights.

[0003] At present, the privacy protection technologies for surveillance video streams mainly include global non-realistic rendering and local de-identification. Global non-realistic rendering constructs and trains a pair of codecs based on each set of keys through a deep learning algorithm. The encoder is used for rendering and desensitization, and the decoder is used for restoration. However, this method easily leads to the loss of semantic information of the original video, which makes it impossible for managers to conduct behavioral analysis. At the same time, each time the key is changed, a pair of codecs needs to be retrained, which has high training costs. Therefore, local de-identification is a type of algorithm that is used more frequently in current algorithms. By analyzing the video stream frame by frame, the privacy area is extracted using methods such as moving object detection and face detection, and finally the privacy area is replaced using methods such as Gaussian blur and image distortion. Researchers have continuously proposed new algorithms and new technologies to meet people's needs, but there are still many difficulties and there is a lot of room for improvement.

[0004] Although the existing technology has made significant progress, there are still many problems to be solved. First, the existing traditional methods have low accuracy in detecting privacy information, and often have false detections and missed detections, which is unacceptable in the highly sensitive field of privacy protection; second, the conflict between privacy protection and video availability has not been completely balanced. The use of overly simple encryption technology will lead to low security of privacy protection, but if overly complex encryption technology is used only to improve the security of privacy protection, it may affect the effectiveness of video for monitoring, tracking and evidence collection; in addition, the real-time performance of privacy protection technology is still a difficulty, especially when processing high-definition or multi-channel video streams, efficient privacy processing algorithms often cannot take into account both processing speed and effect. These are important technical issues in this field. Summary of the invention

[0005] The present invention discloses a surveillance video stream privacy protection method based on deep learning and chaotic encryption to improve the detection accuracy of private information and provide good recoverability for evidence collection while ensuring the security of private content.

[0006] In an embodiment of the present disclosure, the monitoring video stream privacy protection method includes a video stream privacy extraction module, a privacy information encryption module, and a video stream privacy restoration module. The video stream privacy extraction module uses the MTCNN model based on deep separable convolution for face detection and the YOLOv5 model for license plate detection; the privacy information encryption module uses gray-space-based Logistic chaotic encryption and improved position-space-based Arnold scrambling to encrypt the privacy information. The video stream privacy restoration module uses the inverse operation of Logistic chaotic encryption and Arnold scrambling to decrypt the encrypted privacy image.

[0007] This method can be deployed in edge computing devices in real-world scenarios, such as surveillance cameras and smart terminals. The process is as follows: first, provide a set of keys, use the camera to start capturing the video stream, then detect the video stream frame by frame, obtain all privacy areas containing face and license plate information, and then use the pre-provided key to encrypt and scramble the area, output the processed video stream, and finally send the video stream data to the monitoring management room through the Socket connection; when it is necessary to obtain evidence from the encrypted video stream, just enter the correct key, wait for it to be decrypted and reversed, and you can get the initial video stream. The entire business process is as shown in the attached figure. Figure 2 shown.

[0008] For face detection, this method uses the MTCNN (Multi-Task Cascaded Convolutional Networks) model based on depthwise separable convolutions. This model is composed of multiple cascaded convolutional neural networks and can quickly find the location area of the human face. Since the three network structures of this model are all standard convolutional kernels, it has high requirements for the performance of the device. If the device performance is poor, real-time performance cannot be achieved. Therefore, some ordinary convolutional layers of the original model are selected to be replaced with depthwise separable convolution structures to adapt to edge devices with low computing power and improve the real-time performance of the method. For license plate detection, this method uses the YOLOv5 algorithm model to implement. The advantage of the YOLO series of algorithms lies in its rapidity, accuracy, and adaptability to complex environments. This technology has been widely used in fields such as security monitoring, autonomous driving, and intelligent robots and can be well applied to the license plate detection task in surveillance videos. In the image grayscale space, this method selects Logistic chaotic encryption. Chaotic motion has the following characteristics: ① It is neither periodic nor convergent, and the points on the motion trajectory traverse the entire region; ② The motion trajectory continuously stretches and folds within a finite region, making the system output similar to random noise; ③ The system motion is extremely sensitive to the initial value, and the long-term motion of any two arbitrarily close points is unpredictable. Therefore, it can be applied to encryption and decryption. This method requires the parameters of the algorithm to satisfy 0 < X0 < 1 and 3.5699456 < μ ≤ 4. At this time, the Logistic function is in a disordered and unpredictable chaotic state and can be used to generate encrypted chaotic sequences. For the position space, this method selects an improved Arnold scrambling algorithm for encryption. Scrambling encryption changes the pixel arrangement order of the image by performing operations of confusing and rearranging the image, making the image visually difficult to understand or decipher, thereby increasing the security of the image or protecting the privacy of the image content. The original Arnold scrambling algorithm requires the length and width of the input image to be the same, resulting in most images being unable to be scrambled. This method changes the fixed modulus of the original formula and introduces a parameter to reflect the relationship between the length and width of the image, enabling it to scramble images with inconsistent length and width. For the situation where video forensics for restoration is required, this method can use the inverse operations of Logistic chaotic encryption and Arnold scrambling to decrypt the encrypted privacy image.

[0009] Based on the above, compared with most existing methods, the embodiments of the present disclosure improve the detection accuracy of privacy information, reduce the computational amount of deep learning algorithms, and improve the feasibility of using them for real-time detection. While ensuring the security of privacy content, it provides a way for managers to obtain evidence of privacy content in surveillance videos. Brief Description of the Drawings

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0011] Figure 1 It is a schematic diagram of each submodule of a surveillance video stream privacy protection method based on deep learning and chaotic encryption of the present invention.

[0012] Figure 2 It is a schematic diagram of the business process in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with specific examples and with reference to the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0014] See also Figure 1 The present invention provides a surveillance video stream privacy protection method based on deep learning and chaotic encryption, which includes the following three submodules: a video stream privacy extraction module, a privacy information encryption module and a video stream privacy restoration module. The specific implementation methods of each submodule are described in detail below:

[0015] The video stream privacy extraction module uses the MTCNN model based on deep separable convolution for face detection. In this embodiment, MTCNN has a three-layer structure, which consists of three networks, namely P-Net, R-Net and O-Net. The implementation process has four steps. The first is image preprocessing. The image is scaled by different scaling factors to build an image pyramid to detect faces in images of different sizes. Then, P-Net is used to estimate and judge the possible locations of faces in the image, and a fully convolutional neural network (FCN) is used to perform preliminary feature extraction and calibrate borders, and NMS is used to remove repeated borders. R-Net adds a fully connected layer to strictly screen and judge the output results of P-Net, and uses NMS for processing. Finally, Output-Net adds another convolutional layer. This layer of structure will identify the facial area through more supervision, and will regress the facial feature points of the person to obtain 5 facial feature points of the face, and then obtain the location information of the face. This embodiment uses the CelebA (CelebFaces Attributes) dataset to train the MTCNN model. It is necessary to prepare three types of image samples: positive samples, negative samples, and partial samples. The number of each sample accounts for about 1:1:3. Because a large number of negative samples will be lost in the cascade processing, it is necessary to ensure that the proportion of the three samples entering R-Net and O-Net is balanced. Positive samples refer to images containing the target object (here, the face), which helps the model learn the characteristics of the face area, thereby improving the accuracy of the model in the detection task; negative samples refer to pictures that do not contain the face area, which helps the model learn how to distinguish the face area from the background, and can help the model avoid misidentifying the non-face area as the face area, thereby improving the robustness of the model; and partial samples refer to samples that are difficult to clearly classify as positive and negative examples. Usually, only one part is the face area and the other part is the non-face area. It helps the model learn how to deal with blurry situations, so that it can better handle complex scenes and improve the generalization ability of the model. The image sizes required for the three network structures of the model are different. P-Net requires 12×12, R-Net requires 24×24, and O-Net requires 48×48. It is necessary to use image processing libraries such as OpenCV and PIL to crop and scale the images according to the facial feature point annotations of each image in the dataset so that the image size meets the requirements of the model. The specific steps are as follows: First, initialize the ratio of positive, negative and partial samples according to the requirements, then read the txt annotation file provided by the CelebA dataset line by line, and calculate the accurate face position coordinates of the corresponding image through the data provided. Then randomly offset from the coordinate center of the face area, loop to obtain four new sets of coordinates, and crop and scale according to these four sets of coordinates to meet the current model to be trained.Then, calculate the IOU values of the bounding boxes selected by the four groups of coordinates and the accurate face box respectively, and assign the sample types according to the IOU values. If the IOU value is greater than 0.65, it is put into the positive sample; if the IOU value is less than 0.1, it is put into the negative sample; if the IOU value is greater than 0.4 and less than 0.65, it is added to the partial sample. Loop until the txt file is read and all images are processed. First, initialize the proportions of positive, negative, and partial samples according to the requirements, then read the txt file provided by the CelebA dataset line by line, and calculate the accurate face position coordinates of the corresponding images through the data it provides. Then, randomly offset from the coordinate center of the face area, loop to obtain four groups of new coordinates, and perform cropping and scaling according to these four groups of coordinates to make it meet the model to be trained. Then, calculate the IOU values of the bounding boxes selected by the four groups of coordinates and the accurate face box respectively, and assign the sample types according to the IOU values. If the IOU value is greater than 0.65, it is put into the positive sample; if the IOU value is less than 0.1, it is put into the negative sample; if the IOU value is greater than 0.4 and less than 0.65, it is added to the partial sample. After processing, three datasets will be obtained, corresponding to three image types of 12×12, 24×24, and 48×48 respectively. Finally, use these three datasets to train three models respectively.

[0016] In this embodiment, the YOLOv5 model is used for license plate detection, and the CCPD2019 dataset is used to train the YOLOv5 model. Similarly, since the input image size of the YOLOv5 model is 608×608×3, while the image size provided by the dataset is 720×1160×3, it is necessary to preprocess it. After converting the image into a 1:1 image ratio through cropping, padding, etc., and then scaling it to 608×608, it can meet the model input requirements.

[0017] In this embodiment, the face images and license plate images detected by the above two models are regarded as privacy information in the surveillance video, and the privacy information is encrypted using Logistic chaos encryption based on the gray space and improved Arnold scrambling based on the position space in the privacy information encryption module. In this embodiment, the parameters of Logistic chaos encryption need to satisfy 0 < X0 < 1, 3.5699456 < μ ≤ 4. Select a set of X0 and μ that meet the conditions as the key, and substitute them into the Logistic mapping equation X k+1 = μ * X k * (1 - X k ), k = 0, 1,..., n to obtain the chaotic sequence X1, X2,..., X n (0 < X i<1), normalize the chaotic sequence to (0, 255) to get sequence B, then transform the one-dimensional sequence B into a two-dimensional matrix with the same length and width as the image to be encrypted, and use this matrix to perform XOR with the image matrix to get the encrypted image.

[0018] The Arnold scrambling encryption algorithm used in this embodiment is based on the original formula The improvement is to convert the original calculation x ′ and ′ The modulus N used was changed to M and N respectively, and the The improved formula used to adjust the effect of length and width inconsistency is: This embodiment can restore the above two encrypted privacy information in the video stream privacy restoration module. The specific implementation method is: use the inverse operation of Logistic chaotic encryption and Arnold scrambling to decrypt the encrypted privacy image. The inverse operation of Logistic chaotic encryption uses the encrypted key to obtain the same chaotic sequence as the encryption through the Logistic mapping equation, and normalizes the chaotic sequence to (0, 255) to obtain sequence B. Then, the one-dimensional sequence B is converted into a two-dimensional matrix with the same length and width as the encrypted image. The matrix is ​​used to perform XOR with the encrypted image matrix to obtain the decrypted and restored image; the inverse operation of Arnold scrambling can decrypt the encrypted image to achieve the restoration effect through the inverse transformation formula, and its inverse transformation formula is:

[0019] It should be noted that what is disclosed above is only a preferred embodiment of the present invention, and the scope of rights of the present invention cannot be limited thereto. A person skilled in the art can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A surveillance video stream privacy protection method based on deep learning and chaotic encryption: characterized in that, The following steps are involved: Step 1: Input the pre-processed surveillance video screen into the video stream privacy extraction module to extract the privacy information to be encrypted in the screen; Step 2: The privacy information encryption module processes the extracted privacy information using a chaotic algorithm to obtain an encrypted privacy image; Step 3: The video stream privacy restoration module uses the inverse operation of the chaos algorithm to decrypt the encrypted privacy image and obtain the restored privacy information for evidence collection.

2. According to claim 1, the privacy protection method for surveillance video stream based on deep learning and chaotic encryption is characterized in that: The video stream privacy extraction module uses the MTCNN model based on deep separable convolution for face detection and the YOLOv5 model for license plate detection.

3. The surveillance video stream privacy protection method based on deep learning and chaotic encryption according to claim 2 is characterized in that: The privacy information encryption module uses gray-space-based Logistic chaotic encryption and improved position-space-based Arnold scrambling to encrypt the privacy information.

4. The surveillance video stream privacy protection method based on deep learning and chaotic encryption according to claim 3 is characterized in that: The video stream privacy restoration module uses the inverse operation of Logistic chaotic encryption and Arnold scrambling to decrypt the encrypted privacy image.

5. The surveillance video stream privacy protection method based on deep learning and chaotic encryption according to claim 2 is characterized in that: The MTCNN model includes three networks: P-Net, R-Net and O-Net: The original P-Net consists of four ordinary convolutional layers: the convolution kernel size of the first three convolutional layers is 3×3, the stride is 1, the padding is 1, and the number of convolution kernels is 10, 16, and 32 respectively. There is a PReLU activation function after each layer. The activation function of the first convolutional layer is preceded by a maximum pooling layer with a convolution kernel size of 2×2 and a stride of 2; the last convolutional layer contains three tasks: classification, bounding box regression, and key point regression. The convolution kernel size is 1×1, the stride is 1, and the padding is 0. The number of convolution kernels for the three tasks is 2, 4, and 10 respectively; The original R-Net consists of three ordinary convolutional layers and two fully connected layers: the first two convolutional layers have a convolution kernel size of 3×3, a stride of 1, a padding of 1, and the number of convolution kernels is 28 and 48 respectively. Each layer is followed by a maximum pooling layer with a convolution kernel size of 3×3, a stride of 2, and a PReLU activation function; the third convolutional layer has a convolution kernel size of 2×2, a stride of 1, a padding of 0, and a number of convolution kernels of 64, followed by a PReLU activation function; the first fully connected layer has 128 output channels, followed by a PReLU activation function, and the second fully connected layer contains three tasks: classification, bounding box regression, and key point regression. The number of output channels for the three tasks is 2, 4, and 10 respectively; The original O-Net consists of four ordinary convolutional layers and two fully connected layers: the convolution kernel size of the first three convolutional layers is 3×3, the stride is 1, the padding is 1, and the number of convolution kernels is 32, 64, and 64 respectively. There is a maximum pooling layer and a PReLU activation function after each layer. The convolution kernel size used in the maximum pooling layer after each convolutional layer is 3×3, 3×3, and 2×2 respectively, and the step size is 2; the convolution kernel size of the fourth convolutional layer is 2×2, the stride is 1, the padding is 0, the number of convolution kernels is 128, and there is a PReLU activation function after it; the number of output channels of the first fully connected layer is 256, followed by a PReLU activation function, and the second fully connected layer contains three tasks: classification, bounding box regression, and key point regression. The number of output channels of the three tasks is 2, 4, and 10 respectively; The MTCNN model based on depthwise separable convolution is embodied in that the first three ordinary convolutional layers of the P-Net network, the first two ordinary convolutional layers of the R-Net network, and the first three ordinary convolutional layers of the O-Net network in the original model are replaced with depthwise separable convolutional structures, and the specific steps are as follows: Replace the ordinary convolution layer with a kernel size of k×k, a stride of s, a padding of p, and n kernels, and its subsequent activation function, with a structure consisting of a depthwise convolution layer, an activation function, and a pointwise convolution layer, where the kernel size of the depthwise convolution layer is k×k, the stride of s, the padding of p, and the number of kernels is n; the kernel size of the pointwise convolution layer is 1×1, the stride of s, the padding of 0, and the number of kernels is n; By introducing a depth-wise separable convolutional structure, the number of model parameters and the amount of computation during inference are reduced; The MTCNN model based on deep separable convolution is used for face detection. The model input is 12×12×3. After reasoning by three networks, the results of three tasks are output, including face bounding box detection results, face key point detection results, and the probability that the candidate box content is a face.

6. The surveillance video stream privacy protection method based on deep learning and chaotic encryption according to claim 2 is characterized in that: The YOLOv5 model is used for license plate detection. The input of the model is 608×608×3. After reasoning by the YOLOv5 model, the output results include the license plate bounding box detection result and the probability that the candidate box content is the license plate.

7. The surveillance video stream privacy protection method based on deep learning and chaotic encryption according to claim 3 is characterized in that: For the use of Logistic chaotic encryption based on the gray space, the parameters of the encryption algorithm need to satisfy 0 < X0 < 1, 3.5699456 < μ ≤ 4. Select a set of X0 and μ that meet the conditions as the key, and substitute them into the Logistic mapping equation X k+1 = μ * X k * (1 - X k ), k = 0, 1, …, n to obtain the chaotic sequence X1, X2, …, X n (0 < X i < 1), normalize the chaotic sequence to the range (0, 255) to obtain the sequence B. Then convert this one-dimensional sequence B into a two-dimensional matrix with the same length and width as the image to be encrypted, and perform an exclusive OR operation between this matrix and the image matrix to be encrypted to obtain the encrypted image.

8. The surveillance video stream privacy protection method based on deep learning and chaotic encryption according to claim 3 is characterized in that: The improved Arnold scrambling based on the position space is used to encrypt the private information. The original Arnold scrambling calculation formula is: and y′ are the pixel coordinates after transformation, x and y are the pixel coordinates that have not been transformed; a and b are arbitrary integers; M and N are the length and width of the image respectively; this formula requires that the length and width of the input image are consistent, which makes it impossible to scramble most images; The improved Arnold scrambling encryption algorithm changes the modulus N used to calculate x′ and y′ to M and N respectively, and introduces The improved formula used to adjust the effect of length and width inconsistency is:

9. The surveillance video stream privacy protection method based on deep learning and chaotic encryption according to claim 4 is characterized in that: The encrypted privacy image is decrypted by using the inverse operation of Logistic chaotic encryption and Arnold scrambling. The inverse operation of Logistic chaotic encryption uses the encrypted key to obtain the same chaotic sequence as the encryption through the Logistic mapping equation, and the chaotic sequence is normalized to between (0, 255) to obtain a sequence B. Then, the one-dimensional sequence B is converted into a two-dimensional matrix with the same length and width as the encrypted image. The matrix is ​​used to perform XOR with the encrypted image matrix to obtain the decrypted and restored image. The inverse operation of Arnold scrambling can decrypt the encrypted image to achieve the restoration effect through the inverse transformation formula, and the inverse transformation formula is:

Images