Visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatiotemporal information transfer compensation

Through DCGAN space-time information migration compensation technology and joint classification error regularization, the privacy leakage problem of visual privacy protection and fall behavior monitoring is solved, and high-accurate fall behavior monitoring for the elderly is achieved, ensuring the privacy and recognition effect of video content .

CN114998805BActive Publication Date: 2024-11-15NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210684916.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-14
Publication Date
2024-11-15
Estimated Expiration
2042-06-14

AI Technical Summary

Technical Problem

Existing vision-based home health monitoring methods pose a risk of privacy leakage when monitoring the fall behavior of the elderly, and traditional image processing methods are difficult to effectively block video content, affecting the monitoring effect.

Method used

Using the spatial and temporal information migration compensation technology based on DCGAN, dynamic target features are extracted through multi-layer compression perception and low-rank sparse decomposition, combined with joint classification error and label consistency regularization, a visual privacy protection video homomorphic monitoring of home health behavior is designed. Methods to ensure the fidelity and privacy of the target information.

Benefits of technology

It effectively blocks personal privacy without damaging the fidelity of video content, improves the accuracy and recognition rate of fall behavior recognition, avoids overfitting problems, and ensures the privacy and effectiveness of monitoring.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114998805B_ABST
    Figure CN114998805B_ABST
Patent Text Reader

Abstract

A visual privacy protection method for homomorphic monitoring of healthy behavior videos at home based on DCGAN temporal and spatial information migration compensation includes: using multi-layer compressed sensing to achieve visual shielding effects of video frames and extracting foreground frames by performing four-classification of data sets based on behavioral changes; using DCGAN technology to perform temporal and spatial information migration compensation to compensate for compressed images to retain their visual shielding effects and achieve the purpose of feature enhancement; extracting homomorphic BWLBP‑TOP features from the compensated images to quantitatively express the feature content after compensation; proposing a multi-classification recognition based on label consistent regularization joint classification to effectively perform four-classification recognition to achieve homomorphic monitoring of healthy behavior videos at home. The method of identifying changes in indoor human behavior based on visual shielding proposed by the present invention can ensure the safety of the elderly while taking into account privacy and preventing them from being leaked.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for homomorphic monitoring of home health behavior videos with visual privacy protection based on DCGAN spatiotemporal information migration compensation. Background Art

[0002] Due to the aging population and the increase in the number of empty-nest elderly people, research on health monitoring of the elderly has been a hot academic topic. Some survey reports also pointed out that the main reason for most injuries and hospitalizations of the elderly is falls. If accidental falls are not discovered in time, the elderly will suffer serious injuries due to lack of timely treatment. Moreover, the posture of the elderly falling also requires a certain degree of explanation. Falling while sitting and falling while walking are also different first aid measures, because the main injury location is different when falling in different postures. Therefore, research on finding an effective method of home health monitoring will help to respond to potential emergencies more quickly, especially for guardians of elderly people living alone.

[0003] Home health monitoring methods for the elderly can be roughly divided into three types: wearable device-based methods, environmental device-based methods, and vision-based methods. However, wearing wearable devices all the time will cause people a certain degree of discomfort; environmental device-based methods are costly and difficult to maintain, and are not suitable for ordinary families. With the increasing application of cameras in health monitoring systems, vision-based methods for identifying changes in human behavior have attracted widespread attention. This method only focuses on the frames captured by the camera and uses image processing and pattern recognition technology to detect behavioral changes. Not only is the equipment installation cost low and the interference level is low, but the camera can also record various daily activities at the same time. However, traditional home monitoring based on visually clear images brings the risk of privacy leakage. Especially for bathrooms and toilets, which are places where personal privacy is easily leaked, and are the places where accidents most often happen to the elderly. Therefore, designing a home health behavior video monitoring system that takes personal privacy into consideration has far-reaching significance and application value. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatiotemporal information migration compensation, studies a method for extracting human moving targets from visually shielded videos and then enhancing features, and then designs a multi-classification home human posture change recognition scheme based on joint classification error and label consistency regularization based on the extracted temporal texture features.

[0005] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0006] The present invention is a visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatiotemporal information migration compensation, comprising the following steps:

[0007] Step 1: Obtain a fall video dataset and perform multi-layer CS sampling on it to make the video content indistinguishable to the human eye. Then use low-rank sparse decomposition theory and similarity distance frame fusion to extract homomorphic moving targets.

[0008] Step 2: Based on the extracted foreground frame, the DCGAN method is used to compensate for the temporal and spatial information migration, and then the compensation is performed to filter out the noise points;

[0009] Step 3: Video homomorphic BWLBP-TOP feature extraction, which effectively expresses the one-dimensional temporal texture features of the moving target features of a video, retaining not only the spatial feature information but also the temporal feature information;

[0010] Step 4: Based on the joint classification error and label consistency regularization, the home health behavior video homomorphic monitoring is carried out to obtain the recognition results of four human behavior changes.

[0011] Preferably, step 1 further comprises:

[0012] Step 1.1: Select the falls dataset

[0013] Step 1.2: Perform multi-layer CS sampling encoding processing on the fall data set. As the number of dimensionality reduction layers increases, the image quality continues to decline, and the video content gradually reaches a shielding state; the CS3 layer is called the visual shielding layer or compression state;

[0014] Step 1.3: Apply low-rank sparse method to separate the foreground and background of both the original video and the compressed video, and then reduce the random noise generated by the compressed video by superimposing the foreground frames based on the similarity distance; fuse the video frames with high similarity to make the moving targets in the foreground image complete; and then extract the homomorphic moving targets.

[0015] Preferably, step 2 further comprises:

[0016] Step 2.1: Understand that the DCGAN model mainly achieves Nash equilibrium through competition between the generator and the discriminator; the whole process can be summarized as a binary minimax game:

[0017]

[0018] Where: V(D,G) is the cross entropy loss commonly used in binary classification tasks, P data (x) is the real data distribution, P z(z) is the noise distribution, D(x) indicates that x comes from real data, and G(z) indicates the generated sample of random noise after the generator. During the training process, the discriminator D aims to maximize the discrimination accuracy, and the generator G aims to minimize the maximum value of the discrimination accuracy of the discriminator D, that is,

[0019] Step 2.2: Based on DCGAN, a DCGAN network is designed to realize the compensation of spatiotemporal information migration. The input of the compensator is the compressed spatiotemporal features of the dynamic target, and the input of the migrator is the original spatiotemporal features of the dynamic target.

[0020] Step 2.2.1: Fix the compensator, train the migrator, input the real data of batchsize=30 into the migration network, perform back propagation, and optimize the network weights;

[0021] Step 2.2.2: Leave the migrator unchanged and use Reach the maximum, train the compensator to obtain the fake tensor, pass this tensor to the migration network for back propagation, and optimize the network weights;

[0022] Step 2.2.3: The migration condition is to calculate the label value corresponding to each image and the value obtained by Migrator and feed them back to Compensator and Migrator respectively, and then optimize the optimizer accordingly, and then output the image after Compensator feature compensation, repeat the first and second steps, and iterate multiple times until Compensator and Migrator reach Nash equilibrium;

[0023] Step 2.3: The obtained compensation frame is purified by superimposing it with the foreground frame through the low-rank sparse method to reduce the large number of noise points formed under the multi-layer convolution of the Migrator network.

[0024] Preferably, step 3 further comprises:

[0025] Step 3.1: In order to obtain the temporal texture features of the compensated frame based on different contributions of the blocks, the weight matrix of each small block needs to be calculated first:

[0026]

[0027] Where i and j are block indexes, and the values ​​are i and j = 1, 2, 3, ... N; H and W represent the height and width of each small block respectively; F represents the compensation frame;

[0028] Step 3.2: Through exercise intensity C ij Normalized to [0,1] to obtain the weight matrix M of the temporal texture feature ij :

[0029] M ij =C ij / MAX({C ij |i,j=1,2,3,...,N})

[0030] Among them, the function MAX represents the exercise intensity C ij The maximum value in the set of ;

[0031] Step 3.3: LBP-TOP encodes the appearance and motion of the image in three directions, that is, LBP is applied to the three planes of XY, XT, and YT, where the XY plane has rich appearance information, and the XT and YT planes contain important time domain information; the BWLBP-TOP feature descriptors of the three planes are extracted from each block volume and connected into a vector; therefore, the feature histogram formula of a block volume is described as:

[0032]

[0033] in Represent the BWLBP-TOP features of the three planes XY, XT, and YT respectively. The extracted BWLBP-TOP histogram formula is multiplied by the corresponding weights to form the weighted features of the four-classification recognition of the multi-layer compressed video, as shown in the following formula:

[0034] H′ i,j,t =M ij ×H i,j,t

[0035] H′ i,j,t is the time series weighted feature, M ij is the weight of the corresponding block, H i,j,t It is the BWLBP-TOP feature of the corresponding block volume.

[0036] Preferably, step 4 further comprises:

[0037] Step 4.1: The goal is to use the classification error as a parameter in the objective function of dictionary learning so that the dictionary becomes the optimal object for classification; here a linear prediction class definer f(x; W) = Wz is used; the objective function of learning a dictionary D with both reconstruction and discrimination capabilities can be defined as follows:

[0038]

[0039]

[0040] D is the learned overcomplete dictionary, A is a linear transformation matrix, W is the classifier parameter, X = [x1...x N ]∈R K×Nis the sparse coding of the input signal Y, T is a sparse constraint factor, α and β are scalars that control the relative contribution of the corresponding terms; H = [h1...h N ]∈R m×N is the class label of the input signal Y, h i =[0,0...1...0,0] t ∈R m is a signal corresponding to the input i A label vector where the non-zero position value represents y i The category label of Q = [q1...q N ]∈R k×N ] is the discriminative sparse code of the input signal Y for classification, if q i A non-zero value appears in the input signal y i and dictionary item d k Sparse codes that share the same label, is the same as the input signal y i corresponding;

[0041] Assume discriminative sparse coding X' = AX and A∈R K×K is reversible, then D'=DA -1 ,W'=WT -1 The objective function in the above formula can be rewritten as

[0042]

[0043]

[0044] The first term represents the reconstruction error, the second term represents the discriminative sparse coding error, and the third term represents the classification error;

[0045] Step 4.2: The one-dimensional BWLBP-TOP features H' of the four categories obtained in step 3 i,j,t According to the principle of five crossovers, the training set and corresponding parameters are selected and input into the LC-KSVD2 formula obtained in the previous step, and D is the learned overcomplete dictionary and W is the classifier parameter. Then the test set and parameters such as D and W are iteratively calculated according to the label consistency principle to obtain a recognition rate corresponding to each test set. Finally, the confusion matrix is ​​obtained according to the recognition label to analyze the relationship between the four categories, and the recognition rate of the corresponding classification is analyzed to improve compared with the existing method and avoid the problem of overfitting.

[0046] The beneficial effects of the present invention are:

[0047] 1. The present invention uses the WELL algorithm to obtain pseudo-random numbers as original signals and combines them with Gaussian distribution to generate a chaotic pseudo-random Gaussian measurement matrix combined with a multi-layer compressed sensing model. The advantage of the former is that a lot of numbers can be generated in a short time, and the latter is based on the former and realizes image encryption and non-reconstruction.

[0048] 2. In order to solve the problem of whether the compressed state and the original state are isomorphic, the present invention compares the proportion of moving target information based on the foreground frames of the two, verifies that the two are isomorphic, and proves that the subsequent same operation based on the two meets the mapping conditions.

[0049] 3. The present invention is aimed at the DCGAN spatiotemporal information migration compensation algorithm. Parameters are selected according to the number of video frames to ensure the effective retention of timing information, and the compensator and migrator are continuously trained and optimized. The migration conditions are the label values ​​corresponding to four different behavior change classifications. Feeding them back to the optimizer can enhance the spatial information of the four classification conditions.

[0050] 4. The present invention is based on the joint classification error and label consistency regularization as a classifier. Different from the original KSVD method, the present invention adds the influence of a joint classification error and label consistency regularization term in the objective function to learn a dictionary with more balanced reconstruction and discrimination capabilities, thereby improving the recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Schematic diagram of the method flow in an embodiment of the present invention.

[0052] Figure 2 Schematic diagram of extracting moving targets from original and compressed video in an embodiment of the present invention.

[0053] Figure 3 Schematic diagram of the ratio of information amount of foreground frames of original and compressed videos in an embodiment of the present invention.

[0054] Figure 4 Schematic diagram of the network architecture of DCGAN spatiotemporal information migration compensation in an embodiment of the present invention.

[0055] Figure 5 Schematic diagram of feature similarity comparison in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0057] The present invention provides a visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatio-temporal information migration compensation. This method uses a multi-layer compressive sensing method to ensure high-fidelity motion information while achieving visual privacy protection. After extracting moving targets, spatio-temporal information migration compensation and purification are performed on the images based on the DCGAN algorithm under the condition of not destroying privacy protection. Then, one-dimensional spatio-temporal texture features are extracted and four-class classification recognition is performed using joint classification error and label consistency regularization.

[0058] As Figure 1 shown, the specific steps of a visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatio-temporal information migration compensation in the present invention are as follows:

[0059] Step 1: Obtain a fall video dataset, perform multi-layer compressive sensing on the video to obtain a compressed video, and simultaneously extract moving targets from the compressed state and the original state to obtain foreground frames. Calculate the proportion of foreground information in the foreground frames to prove that the compressed state and the original state are homomorphic.

[0060] Step 1.1: In this embodiment, the commonly used fall behavior datasets URFD and FDD are selected. Among them, URFD contains 30 fall videos and 30 daily activity videos. All fall events and daily activities are obtained from an RGB camera with a resolution of 640×480. FDD contains 128 fall videos and 128 daily activity videos with a resolution of 320×240. Since each single video in the two datasets contains multiple behaviors, such as walking, sitting, falling, etc., the present invention performs segmentation processing on each video, dividing each video into short video segments composed of a finite number of video frames, so that each short video only contains one type of behavior change, such as sit-walk, sit-fall, walk, walk-fall. After the segmentation processing, each video is 30 frames.

[0061] Step 1.2: After obtaining pseudo-random numbers through the WELL algorithm, combine them with the Gaussian distribution as the original signal to generate a chaotic pseudo-random Gaussian measurement matrix. It is similar to the Mersenne Twister algorithm in terms of function, and the generation process can be recursively defined as follows:

[0062] φ n+1 =(a×(φ n )+c)mod m

[0063] where φ is the pseudo-random number sequence, m is the modulus (0 < m), a is the multiplier (0 < a < m), c is the increment (0 < c < m), and φ0 is the seed (0 < φ0 < m). The WELL algorithm can generate a lot of numbers in a short time, which is beneficial for applications that require a lot of numbers.

[0064] The image X of size M×N is divided into n image blocks x containing 2×2 pixel values, that is, X=[x1,x2....,x n ], and at the same time the observation matrix of the same size The same treatment is performed, that is, Then the blocks at the same position are operated with inner products, and the same observations are made for each image block:

[0065]

[0066] In order to enhance the security of privacy protection, the single-layer compression is expanded to multi-layer compression, which is implemented as follows:

[0067]

[0068] In the four categories (A) sit-walk, (B) sit-fall, (C walk, (D walk-fall), multi-layer compression Figure 2 As shown, the number of compression layers are 0, 1, 2, and 3 respectively.

[0069] Step 1.3: Apply low-rank sparse method to separate the foreground and background of both the original video and the compressed video, and then reduce the random noise generated by the compressed video by superimposing the foreground frames based on the similarity distance; fuse the video frames with high similarity to make the moving targets in the foreground image complete; and then perform homomorphic moving target extraction.

[0070] In order to prove that the compressed video does not lose effective information compared to the original video, an experiment on the proportion of foreground information was conducted on all videos, thus proving that the compressed state and the original state are the same. First, the grayscale image of the foreground frame is converted into a binary image. After the image is binarized, the moving target is the part with a smaller grayscale value and is classified as 0, while the unchanged background is the part with a larger grayscale value and is classified as 1. Then, according to H(X)=Q(X) / S(X), the information content of the 0 value is calculated, where Q(X) is the number of 0 values ​​in the image, and S(X) is the size of the image. For example, the original video size of the FDD dataset is 240*320, and the corresponding visual shielding layer video size is 30*40; the H(X) calculated in this way is the amount of information of each image. Figure 3 As shown, the dark solid line of the undulating curve is the proportion of foreground information corresponding to the original video, and the light solid line of the undulating curve is the proportion of foreground information corresponding to the compressed video. This embodiment can prove that the foreground frame in the compressed state and the foreground frame in the original state are the same information.

[0071] Step 2: Based on the homomorphic characteristics, both perform DCGAN spatiotemporal information migration compensation simultaneously. The original foreground frame is input into the migration network Migrator, and the compressed foreground frame is input into the compensation network Compensator to obtain a compensated frame with a size of 240×320.

[0072] Step 2.1: Understand that the DCGAN model mainly achieves Nash equilibrium through competition between the generator and the discriminator; the whole process can be summarized as a binary minimax game:

[0073]

[0074] Where: V(D,G) is the cross entropy loss commonly used in binary classification tasks, P data (x) is the real data distribution, P z (z) is the noise distribution, D(x) indicates that x comes from real data, and G(z) indicates the generated sample of random noise after the generator. During the training process, the discriminator D aims to maximize the discrimination accuracy, and the generator G aims to minimize the maximum value of the discrimination accuracy of the discriminator D, that is,

[0075] Step 2.2: Based on DCGAN, a DCGAN network is designed to realize the compensation of spatiotemporal information migration. The input of the compensator is the compressed spatiotemporal features of the dynamic target, and the input of the migrator is the original spatiotemporal features of the dynamic target.

[0076] Step 2.2.1: Fix the compensator and train the migrator to distribute P from the real data data (x), sampling x from the prior distribution P z Sample z in (z), generate data x through the compensator, and input x and z into the migrator for training so that the objective function V is maximized.

[0077]

[0078] For the training of the migration network, we first input the real data of batchsize=30 into the migration network, perform back propagation, and optimize the network weights. Because each video is divided into 30 frames, when the batchsize is 30, each foreground frame will not be turned into an independent image like when the batchsize is 1, but 30 frames will be put into a group for training, retaining the timing information before and after, so the time information is retained.

[0079] Step 2.2.2: Leave the migrator unchanged and use Reach the maximum, train the compensator to obtain the fake tensor, pass this tensor to the migration network for back propagation, and optimize the network weights.

[0080] Step 2.2.3: The migration condition is to calculate the label value corresponding to each image and the value obtained by Migrator and feed it back to Compensator and Migrator respectively, and then optimize the optimizer accordingly, and then output the image after Compensator feature compensation, repeat the first and second steps, and iterate multiple times until Compensator and Migrator reach Nash equilibrium. The overall structure of the above three steps is as follows Figure 4 shown.

[0081] refer to Figure 4 The designed DCGAN network includes a compensator and a migrator. The compensator includes two units consisting of a deconvolution layer, a BN layer, a ReLU layer, a convolution layer, and a Tanh layer in sequence; the migrator includes six units consisting of a convolution layer, a BN layer, a Leaky ReLU layer, and a Sigmoid layer in sequence.

[0082] Step 2.3: Since the Migrator network has many layers, although the features are enhanced after multiple convolutions, a large number of noise points are also formed in the non-moving target area. Therefore, it is necessary to purify the obtained compensation frame by superimposing it with the foreground frame through the low-rank sparse method to reduce the noise points.

[0083] Step 3: Homomorphic BWLBP-TOP feature extraction. Considering that some regions contribute more to the behavior recognition results than other regions, this block division method can weight each region according to its contribution, that is, give a higher weight to the region with significant movement, making the fall detection result more accurate. Therefore, the equivalent LBP-TOP mode of the block is used as the spatiotemporal feature of the CS state video.

[0084] Step 3.1: In order to obtain the temporal texture features of the compensated frame based on different contributions of the blocks, the weight matrix of each small block is first calculated:

[0085]

[0086] Among them, i, j are block indexes, and the values ​​are i, j = 1, 2, 3, ... N; H and W represent the height and width of each small block respectively; F represents the compensation frame.

[0087] Step 3.2: Through exercise intensity C ij Normalized to [0,1] to obtain the weight matrix M of the temporal texture feature ij :

[0088] M ij =C ij / MAX({C ij|i,j=1,2,3,...,N})

[0089] Among them, the function MAX represents the exercise intensity C ij The maximum value in the collection.

[0090] Step 3.3: LBP-TOP encodes the appearance and motion of the image in three directions, that is, LBP is applied to the three planes XY, XT, and YT. The XY plane has rich appearance information, and the XT and YT planes contain important time domain information. The BWLBP-TOP feature descriptors of the three planes are extracted from each block volume and connected into a vector; therefore, the feature histogram formula of a block volume is described as:

[0091]

[0092] in Represent the BWLBP-TOP features of the three planes XY, XT, and YT respectively. The extracted BWLBP-TOP histogram formula is multiplied by the corresponding weights to form the weighted features of the four-classification recognition of the multi-layer compressed video, as shown in the following formula:

[0093] H′ i,j,t =M ij ×H i,j,t

[0094] H′ i,j,t is the time series weighted feature, M ij is the weight of the corresponding block, H i,j,t It is the BWLBP-TOP feature of the corresponding block volume.

[0095] The present invention uses the compensated frame obtained by DCGAN to extract the homomorphic BWLBP-TOP one-dimensional temporal texture features, and compares the similarity with the BWLBP-TOP features that have not undergone DCGAN compensation and purifying steps. That is, the compensated BWLBP-TOP features are respectively compared with the original BWLBP-TOP features and the compressed BWLBP-TOP features based on the R=corrcoef(A,B) function for similarity, and the correlation coefficient matrix between the two random variables A and B is returned. The correlation coefficient of two random variables is used to measure their linear correlation. The Pearson correlation coefficient is defined as:

[0096]

[0097] where μ A and σ A are the mean and standard deviation of A, μ B and σ B are the mean and standard deviation of B. The correlation matrix of two random variables is the matrix of correlation coefficients for every pair of combinations of the variables.

[0098]

[0099] Since A and B are always directly related to themselves, the elements on the diagonal are 1, that is,

[0100]

[0101] The larger the absolute value of the correlation coefficient, the stronger the correlation; the closer the correlation coefficient is to 1 or -1, the stronger the correlation; the closer the correlation coefficient is to 0, the weaker the correlation. Figure 5 , where the dark line is the similarity between the BWLBP-TOP feature of DCGAN and the original BWLBP-TOP feature, and the light line is the similarity between the BWLBP-TOP feature of DCGAN and the compressed BWLBP-TOP feature. It can be seen from the figure that the image after DCGAN feature compensation retains the temporal texture characteristics of the original state and the compressed state, and the similarity between the BWLBP-TOP features of the compensated state and the compressed state is higher, indicating that the image after DCGAN spatiotemporal information migration compensation also retains its privacy protection ability and does not expose too much information due to compensation.

[0102] Step 4: Homomorphic monitoring of home health behavior videos based on joint classification error and label consistency regularization. After obtaining the one-dimensional BWLBP-TOP feature in the previous step, the LC-KSVD2 method can be used for multi-classification recognition. The five-way crossover method is used for classification, and four sets of features are taken as training sets and one set of features as test sets. 20 cycles are performed, and then the maximum value of these 100 cycles is taken.

[0103] Step 4.1: The goal is to use the classification error as a parameter in the objective function of dictionary learning so that the dictionary becomes the optimal object for classification. Here, a linear prediction class definer f(x; W) = Wx is used. The objective function of learning a dictionary D with both reconstruction and discrimination capabilities can be defined as follows:

[0104]

[0105]

[0106] D is the learned overcomplete dictionary, A is a linear transformation matrix, W is the classifier parameter, X = [x1...x N ]∈R K×N is the sparse coding of the input signal Y, T is a sparse constraint factor, α and β are scalars that control the relative contribution of the corresponding terms; H = [h1...h N ]∈R m×N is the class label of the input signal Y, h i=[0,0...1...0,0] t ∈R m is a signal corresponding to the input i A label vector where the non-zero position value represents y i The category label of Q = [q1...q N ]∈R k×N ] is the discriminative sparse code of the input signal Y for classification, if q i A non-zero value appears in the input signal y i and dictionary item d k Sparse codes that share the same label, is the same as the input signal y i Corresponding.

[0107] Assume discriminative sparse coding X' = AX and A∈R K×K is reversible, then D'=DA -1 ,W'=WT -1 The objective function in the above formula can be rewritten as

[0108]

[0109]

[0110] The first term represents the reconstruction error, the second term represents the discriminative sparse coding error, and the third term represents the classification error.

[0111] Step 4.2: The one-dimensional BWLBP-TOP features H' of the four categories obtained in step 3 i,j,t According to the principle of five crossovers, the training set and corresponding parameters are selected and input into the LC-KSVD2 formula obtained in the previous step, and D is the learned overcomplete dictionary and W is the classifier parameter. Then the test set and parameters such as D and W are iteratively calculated according to the label consistency principle to obtain a recognition rate corresponding to each test set. Finally, the confusion matrix is ​​obtained according to the recognition label to analyze the relationship between the four categories, and the recognition rate of the corresponding classification is analyzed to improve compared with the existing method and avoid the problem of overfitting.

[0112] The following analysis and application guidance can be obtained from the confusion matrix recognition rate results: For the four behavioral changes: sit-walk, sit-fall, walk, walk-fall, there is a certain probability of confusion between the two behavioral changes of sit-walk and walk, because the postures have relatively overlapping trajectories, and sit-fall and walk-fall are very different from other behaviors, so the recognition accuracy of these two items is relatively high. After reclassifying the fall detection dataset according to the four categories, the recognition rate of the existing "Research on Fall Behavior Detection in Indoor Surveillance Videos under Visual Privacy Protection" method is compared with that of the present invention. It can be obtained that under the FDD dataset, the highest recognition rate of the existing method is 81.50%, and the highest recognition rate of the present invention is 88.24%; under the URFD dataset, the highest recognition rate of the existing method is 82.61%, and the highest recognition rate of the present invention is 92.31%; under the mixed dataset, the highest recognition rate of the existing method is 75.21%, and the highest recognition rate of the present invention is 85.71%.

[0113] The DCGAN spatiotemporal information transfer compensation network is subjected to ablation experiments to determine the contribution of the network to quantitative relationships such as recognition rate. All experimental settings remain unchanged except for the DCGAN spatiotemporal information transfer compensation network. The compressed BWLBP-TOP features and compensated BWLBP-TOP features are extracted and input into the LC-KSVD2 classifier respectively to compare their recognition results.

[0114] Taking 316 video data of a mixed data set as an example, 80% of the data set was randomly selected as a training set and 20% of the data set was randomly selected as a test set. The five-number crossover algorithm was used for 20 cycles. Finally, the recognition rate of the compensated BWLBP-TOP feature was 85.71%, while the recognition rate of the compressed BWLBP-TOP feature was 82.81%. This proves that the DCGAN spatiotemporal information migration compensation network does play a role in feature compensation in quantitative relationships such as recognition rate, and its recognition rate is higher than that of the non-compensated network in four-category recognition.

[0115] The visual privacy protection home health behavior video homomorphic monitoring obtained by the present invention can not only perform four-category recognition well under the condition of visual privacy protection, but also avoid the problem of overfitting, and the recognition result has strong reliability.

[0116] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A visual privacy-preserving home health behavior video homomorphic monitoring method based on DCGAN spatiotemporal information migration compensation, Features: The monitoring method comprises the following steps: Step 1: Obtain a fall video dataset and perform multi-layer CS sampling on it to make the video content indistinguishable to the human eye. Then use low-rank sparse decomposition theory and similarity distance frame fusion to extract homomorphic moving targets and obtain foreground frames. Step 2: Based on the extracted foreground frame, the DCGAN method is used to compensate for the temporal and spatial information migration, and then the compensation is performed to filter out the noise points; Step 3: Extract the video homomorphic BWLBP-TOP features to obtain the one-dimensional temporal texture features, i.e., temporal weighted features, that express the moving target features of a video, while retaining the spatial and temporal feature information; The video homomorphic BWLBP-TOP feature extraction in step 3 specifically includes the following steps: Step 3.1: In order to obtain the temporal texture features of the compensated frame based on different block contributions, it is necessary to first calculate the weight matrix of each small block; similar to the block-based feature extraction method, the compensated frame is divided into non-overlapping W×H small blocks; then, the motion intensity (contribution) C of each small block ij It is obtained by adding the motion prospects of this area and its neighboring areas over time; Where i and j are block indexes, and the values ​​are i and j = 1, 2, 3, ... N; H and W represent the height and width of each small block respectively; F represents the compensation frame; Step 3.2: Through exercise intensity C ij Normalized to [0,1] to obtain the weight matrix M of the temporal texture feature ij : M ij =C ij / MAX({C ij |i,j=1,2,3,…,N}) Among them, the function MAX represents the exercise intensity C ij The maximum value in the set of ; Step 3.3: LBP-TOP encodes the appearance and motion of the image in three directions, that is, LBP is applied to the three planes of XY, XT, and YT, where the XY plane has rich appearance information, and the XT and YT planes contain important time domain information; the BWLBP-TOP feature descriptors of the three planes are extracted from each block volume and connected into a vector; therefore, the feature histogram formula of a block volume is described as: in Represent the BWLBP-TOP features of the three planes XY, XT, and YT respectively. The extracted BWLBP-TOP histogram formula is multiplied by the corresponding weights to form the weighted features of the four-classification recognition of the multi-layer compressed video, as shown in the following formula: H′ i,j,t =M ij ×H i,j,t H' i,j,t is the time series weighted feature, M ij is the weight of the corresponding block, H i,j,t is the BWLBP-TOP feature of the corresponding block volume; Step 4: Combine the obtained BWLBP-TOP features, regularize LC-KSVD2 based on classification error and label consistency, realize home health behavior video homomorphic monitoring, and identify changes in human behavior; The home health behavior video homomorphic monitoring based on joint classification error and label consistency regularization in step 4 includes the following steps: Step 4.1: The goal is to use the classification error as a parameter in the objective function of dictionary learning so that the dictionary becomes the optimal object for classification; use the linear prediction class definer f(x; W) = Wx; the objective function of learning a dictionary D with both reconstruction and discrimination capabilities is defined as follows: D is the learned overcomplete dictionary, A is a linear transformation matrix, W is the classifier parameter, X = [x 1 ...x N ]∈R K×N is the sparse coding of the input signal Y, T is a sparse constraint factor, α and β are scalars that control the relative contribution of the corresponding terms; H = [h 1 ...h N ]∈R m×N is the class label of the input signal Y, h i =[0,0...1...0,0] t ∈R m is a signal corresponding to the input i A label vector where the non-zero position value represents y i The category label of 1 …q N ]∈R k×N ] is the discriminative sparse code of the input signal Y for classification, if q i A non-zero value appears in the input signal y i and dictionary item d k Sparse codes that share the same label, is the same as the input signal y i corresponding; Assume discriminative sparse coding X' = AX and A∈R K×K is reversible, then D'=DA -1 ,W'=WT -1 ; The objective function in the above formula is rewritten as: The first term represents the reconstruction error, the second term represents the discriminative sparse coding error, and the third term represents the classification error; Step 4.2: The one-dimensional BWLBP-TOP features H' of the four categories obtained in step 3 i,j,t According to the principle of five crossovers, the training set and corresponding parameters are selected and input into the LC-KSVD2 formula obtained in the previous step, where D is the learned overcomplete dictionary and W is the classifier parameter. Then the test set and D and W parameters are iteratively calculated according to the label consistency principle to obtain a recognition rate corresponding to each test set.

2. According to claim 1, the visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatiotemporal information migration compensation, Features: Step 1 Extracting homomorphic moving targets includes the following steps: Step 1.1: Select the falls dataset Step 1.2: Perform multi-layer CS sampling encoding processing on the fall data set. As the number of dimensionality reduction layers increases, the image quality continues to decline, and the video content gradually reaches a shielding state; the CS3 layer is called the visual shielding layer or compression state; Step 1.3: Apply low-rank sparse method to separate the foreground and background of both the original video and the compressed video, and then reduce the random noise generated by the compressed video by superimposing the foreground frames based on the similarity distance; fuse the video frames with high similarity to make the moving targets in the foreground image complete; and then extract the homomorphic moving targets.

3. According to claim 1, the visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatiotemporal information migration compensation, Features: The DCGAN spatiotemporal information migration compensation process in step 2 specifically includes the following steps: Step 2.1: The process of the DCGAN model is summarized as a binary minimax game: Where: V(D,G) is the cross entropy loss commonly used in binary classification tasks, P data (x) is the real data distribution, P z (z) is the noise distribution, D(x) indicates that x comes from real data, and G(z) indicates the generated sample of random noise after the generator. During the training process, the discriminator D aims to maximize the discrimination accuracy, and the generator G aims to minimize the maximum value of the discrimination accuracy of the discriminator D, that is, Step 2.2: Design a DCGAN network to realize the compensation of spatiotemporal information migration. The input of the compensator is the compressed spatiotemporal features of the target, and the input of the migrator is the original spatiotemporal features of the target. After multiple iterations, the output results are: Step 2.3: The obtained compensation frame is purified by superimposing it with the foreground frame through a low-rank sparse method to reduce noise points.

4. According to claim 3, the visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatiotemporal information migration compensation, Features: In step 2.2, the designed DCGAN network includes a compensator and a migrator; The compensator includes two units consisting of a deconvolution layer, a BN layer, a ReLU layer, a convolution layer, and a Tanh layer in sequence; The migrator includes six units consisting of a convolutional layer, a BN layer, a LeakyReLU layer, and a Sigmoid layer.

5. According to claim 3, the visual privacy protection home health behavior video homomorphic monitoring method based on DCGAN spatiotemporal information migration compensation, Features: Step 2.2 includes the following steps: Step 2.2.1: Fix the compensator, train the migrator, input the real data of batchsize=30 into the migration network, perform back propagation, and optimize the network weights; Step 2.2.2: Leave the migrator unchanged and use Reach the maximum, train the compensator to obtain the fake tensor, pass this tensor to the migration network for back propagation, and optimize the network weights; Step 2.2.3: The migration condition is to calculate the label value corresponding to each image and the value obtained by Migrator and feed it back to Compensator and Migrator respectively, and then optimize the optimizer accordingly, and then output the image after Compensator feature compensation, repeat the first and second steps, and iterate multiple times until Compensator and Migrator reach Nash equilibrium.