Intelligent anti-peeping electronic device display method, system and electronic device

Abstract

The present application discloses an intelligent anti-peeping electronic device display method, system and electronic device. It combines gesture recognition based on the whole image and intention recognition based on the facial image, and effectively fuses the two features in the high-dimensional feature space mapped by the convolutional neural network through the Gaussian density map, and further increases the features through discretization so as not to cause information loss during fusion, and adopts the Gaussian mixture model to achieve response homogenization between the Gaussian distributions through weighting coefficients, thereby achieving robustness to the feature scale and effectively eliminating the difference between the feature scales. In this way, it is possible to accurately identify whether the person to be monitored has the intention of peeping, so as to effectively protect the screen content when it is detected that someone has the intention of peeping.

Description

Technical Field

[0001] The present invention relates to the field of intelligent anti-peeping, and more specifically, to an intelligent anti-peeping electronic device display method, system and electronic device. Background Art

[0002] With the development of information science and technology, the Internet and various electronic products of network terminals have become inseparable from people's daily life. At present, the information display of electronic products is mostly conveyed to users through screens, which can easily lead to the user's information being peeped and leaked, especially when certain confidentiality is very strong and personal private information is to be processed.

[0003] There are two main methods to prevent peeping on computer screens: active peeping and passive peeping. Passive peeping is to use a special screen film to prevent the screen content from being seen at a certain angle. Active peeping detects whether someone has reached the visible range, and if someone has, the computer window will be actively hidden or the screen saver mode will be directly entered to achieve the purpose of preventing peeping. Comparing the above two methods, active peeping is obviously more secure and suitable for some occasions with relatively high confidentiality requirements.

[0004] However, most active anti-peeping methods put the screen into protection mode once it detects that someone other than the user is peeping at the contents of the screen. However, when the screen is in protection mode, the user cannot see the contents of the screen. Any security system has a contradiction between sensitivity and accuracy. That is, if the anti-peeping detection is too sensitive, for example, once someone is detected, it will automatically switch to screen protection mode, which will affect the normal use of the user. However, if the anti-peeping detection is too insensitive, it will cause content leakage. Therefore, before confirming whether to turn on the screen protection, it is very important to accurately analyze and judge whether the user has the intention to peek.

[0005] Therefore, a smart anti-peeping electronic device display solution is expected to accurately identify whether the person to be monitored has the intention of peeping, so as to achieve the unity of anti-peeping in terms of sensitivity and accuracy. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide an intelligent anti-voyeurism electronic device display method, system and electronic device, which combines gesture recognition based on the whole image and intention recognition based on the facial image, and effectively fuses the two features in the high-dimensional feature space under the convolutional neural network mapping through the Gaussian density map, and further enhances the discretization of the features so as not to cause information loss during fusion, and adopts the Gaussian mixture model to achieve response homogenization between the various Gaussian distributions through weighting coefficients, thereby achieving robustness to the feature scale, and can effectively eliminate the differences between the feature scales. In this way, it is possible to accurately identify whether the person to be monitored has the intention of peeping, so as to effectively protect the screen content when it is detected that someone has the intention of peeping.

[0007] According to one aspect of the present application, there is provided an intelligent anti-peeping electronic device display method, which includes:

[0008] Acquire full-body images of the person to be monitored at a series of time points within a predetermined time;

[0009] Inputting the whole body images of the person to be monitored at the series of time points into a first convolutional neural network as a filter to obtain a first feature vector;

[0010] Acquire facial images of the person to be monitored at the series of time points;

[0011] Inputting the facial images of the person to be monitored at the series of time points into a second convolutional neural network as a filter to obtain a second feature vector;

[0012] The first eigenvector and the second eigenvector are fused based on a Gaussian density map to obtain a Gaussian distribution vector, each position of the Gaussian distribution vector is a Gaussian distribution, wherein the fusion based on the Gaussian density map represents constructing the Gaussian density map using a mean vector constructed by the means of the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean vector of the Gaussian density map and a covariance matrix constructed by the variances between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map;

[0013] Calculating a Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector, wherein the weight of each position of the weighted vector is 1;

[0014] Gaussian discretization is performed on the Gaussian distribution of each position of the Gaussian mixture model to obtain a feature matrix;

[0015] Passing the feature matrix through a classifier to obtain a classification result indicating whether the person to be monitored has a peeping intention; and

[0016] In response to the classification result that the person to be monitored has a peeping intention, a screen saver mode is activated.

[0017] In the above-mentioned intelligent anti-voyeurism electronic device display method, the whole body images of the person to be monitored at the series of time points are input into the first convolutional neural network as a filter to obtain a first feature vector, including: each layer of the first convolutional neural network except the last layer spatially encodes the whole body images of the person to be monitored at the series of time points according to the following formula to obtain a first feature map;

[0018] The formula is:

[0019] f i =Sigmoid(N i ×f i-1 +B i )

[0020] Among them, f i-1 is the input of the first convolutional neural network in the i-th layer, f i is the output of the first convolutional neural network in the i-th layer, N i is the filter unit of the first convolutional neural network in the i-th layer, and B i is the bias matrix of the i-th layer of the first convolutional neural network, Sigmoid represents a nonlinear activation function; and the last layer of the first convolutional neural network performs global pooling processing on the first feature map along the height dimension and the width dimension to obtain the first feature vector.

[0021] In the above-mentioned intelligent anti-peeping electronic device display method, the facial images of the person to be monitored at the series of time points are input into the second convolutional neural network as a filter to obtain a second feature vector, including: each layer of the first convolutional neural network except the last layer spatially encodes the facial images of the person to be monitored at the series of time points according to the following formula to obtain a second feature map;

[0022] The formula is:

[0023] f i =Sigmoid(N i ×f i-1 +B i )

[0024] Among them, f i-1 is the input of the second convolutional neural network in the i-th layer, f i is the output of the second convolutional neural network in the i-th layer, N iis the filter unit of the second convolutional neural network in the i-th layer, and B i is the bias matrix of the i-th layer of the second convolutional neural network, Sigmoid represents a nonlinear activation function; and the last layer of the second convolutional neural network performs global pooling processing on the second feature map along the height dimension and the width dimension to obtain the second feature vector.

[0025] In the above-mentioned intelligent anti-peeping electronic device display method, the first feature vector and the second feature vector are fused based on the Gaussian density map to obtain a Gaussian distribution vector, including: converting the eigenvalues ​​of each position in the first feature vector and the second feature vector into a probability interval of 0 to 1 to obtain a first probabilistic feature vector and a second probabilistic feature vector; and, the first probabilistic feature vector and the second probabilistic feature vector are fused based on the Gaussian density map to obtain the Gaussian distribution vector according to the following formula, wherein the formula is:

[0026]

[0027] Among them, μ(f1, f2) represents calculating the mean between the eigenvalues ​​at each position of the first probabilistic feature vector and the second probabilistic feature vector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and ∑(f1, f2) represents calculating the variance between the eigenvalues ​​at each position between the first probabilistic feature vector and the second probabilistic feature vector to obtain the covariance matrix of the Gaussian feature map.

[0028] In the above-mentioned intelligent anti-peeping electronic device display method, fusing the first feature vector and the second feature vector based on a Gaussian density map to obtain a Gaussian distribution vector includes: fusing the first feature vector and the second feature vector based on a Gaussian density map to obtain the Gaussian distribution vector according to the following formula, wherein the formula is:

[0029]

[0030] Among them, μ(f1, f2) represents calculating the mean between the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and ∑(f1, f2) represents calculating the variance between the eigenvalues ​​at each position between the first eigenvector and the second eigenvector to obtain the covariance matrix of the Gaussian feature map.

[0031] In the above-mentioned intelligent anti-peeping electronic device display method, the Gaussian mixture model of the Gaussian distribution vector is calculated based on a weighted vector having the same length as the vector length of the Gaussian distribution vector, including: based on a weighted vector having the same length as the vector length of the Gaussian distribution vector, the Gaussian mixture model of the Gaussian distribution vector is calculated using the following formula;

[0032] Wherein, the formula is:

[0033]

[0034]

[0035] Among them, v(π i ) represents the weighted vector of length L, and ⊙ represents the point multiplication by position.

[0036] In the above-mentioned intelligent anti-peeping electronic device display method, the feature matrix is ​​passed through a classifier to obtain a classification result for indicating whether the person to be monitored has a peeping intention, including: using at least one fully connected layer of the classifier to fully connect the feature matrix to obtain a classification feature vector; inputting the classification feature vector into the Softmax classification function of the classifier to obtain a first probability that the classification feature vector belongs to the person to be monitored with a peeping intention and a second probability that the classification feature vector belongs to the person to be monitored without a peeping intention; and, based on a comparison of the first probability and the second probability, generating the classification result.

[0037] According to another aspect of the present application, there is provided an intelligent anti-peeping electronic device display system, comprising:

[0038] A full-body image acquisition unit, used to acquire full-body images of the person to be monitored at a series of time points within a predetermined time;

[0039] a first convolutional neural network processing unit, configured to input the whole-body images of the person to be monitored at the series of time points obtained by the whole-body image acquisition unit into a first convolutional neural network as a filter to obtain a first feature vector;

[0040] A facial image acquisition unit, used to acquire facial images of the person to be monitored at the series of time points;

[0041] a second convolutional neural network processing unit, configured to input the facial images of the person to be monitored at the series of time points obtained by the facial image acquisition unit into a second convolutional neural network as a filter to obtain a second feature vector;

[0042] a Gaussian density map fusion unit, configured to fuse the first eigenvector obtained by the first convolutional neural network processing unit and the second eigenvector obtained by the second convolutional neural network processing unit based on a Gaussian density map to obtain a Gaussian distribution vector, wherein each position of the Gaussian distribution vector is a Gaussian distribution, wherein the fusion based on the Gaussian density map represents constructing the Gaussian density map using a mean vector constructed by the mean of the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean vector of the Gaussian density map and a covariance matrix constructed by the variance between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map;

[0043] a Gaussian mixture model calculation unit, configured to calculate a Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector obtained by the Gaussian density map fusion unit, wherein the weight of each position of the weighted vector is 1;

[0044] A Gaussian discretization unit, used for performing Gaussian discretization on the Gaussian distribution of each position of the Gaussian mixture model obtained by the Gaussian mixture model calculation unit to obtain a feature matrix;

[0045] A classification unit, used for passing the feature matrix obtained by the Gaussian discretization unit through a classifier to obtain a classification result indicating whether the person to be monitored has a peeping intention; and

[0046] The response unit is used to start the screen protection mode in response to the classification result obtained by the classification unit that the monitored person has a peeping intention.

[0047] In the above-mentioned intelligent anti-voyeurism electronic device display system, the first convolutional neural network processing unit is further used to: each layer of the first convolutional neural network except the last layer spatially encodes the full-body image of the person to be monitored at the series of time points according to the following formula to obtain a first feature map; the formula is:

[0048] f i =Sigmoid(N i ×fi i-1 +B i )

[0049] Among them, f i-1 is the input of the first convolutional neural network in the i-th layer, f i is the output of the first convolutional neural network in the i-th layer, N i is the filter unit of the first convolutional neural network in the i-th layer, and B iis the bias matrix of the i-th layer of the first convolutional neural network, Sigmoid represents a nonlinear activation function; and the last layer of the first convolutional neural network performs global pooling processing on the first feature map along the height dimension and the width dimension to obtain the first feature vector.

[0050] In the above-mentioned intelligent anti-voyeurism electronic device display system, the second convolutional neural network processing unit is further used to: each layer of the first convolutional neural network except the last layer spatially encodes the facial images of the monitored person at the series of time points according to the following formula to obtain a second feature map; the formula is:

[0051] f i =Sigmoid(N i ×f i-1 +B i )

[0052] Among them, f i-1 is the input of the second convolutional neural network in the i-th layer, f i is the output of the second convolutional neural network in the i-th layer, N i is the filter unit of the second convolutional neural network in the i-th layer, and B i is the bias matrix of the i-th layer of the second convolutional neural network, Sigmoid represents a nonlinear activation function; and the last layer of the second convolutional neural network performs global pooling processing on the second feature map along the height dimension and the width dimension to obtain the second feature vector.

[0053] In the above-mentioned intelligent anti-peeping electronic device display system, the Gaussian density map fusion unit is further used to: convert the eigenvalues ​​of each position in the first feature vector and the second feature vector into a probability interval of 0 to 1 to obtain a first probabilistic feature vector and a second probabilistic feature vector; and, fuse the first probabilistic feature vector and the second probabilistic feature vector based on the Gaussian density map according to the following formula to obtain the Gaussian distribution vector, wherein the formula is:

[0054]

[0055] Among them, μ(f1, f2) represents calculating the mean between the eigenvalues ​​at each position of the first probabilistic feature vector and the second probabilistic feature vector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and ∑(f1, f2) represents calculating the variance between the eigenvalues ​​at each position between the first probabilistic feature vector and the second probabilistic feature vector to obtain the covariance matrix of the Gaussian feature map.

[0056] In the above-mentioned intelligent anti-peeping electronic device display system, the Gaussian density map fusion unit is further used to: fuse the first feature vector and the second feature vector based on the Gaussian density map according to the following formula to obtain the Gaussian distribution vector, wherein the formula is:

[0057]

[0058] Among them, μ(f1, f2) represents calculating the mean between the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and ∑(f1, f2) represents calculating the variance between the eigenvalues ​​at each position between the first eigenvector and the second eigenvector to obtain the covariance matrix of the Gaussian feature map.

[0059] In the above-mentioned intelligent anti-peeping electronic device display system, the Gaussian mixture model calculation unit is further used to: calculate the Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector using the following formula; wherein the formula is:

[0060]

[0061]

[0062] Among them, v(π i ) represents the weighted vector of length L, and ⊙ represents the point multiplication by position.

[0063] In the above-mentioned intelligent anti-peeping electronic device display system, the classification unit is further used to: use at least one fully connected layer of the classifier to fully connect the feature matrix to obtain a classification feature vector; input the classification feature vector into the Softmax classification function of the classifier to obtain a first probability that the classification feature vector belongs to the person to be monitored with a peeping intention and a second probability that the classification feature vector belongs to the person to be monitored without a peeping intention; and, based on the comparison of the first probability and the second probability, generate the classification result.

[0064] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the intelligent anti-peeping electronic device display method as described above.

[0065] According to yet another aspect of the present application, a computer-readable medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the intelligent anti-peeping electronic device display method as described above.

[0066] Compared with the prior art, the intelligent anti-peeping electronic device display method, system and electronic device provided by the present application combine gesture recognition based on the whole image and intention recognition based on the facial image, and effectively fuse the two features in the high-dimensional feature space mapped by the convolutional neural network through the Gaussian density map, and further enhance the features through discretization so as not to generate information loss during fusion, and use the Gaussian mixture model to achieve response homogenization between the Gaussian distributions through weighting coefficients, thereby achieving robustness to the feature scale and effectively eliminating the difference between the feature scales. In this way, it is possible to accurately identify whether the person to be monitored has the intention of peeping, so as to effectively protect the screen content when someone is detected to have the intention of peeping. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0068] Figure 1 This is an application scenario diagram of the intelligent anti-peeping electronic device display method according to an embodiment of the present application;

[0069] Figure 2 It is a flow chart of a smart anti-peeping electronic device display method according to an embodiment of the present application;

[0070] Figure 3 A schematic diagram of the system architecture of an intelligent anti-peeping electronic device display method according to an embodiment of the present application;

[0071] Figure 4 A block diagram of an intelligent anti-peeping electronic device display system according to an embodiment of the present application;

[0072] Figure 5 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0073] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0074] Scenario Overview

[0075] As mentioned above, there are two main methods to prevent peeping on computer screens: active peeping and passive peeping. Passive peeping is to use a special screen film to prevent the screen content from being seen at a certain angle. Active peeping detects whether someone has reached the visible range, and if someone has, the computer window is actively hidden or the screen saver mode is directly entered to achieve the purpose of preventing peeping. Comparing the above two methods, it is obvious that active peeping is more secure and is suitable for some occasions with relatively high confidentiality requirements.

[0076] However, most active anti-peeping methods are to put the screen into protection mode once it is detected that someone other than the user is peeping at the contents of the screen, but when entering the screen protection mode, the user cannot see the contents on the screen. Any security system has a contradiction between sensitivity and accuracy, that is, if the anti-peeping detection is too sensitive, for example, once someone is detected, it will automatically switch to the screen protection mode, which will affect the normal use of the user, but if the anti-peeping detection is too insensitive, it will cause content leakage. Therefore, in order to accurately identify whether the monitored person has the intention of peeping, so as to achieve the unity of sensitivity and accuracy, an intelligent anti-peeping electronic device display method is desired.

[0077] It should be understood that the sensitivity of anti-peeping detection should be related to the intention of the intruder. That is, if the intruder's intention is to peek at the contents of the screen, the sensitivity level should be increased, and if the intruder's intention is not to peek at the screen, but just to pass by the screen, the sensitivity level should be reduced.

[0078] Based on this, in the technical solution of the present application, first, a full-body image of the person to be monitored at a series of time points within a predetermined time is obtained, and input into a first convolutional neural network as a filter to obtain a first feature vector. At the same time, a facial image of the person to be monitored at the series of time points is obtained, and input into a second convolutional neural network as a filter to obtain a second feature vector.

[0079] Then, after converting both the first eigenvector and the second eigenvector to the interval [0,1], a fusion based on the Gaussian density map is performed, namely:

[0080]

[0081] That is, the mean between the eigenvalues ​​at each position of the first eigenvector and the second eigenvector is calculated as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and the variance between the eigenvalues ​​at each position between the first eigenvector and the second eigenvector is calculated to obtain the covariance matrix of the Gaussian feature map, where each position of the Gaussian distribution vector corresponds to a Gaussian distribution.

[0082] Then, the Gaussian mixture model based on the Gaussian distribution vector is calculated, that is:

[0083]

[0084]

[0085] That is, a weighted vector of length L is point-multiplied with the Gaussian density map, wherein the sum of the weights at each position of the weighted vector is one.

[0086] Then, the Gaussian distribution of each position of the Gaussian mixture model is Gaussian discretized to obtain a feature matrix, and the feature matrix is ​​input into the classifier to obtain the classification result.

[0087] It should be understood that, in this way, when the feature vectors are fused through the Gaussian density map, since Gaussian density is widely used as a learning target for methods based on convolutional neural networks, it realizes the effective fusion of features in the high-dimensional feature space under the convolutional neural network mapping, and does not generate information loss during fusion by discretizing and expanding the features. In addition, the Gaussian mixture model realizes the response homogenization between each Gaussian distribution through the weighting coefficient, thereby realizing the robustness to the feature scale, and can effectively eliminate the difference between the feature scales.

[0088] Based on this, the present application proposes an intelligent anti-voyeurism electronic device display method, which includes: obtaining full-body images of a person to be monitored at a series of time points within a predetermined time; inputting the full-body images of the person to be monitored at the series of time points into a first convolutional neural network as a filter to obtain a first feature vector; obtaining facial images of the person to be monitored at the series of time points; inputting the facial images of the person to be monitored at the series of time points into a second convolutional neural network as a filter to obtain a second feature vector; fusing the first feature vector and the second feature vector based on a Gaussian density map to obtain a Gaussian distribution vector, wherein each position of the Gaussian distribution vector is a Gaussian distribution, wherein the fusion based on the Gaussian density map represents the first feature vector and the second feature vector. The Gaussian density map is constructed by using a mean vector constructed by the means of the eigenvalues ​​at each position of the quantity as the mean vector of the Gaussian density map and a covariance matrix constructed by using the variances between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map; a Gaussian mixture model of the Gaussian distribution vector is calculated based on a weighted vector having the same length as the vector length of the Gaussian distribution vector, wherein the weight of each position of the weighted vector is 1; Gaussian discretization is performed on the Gaussian distribution at each position of the Gaussian mixture model to obtain a feature matrix; the feature matrix is ​​passed through a classifier to obtain a classification result for indicating whether the person to be monitored has a voyeuristic intention; and, in response to the classification result that the person to be monitored has a voyeuristic intention, a screen saver mode is started.

[0089] Figure 1 The figure shows an application scenario diagram of the intelligent anti-peeping electronic device display method according to an embodiment of the present application. Figure 1 As shown, in this application scenario, first, by setting it on an electronic device (for example, Figure 1 A camera on a T as shown in FIG. Figure 1 C) obtaining a series of time points within a predetermined time period for the person to be monitored (for example, Figure 1 The whole body image of the person to be monitored and the facial image of the person to be monitored at the series of time points are obtained, wherein the electronic device includes but is not limited to a computer, etc. Then, the obtained whole body image of the person to be monitored and the facial image of the person to be monitored are input into a server (for example, a server) that is equipped with an intelligent anti-peeping electronic device display algorithm. Figure 1The server S shown in the figure is capable of processing the full body image and the facial image of the person to be monitored by using an intelligent anti-peeping electronic device display algorithm to generate a classification result indicating whether the person to be monitored has a peeping intention. Furthermore, in response to the classification result that the person to be monitored has a peeping intention, the screen saver mode is activated to ensure the security of the content information.

[0090] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0091] Exemplary Methods

[0092] Figure 2 The figure shows a flow chart of an intelligent anti-peeping electronic device display method. Figure 2 As shown, according to the embodiment of the present application, the intelligent anti-peeping electronic device display method includes: S110, obtaining full-body images of a person to be monitored at a series of time points within a predetermined time; S120, inputting the full-body images of the person to be monitored at the series of time points into a first convolutional neural network as a filter to obtain a first feature vector; S130, obtaining facial images of the person to be monitored at the series of time points; S140, inputting the facial images of the person to be monitored at the series of time points into a second convolutional neural network as a filter to obtain a second feature vector; S150, fusing the first feature vector and the second feature vector based on a Gaussian density map to obtain a Gaussian distribution vector, wherein each position of the Gaussian distribution vector is a Gaussian distribution, wherein the fusion based on the Gaussian density map represents the first feature vector and the second feature vector. The Gaussian density map is constructed by using a mean vector constructed by the means of the eigenvalues ​​at each position of the eigenvector as the mean vector of the Gaussian density map and a covariance matrix constructed by using the variances between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map; S160, calculating a Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector, wherein the weight of each position of the weighted vector is 1; S170, performing Gaussian discretization on the Gaussian distribution at each position of the Gaussian mixture model to obtain a feature matrix; S180, passing the feature matrix through a classifier to obtain a classification result for indicating whether the person to be monitored has the intention of peeping; and, S190, in response to the classification result that the person to be monitored has the intention of peeping, starting the screen saver mode.

[0093] Figure 3 The figure shows a schematic diagram of the structure of the intelligent anti-peeping electronic device display method according to the embodiment of the present application. Figure 3As shown, in the network architecture of the intelligent anti-peeping electronic device display method, first, the full-body images of the person to be monitored at the series of time points (for example, Figure 3 The first convolutional neural network (e.g., Figure 3 ) to obtain the first feature vector (e.g., as Figure 3 Then, the facial images of the person to be monitored at the series of time points obtained (for example, Figure 3 Q) is input to a second convolutional neural network (e.g., as shown in Figure 3 ) to obtain the second feature vector (e.g., as Figure 3 Then, the first feature vector and the second feature vector are fused based on the Gaussian density map to obtain a Gaussian distribution vector (for example, Figure 3 V), each position of the Gaussian distribution vector is a Gaussian distribution, wherein the fusion based on the Gaussian density map represents a mean vector constructed by the mean of the eigenvalues ​​of each position of the first eigenvector and the second eigenvector as the mean vector of the Gaussian density map (for example, Figure 3 V1 as shown in FIG. 1 ) and a covariance matrix constructed with the variance between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map (for example, as Figure 3 M1 as shown in FIG. 2 ) to construct the Gaussian density map (e.g., Figure 3 F1 as shown in FIG); then, based on a weighted vector having the same length as the vector length of the Gaussian distribution vector (e.g., Figure 3 V2 as shown in FIG. 1 ), calculate the Gaussian mixture model of the Gaussian distribution vector (for example, as Figure 3 GM shown in FIG); then, the Gaussian distribution of each position of the Gaussian mixture model is Gaussian discretized to obtain a feature matrix (for example, Figure 3 MF as shown in ); then, the feature matrix is ​​passed through a classifier (e.g., Figure 3 The classifier shown in the figure) is used to obtain a classification result indicating whether the person to be monitored has a voyeuristic intention; and finally, in response to the classification result that the person to be monitored has a voyeuristic intention, the screen saver mode is started.

[0094] In step S110 and step S120, full-body images of the person to be monitored at a series of time points within a predetermined time are obtained; and the full-body images of the person to be monitored at the series of time points are input into the first convolutional neural network as a filter to obtain a first eigenvector. As mentioned above, it should be understood that the sensitivity of anti-peeping detection should be related to the intention of the intruder. That is, if the intention of the intruder is to peek at the contents of the screen, the sensitivity level should be increased, and if the intention of the intruder is not to peek at the screen, but to simply pass in front of the screen, the sensitivity level should be lowered. Therefore, in the technical solution of the present application, it is expected to be able to comprehensively and accurately judge whether the person to be monitored has the intention of peeping by combining gesture recognition based on the whole image and intention recognition based on the facial image, thereby ensuring the security of the information.

[0095] That is, in the technical solution of the present application, first, it is necessary to obtain full-body images of the person to be monitored at a series of time points within a predetermined time through a camera set on an electronic device, wherein the electronic device includes but is not limited to a computer, etc. Then, the full-body images of the person to be monitored at the series of time points are input into the first convolutional neural network as a filter for processing to extract the feature distribution representation of the local features of the full-body images of the person to be monitored in a high-dimensional space, thereby obtaining a first feature vector.

[0096] Specifically, in an embodiment of the present application, the whole-body images of the person to be monitored at the series of time points are input into the first convolutional neural network as a filter to obtain a first feature vector, comprising: firstly, each layer of the first convolutional neural network except the last layer spatially encodes the whole-body images of the person to be monitored at the series of time points according to the following formula to obtain a first feature map;

[0097] The formula is:

[0098] f i =Sigmoid(N i ×f i-1 +B i )

[0099] Among them, f i-1 is the input of the first convolutional neural network in the i-th layer, f i is the output of the first convolutional neural network in the i-th layer, N i is the filter unit of the first convolutional neural network in the i-th layer, and B iis the bias matrix of the first convolutional neural network of the i-th layer, and Sigmoid represents a nonlinear activation function. Then, the last layer of the first convolutional neural network performs global pooling processing on the first feature map along the height dimension and the width dimension to obtain the first feature vector. It should be understood that through global pooling processing, the number of parameters can be reduced, thereby reducing overfitting to improve the accuracy of classification.

[0100] In step S130 and step S140, facial images of the person to be monitored at the series of time points are obtained; and the facial images of the person to be monitored at the series of time points are input into the second convolutional neural network as a filter to obtain a second feature vector. It should be understood that in order to be able to comprehensively and accurately judge whether the person to be monitored has the intention of peeping by combining the posture recognition based on the whole image and the intention recognition based on the facial image. In the technical solution of the present application, it is also necessary to obtain the facial images of the person to be monitored at the series of time points through a camera set on an electronic device, wherein the electronic device includes but is not limited to a computer, etc.; further, the facial images of the person to be monitored at the series of time points are input into the second convolutional neural network as a filter for processing to extract the feature distribution representation of the local features of the facial images of the person to be monitored in the high-dimensional space, thereby obtaining the second feature vector.

[0101] Specifically, in an embodiment of the present application, the process of inputting the facial images of the person to be monitored at the series of time points into the second convolutional neural network as a filter to obtain the second feature vector includes: first, each layer of the first convolutional neural network except the last layer spatially encodes the facial images of the person to be monitored at the series of time points according to the following formula to obtain a second feature map;

[0102] The formula is:

[0103] f i =Sigmoid(N i ×f i-1 +B i )

[0104] Among them, f i-1 is the input of the second convolutional neural network in the i-th layer, f i is the output of the second convolutional neural network in the i-th layer, N i is the filter unit of the second convolutional neural network in the i-th layer, and B iis the bias matrix of the second convolutional neural network of the i-th layer, and Sigmoid represents a nonlinear activation function. Then, the last layer of the second convolutional neural network performs global pooling processing on the second feature map along the height dimension and the width dimension to obtain the second feature vector. It should be understood that through global pooling processing, the number of parameters can be reduced, thereby reducing overfitting to improve the accuracy of classification.

[0105] In step S150, the first feature vector and the second feature vector are fused based on a Gaussian density map to obtain a Gaussian distribution vector, each position of the Gaussian distribution vector is a Gaussian distribution, wherein the fusion based on the Gaussian density map means that the mean vector constructed by the mean between the eigenvalues ​​of each position of the first feature vector and the second feature vector is used as the mean vector of the Gaussian density map and the covariance matrix constructed by the variance between each position of the first feature vector and the second feature vector is used as the covariance matrix of the Gaussian density map to construct the Gaussian density map. It should be understood that in the process of combining the gesture recognition based on the whole image and the intention recognition based on the facial image, it is necessary to consider how to effectively fuse the features of the two in a high-dimensional feature space and eliminate the difference between the feature scales. In addition, in the technical solution of the present application, it is also considered that when the feature vectors are fused through the Gaussian density map, since the Gaussian density is widely used as a learning target of the method based on the convolutional neural network, it realizes the effective fusion of features in the high-dimensional feature space under the mapping of the convolutional neural network.

[0106] Therefore, in the technical solution of the present application, the first eigenvector and the second eigenvector are fused based on a Gaussian density map to obtain a Gaussian distribution vector, where each position of the Gaussian distribution vector is a Gaussian distribution. In a specific example, the fusion based on the Gaussian density map represents the construction of the Gaussian density map using a mean vector constructed by the mean between the eigenvalues ​​of each position of the first eigenvector and the second eigenvector as the mean vector of the Gaussian density map and a covariance matrix constructed by the variance between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map.

[0107] Specifically, in an embodiment of the present application, the process of fusing the first eigenvector and the second eigenvector based on a Gaussian density map to obtain a Gaussian distribution vector includes: first, converting the eigenvalues ​​of each position in the first eigenvector and the second eigenvector into a probability interval of 0 to 1 to obtain a first probabilistic eigenvector and a second probabilistic eigenvector. It should be understood that by normalizing the eigenvalues ​​of each position in the first eigenvector and the second eigenvector, the dimensional influence between the eigenvalues ​​can be eliminated, which is convenient for measurement and subsequent calculations. Then, the first probabilistic eigenvector and the second probabilistic eigenvector are fused based on a Gaussian density map to obtain the Gaussian distribution vector according to the following formula, wherein the formula is:

[0108]

[0109] Among them, μ(f1, f2) represents calculating the mean between the eigenvalues ​​at each position of the first probabilistic feature vector and the second probabilistic feature vector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and ∑(f1, f2) represents calculating the variance between the eigenvalues ​​at each position between the first probabilistic feature vector and the second probabilistic feature vector to obtain the covariance matrix of the Gaussian feature map.

[0110] In particular, in a specific example, the process of fusing the first feature vector and the second feature vector based on a Gaussian density map to obtain a Gaussian distribution vector further includes: fusing the first feature vector and the second feature vector based on a Gaussian density map to obtain the Gaussian distribution vector using the following formula, wherein the formula is:

[0111]

[0112] Among them, μ(f1, f2) represents calculating the mean between the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and ∑(f1, f2) represents calculating the variance between the eigenvalues ​​at each position between the first eigenvector and the second eigenvector to obtain the covariance matrix of the Gaussian feature map.

[0113] In step S160, based on a weighted vector having the same length as the vector length of the Gaussian distribution vector, a Gaussian mixture model of the Gaussian distribution vector is calculated, wherein the weight of each position of the weighted vector is 1. That is, in the technical solution of the present application, after obtaining the Gaussian distribution vector, a Gaussian mixture model based on the Gaussian distribution vector is calculated to eliminate the difference between the characteristic scales of the gesture recognition based on the whole image and the intention recognition based on the facial image, thereby achieving a better fusion effect. It should be understood that the Gaussian mixture model achieves response homogenization between the various Gaussian distributions through weighted coefficients, thereby achieving robustness to the characteristic scale, which can effectively eliminate the difference between the characteristic scales.

[0114] Specifically, in an embodiment of the present application, the process of calculating the Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector includes: calculating the Gaussian mixture model of the Gaussian distribution vector using the following formula based on a weighted vector having the same length as the vector length of the Gaussian distribution vector;

[0115] Wherein, the formula is:

[0116]

[0117]

[0118] Among them, v(π i ) represents the weighted vector of length L, and ⊙ represents point multiplication by position. That is, the weighted vector of length L is point multiplied with the Gaussian density map, wherein the sum of the weights of each position of the weighted vector is one.

[0119] In step S170, the Gaussian distribution of each position of the Gaussian mixture model is Gaussian discretized to obtain a feature matrix. It should be understood that by performing Gaussian discretization on the Gaussian distribution of each position of the Gaussian mixture model, no information loss will occur when the information of the two is fused, thereby making the subsequent classification effect better, so that the accuracy of judging whether the monitored person has the intention of peeping is higher.

[0120] In step S180 and step S190, the feature matrix is ​​passed through a classifier to obtain a classification result indicating whether the person to be monitored has a voyeuristic intention, and in response to the classification result that the person to be monitored has a voyeuristic intention, the screen saver mode is activated. Accordingly, in a specific example, first, the feature matrix is ​​fully connected encoded using at least one fully connected layer of the classifier to obtain a classification feature vector. Then, the classification feature vector is input into the Softmax classification function of the classifier to obtain a first probability that the classification feature vector belongs to the person to be monitored with a voyeuristic intention and a second probability that the classification feature vector belongs to the person to be monitored without a voyeuristic intention. Finally, based on the comparison of the first probability and the second probability, the classification result is generated. Specifically, when the first probability is greater than the second probability, the classification result is that the person to be monitored has a voyeuristic intention, and in response to the classification result that the person to be monitored has a voyeuristic intention, the screen saver mode is activated; when the first probability is less than the second probability, the classification result is that the person to be monitored has no voyeuristic intention.

[0121] In summary, the intelligent anti-peeping electronic device display method of the embodiment of the present application is explained, which combines the posture recognition based on the whole image and the intention recognition based on the facial image, and effectively fuses the two features in the high-dimensional feature space under the convolutional neural network mapping through the Gaussian density map, and further expands the features through discretization so as not to generate information loss during fusion, and adopts the Gaussian mixture model to achieve response homogenization between the various Gaussian distributions through weighting coefficients, thereby achieving robustness to the feature scale, and can effectively eliminate the differences between the feature scales. In this way, it is possible to accurately identify whether the person to be monitored has the intention of peeping, so as to effectively protect the screen content when it is detected that someone has the intention of peeping.

[0122] Exemplary Systems

[0123] Figure 4 The figure shows a block diagram of an intelligent anti-peeping electronic device display system according to an embodiment of the present application. Figure 4As shown, according to an embodiment of the present application, the intelligent anti-peeping electronic device display system 400 includes: a full-body image acquisition unit 410, which is used to acquire full-body images of a person to be monitored at a series of time points within a predetermined time; a first convolutional neural network processing unit 420, which is used to input the full-body images of the person to be monitored at the series of time points obtained by the full-body image acquisition unit 410 as a first convolutional neural network of a filter to obtain a first feature vector; a facial image acquisition unit 430, which is used to acquire the facial images of the person to be monitored at the series of time points; a second convolutional neural network processing unit 440, which is used to input the facial images of the person to be monitored at the series of time points obtained by the facial image acquisition unit 430 as a second convolutional neural network of a filter to obtain a second feature vector; a Gaussian density map fusion unit 450, which is used to fuse the first feature vector obtained by the first convolutional neural network processing unit 420 and the second feature vector obtained by the second convolutional neural network processing unit 440 based on a Gaussian density map to obtain a Gaussian distribution vector, wherein each position of the Gaussian distribution vector is a Gaussian distribution, wherein the Gaussian density map based on the Gaussian density map Fusion means constructing the Gaussian density map by using the mean vector constructed by the mean between the eigenvalues ​​of each position of the first eigenvector and the second eigenvector as the mean vector of the Gaussian density map and the covariance matrix constructed by using the variance between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map; a Gaussian mixture model calculation unit 460, which is used to calculate the Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector obtained by the Gaussian density map fusion unit 450, wherein, The weight of each position of the weighted vector is 1; a Gaussian discretization unit 470 is used to perform Gaussian discretization on the Gaussian distribution of each position of the Gaussian mixture model obtained by the Gaussian mixture model calculation unit 460 to obtain a feature matrix; a classification unit 480 is used to pass the feature matrix obtained by the Gaussian discretization unit 470 through a classifier to obtain a classification result indicating whether the person to be monitored has the intention of peeping; and a response unit 490 is used to start the screen saver mode in response to the classification result obtained by the classification unit 480 that the person to be monitored has the intention of peeping.

[0124] In one example, in the above-mentioned intelligent anti-voyeurism electronic device display system 400, the first convolutional neural network processing unit 420 is further used to: each layer of the first convolutional neural network except the last layer spatially encodes the full-body image of the monitored person at the series of time points according to the following formula to obtain a first feature map; the formula is:

[0125] f i =Sigmoid(Ni ×f i-1 +B i )

[0126] Among them, f i-1 is the input of the first convolutional neural network in the i-th layer, f i is the output of the first convolutional neural network in the i-th layer, N i is the filter unit of the first convolutional neural network in the i-th layer, and B i is the bias matrix of the i-th layer of the first convolutional neural network, Sigmoid represents a nonlinear activation function; and the last layer of the first convolutional neural network performs global pooling processing on the first feature map along the height dimension and the width dimension to obtain the first feature vector.

[0127] In one example, in the above-mentioned intelligent anti-peeping electronic device display system 400, the second convolutional neural network processing unit 440 is further used to: each layer of the first convolutional neural network except the last layer spatially encodes the facial images of the person to be monitored at the series of time points according to the following formula to obtain a second feature map; the formula is:

[0128] f i =Sigmoid(N i ×f i-1 +B i )

[0129] Among them, f i-1 is the input of the second convolutional neural network in the i-th layer, f i is the output of the second convolutional neural network in the i-th layer, N i is the filter unit of the second convolutional neural network in the i-th layer, and B i is the bias matrix of the i-th layer of the second convolutional neural network, Sigmoid represents a nonlinear activation function; and the last layer of the second convolutional neural network performs global pooling processing on the second feature map along the height dimension and the width dimension to obtain the second feature vector.

[0130] In one example, in the above-mentioned intelligent anti-peeping electronic device display system 400, the Gaussian density map fusion unit 450 is further used to: convert the eigenvalues ​​of each position in the first feature vector and the second feature vector into a probability interval of 0 to 1 to obtain a first probabilistic feature vector and a second probabilistic feature vector; and, fuse the first probabilistic feature vector and the second probabilistic feature vector based on the Gaussian density map according to the following formula to obtain the Gaussian distribution vector, wherein the formula is:

[0131]

[0132] Among them, μ(f1, f2) represents calculating the mean between the eigenvalues ​​at each position of the first probabilistic feature vector and the second probabilistic feature vector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and ∑(f1, f2) represents calculating the variance between the eigenvalues ​​at each position between the first probabilistic feature vector and the second probabilistic feature vector to obtain the covariance matrix of the Gaussian feature map.

[0133] In one example, in the above-mentioned intelligent anti-peeping electronic device display system 400, the Gaussian density map fusion unit 450 is further used to: fuse the first feature vector and the second feature vector based on the Gaussian density map to obtain the Gaussian distribution vector according to the following formula, wherein the formula is:

[0134]

[0135] Among them, μ(f1, f2) represents calculating the mean between the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian feature map, and ∑(f1, f2) represents calculating the variance between the eigenvalues ​​at each position between the first eigenvector and the second eigenvector to obtain the covariance matrix of the Gaussian feature map.

[0136] In one example, in the above-mentioned intelligent anti-peeping electronic device display system 400, the Gaussian mixture model calculation unit 460 is further used to: calculate the Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector using the following formula; wherein the formula is:

[0137]

[0138]

[0139] Among them, v(π i ) represents the weighted vector of length L, and ⊙ represents the point multiplication by position.

[0140] In one example, in the above-mentioned intelligent anti-peeping electronic device display system 400, the classification unit 480 is further used to: use at least one fully connected layer of the classifier to fully connect the feature matrix to obtain a classification feature vector; input the classification feature vector into the Softmax classification function of the classifier to obtain a first probability that the classification feature vector belongs to the person to be monitored with the intention of peeping and a second probability that the classification feature vector belongs to the person to be monitored without the intention of peeping; and, based on the comparison of the first probability and the second probability, generate the classification result.

[0141] Here, those skilled in the art can understand that the specific functions and operations of the various units and modules in the above-mentioned intelligent anti-peeping electronic device display system 400 have been described in the above reference. Figures 1 to 3 The description of the intelligent anti-peeping electronic device display method has been introduced in detail, and therefore, its repeated description will be omitted.

[0142] As described above, the intelligent anti-peeping electronic device display system 400 according to the embodiment of the present application can be implemented in various terminal devices, such as a server of an intelligent anti-peeping electronic device display algorithm. In one example, the intelligent anti-peeping electronic device display system 400 according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the intelligent anti-peeping electronic device display system 400 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the intelligent anti-peeping electronic device display system 400 can also be one of the many hardware modules of the terminal device.

[0143] Alternatively, in another example, the smart anti-voyeurism electronic device display system 400 and the terminal device may also be separate devices, and the smart anti-voyeurism electronic device display system 400 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0144] Exemplary Electronic Devices

[0145] Below, reference Figure 5 To describe the electronic device according to the embodiment of the present application. Figure 5 As shown, the electronic device 10 includes one or more processors 11 and a memory 12. The processor 11 may be a central processing unit (CPU) or other processing units having data processing capability and / or instruction execution capability, and may control other components in the electronic device 10 to perform desired functions.

[0146] The memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the functions of the intelligent anti-peeping electronic device display method of each embodiment of the present application described above and / or other desired functions. Various contents such as the first eigenvector, Gaussian density map, etc. may also be stored in the computer-readable storage medium.

[0147] In one example, the electronic device 10 may further include: an input system 13 and an output system 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0148] The input system 13 may include, for example, a keyboard, a mouse, and the like.

[0149] The output system 14 can output various information to the outside, including classification results, etc. The output system 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0150] Of course, to simplify, Figure 5 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0151] Exemplary computer program products and computer-readable storage media

[0152] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the functions of the intelligent anti-peeping electronic device display method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0153] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the intelligent anti-peeping electronic device display method described in the above "Exemplary Method" section of this specification.

[0155] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, system or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0156] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0157] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0158] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0159] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0160] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An intelligent anti-peeping electronic device display method, characterized in that: include: Acquire full-body images of the person to be monitored at a series of time points within a predetermined time; Inputting the whole body images of the person to be monitored at the series of time points into a first convolutional neural network as a filter to obtain a first feature vector; Acquire facial images of the person to be monitored at the series of time points; Inputting the facial images of the person to be monitored at the series of time points into a second convolutional neural network as a filter to obtain a second feature vector; The first eigenvector and the second eigenvector are fused based on a Gaussian density map to obtain a Gaussian distribution vector, each position of the Gaussian distribution vector is a Gaussian distribution, wherein the fusion based on the Gaussian density map represents constructing the Gaussian density map using a mean vector constructed by the means of the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean vector of the Gaussian density map and a covariance matrix constructed by the variances between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map; Calculating a Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector, wherein the weight of each position of the weighted vector is 1; Gaussian discretization is performed on the Gaussian distribution of each position of the Gaussian mixture model to obtain a feature matrix; Passing the feature matrix through a classifier to obtain a classification result indicating whether the person to be monitored has a peeping intention; and In response to the classification result that the person to be monitored has a peeping intention, starting a screen saver mode; The whole-body images of the person to be monitored at the series of time points are input into a first convolutional neural network as a filter to obtain a first feature vector, including: Each layer of the first convolutional neural network except the last layer spatially encodes the whole body images of the person to be monitored at the series of time points to obtain a first feature map according to the following formula; The formula is: ， in, is the input of the first convolutional neural network in layer i, is the output of the first convolutional neural network in layer i, is the filter unit of the first convolutional neural network in the i-th layer, and is the bias matrix of the first convolutional neural network in the i-th layer, represents a nonlinear activation function; and The last layer of the first convolutional neural network performs global pooling processing on the first feature map along the height dimension and the width dimension to obtain the first feature vector; The step of inputting the facial images of the person to be monitored at the series of time points into a second convolutional neural network as a filter to obtain a second feature vector includes: Each layer of the second convolutional neural network except the last layer performs spatial encoding on the facial images of the person to be monitored at the series of time points according to the following formula to obtain a second feature map; The formula is: ， in, is the input of the second convolutional neural network in the i-th layer, is the output of the second convolutional neural network in the i-th layer, is the filter unit of the second convolutional neural network in the i-th layer, and is the bias matrix of the second convolutional neural network in the i-th layer, represents a nonlinear activation function; and The last layer of the second convolutional neural network performs global pooling processing on the second feature map along the height dimension and the width dimension to obtain the second feature vector.

2. The intelligent anti-peeping electronic device display method according to claim 1, wherein: The first eigenvector and the second eigenvector are fused based on a Gaussian density map to obtain a Gaussian distribution vector, comprising: Converting the eigenvalues ​​of each position in the first eigenvector and the second eigenvector into a probability interval between 0 and 1 to obtain a first probabilistic eigenvector and a second probabilistic eigenvector; and The first probabilistic feature vector and the second probabilistic feature vector are fused based on a Gaussian density map using the following formula to obtain the Gaussian distribution vector, wherein the formula is: ， in, represents calculating the mean of the eigenvalues ​​at each position of the first probabilistic feature vector and the second probabilistic feature vector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian density map, and It represents calculating the variance between the eigenvalues ​​at each position between the first probabilized feature vector and the second probabilized feature vector to obtain the covariance matrix of the Gaussian density map.

3. The intelligent anti-peeping electronic device display method according to claim 1, wherein: The first eigenvector and the second eigenvector are fused based on a Gaussian density map to obtain a Gaussian distribution vector, comprising: The first eigenvector and the second eigenvector are fused based on a Gaussian density map using the following formula to obtain the Gaussian distribution vector, wherein the formula is: in, represents calculating the mean of the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean of the Gaussian distribution to obtain the mean vector of the Gaussian density map, and It represents calculating the variance between the eigenvalues ​​at each position between the first eigenvector and the second eigenvector to obtain the covariance matrix of the Gaussian density map.

4. The intelligent anti-peeping electronic device display method according to claim 3, wherein: Calculating a Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector comprises: Based on a weighted vector having the same length as the vector length of the Gaussian distribution vector, a Gaussian mixture model of the Gaussian distribution vector is calculated using the following formula; Wherein, the formula is: ， ， in, represents the weight vector of length L, Represents point product by position.

5. The intelligent anti-peeping electronic device display method according to claim 4, wherein: The feature matrix is ​​passed through a classifier to obtain a classification result indicating whether the person to be monitored has a peeping intention, including: Performing fully connected encoding on the feature matrix using at least one fully connected layer of the classifier to obtain a classification feature vector; and Inputting the classification feature vector into the Softmax classification function of the classifier to obtain a first probability that the classification feature vector belongs to the person to be monitored having a voyeuristic intention and a second probability that the classification feature vector belongs to the person to be monitored having no voyeuristic intention; and The classification result is generated based on a comparison between the first probability and the second probability.

6. An intelligent anti-peeping electronic device display system, characterized in that: include: A full-body image acquisition unit, used to acquire full-body images of the person to be monitored at a series of time points within a predetermined time; a first convolutional neural network processing unit, configured to input the whole-body images of the person to be monitored at the series of time points obtained by the whole-body image acquisition unit into a first convolutional neural network as a filter to obtain a first feature vector; A facial image acquisition unit, used to acquire facial images of the person to be monitored at the series of time points; a second convolutional neural network processing unit, configured to input the facial images of the person to be monitored at the series of time points obtained by the facial image acquisition unit into a second convolutional neural network as a filter to obtain a second feature vector; a Gaussian density map fusion unit, configured to fuse the first eigenvector obtained by the first convolutional neural network processing unit and the second eigenvector obtained by the second convolutional neural network processing unit based on a Gaussian density map to obtain a Gaussian distribution vector, wherein each position of the Gaussian distribution vector is a Gaussian distribution, wherein the fusion based on the Gaussian density map represents constructing the Gaussian density map using a mean vector constructed by the mean of the eigenvalues ​​at each position of the first eigenvector and the second eigenvector as the mean vector of the Gaussian density map and a covariance matrix constructed by the variance between each position of the first eigenvector and the second eigenvector as the covariance matrix of the Gaussian density map; a Gaussian mixture model calculation unit, configured to calculate a Gaussian mixture model of the Gaussian distribution vector based on a weighted vector having the same length as the vector length of the Gaussian distribution vector obtained by the Gaussian density map fusion unit, wherein the weight of each position of the weighted vector is 1; A Gaussian discretization unit, used for performing Gaussian discretization on the Gaussian distribution of each position of the Gaussian mixture model obtained by the Gaussian mixture model calculation unit to obtain a feature matrix; A classification unit, used for passing the feature matrix obtained by the Gaussian discretization unit through a classifier to obtain a classification result indicating whether the person to be monitored has a peeping intention; and A response unit, configured to start a screen saver mode in response to the classification result obtained by the classification unit that the person to be monitored has a peeping intention; Wherein, the first convolutional neural network processing unit includes: Each layer of the first convolutional neural network except the last layer spatially encodes the whole body images of the person to be monitored at the series of time points to obtain a first feature map according to the following formula; The formula is: ， in, is the input of the first convolutional neural network in layer i, is the output of the first convolutional neural network in layer i, is the filter unit of the first convolutional neural network in the i-th layer, and is the bias matrix of the first convolutional neural network in the i-th layer, represents a nonlinear activation function; and The last layer of the first convolutional neural network performs global pooling processing on the first feature map along the height dimension and the width dimension to obtain the first feature vector; Wherein, the second convolutional neural network processing unit includes: Each layer of the second convolutional neural network except the last layer performs spatial encoding on the facial images of the person to be monitored at the series of time points according to the following formula to obtain a second feature map; The formula is: ， in, is the input of the second convolutional neural network in the i-th layer, is the output of the second convolutional neural network in the i-th layer, is the filter unit of the second convolutional neural network in the i-th layer, and is the bias matrix of the second convolutional neural network in the i-th layer, represents a nonlinear activation function; and The last layer of the second convolutional neural network performs global pooling processing on the second feature map along the height dimension and the width dimension to obtain the second feature vector.

7. An electronic device, comprising: processor; as well as A memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the intelligent anti-peeping electronic device display method as described in any one of claims 1-5.

Images