Online classroom concentration recognition method with visual privacy protection function

Abstract

The present invention relates to the fields of computer vision and pattern recognition, and specifically to a method for identifying online classroom concentration with a visual privacy protection function. The method comprises the following steps: first, achieving a visual privacy protection effect by performing style transfer on video data, extracting image color features and salient structural features of the processed video frame sequence and mapping them to an image visual privacy protection score, obtaining a recognition rate for concentration recognition from the processed video frame sequence, and finally establishing a binary statistical model between the visual privacy protection score and the online classroom concentration recognition rate. The present invention addresses the privacy contradiction between video surveillance and intelligent applications, balances the conflict between privacy protection and intelligent applications while protecting the privacy of video data, and has high practical application value.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and pattern recognition, and in particular to an online classroom concentration recognition method with a visual privacy protection function. Background Art

[0002] With the widespread adoption and continuous development of internet technology, non-traditional teaching models, particularly online classroom teaching models, have become a hot research topic. Over the past few years, online education has made significant progress. From simple text and video tutorials to rich interactive courses and virtual labs, the content and formats of online education have become increasingly diverse. Despite its many advantages, online education also faces numerous challenges in its specific application in real-time video interactive classrooms. In traditional offline classrooms, teachers can adjust their teaching methods and progress based on student attendance. However, in real-time video online teaching, teachers can only assess students' learning status through computer screens and audio. Since the display screen is primarily used to play teaching materials, it is difficult for teachers to detect students' dynamics, which in turn affects teaching quality. Computer-based attention monitoring systems can address this issue.

[0003] Based on the source of the detection signal, the methods for monitoring concentration are mainly divided into three categories: (1) Methods based on electroencephalography technology, which monitor students' brain waves through head-mounted devices to obtain students' brain activity; (2) Methods based on activity sensing technology, which use sensor devices to monitor students' physical activities, such as heart rate, breathing, body temperature, etc., to obtain students' physiological status; (3) Methods based on face recognition technology, which use cameras to capture students' facial expressions and eye movements to infer students' concentration. First, although the concentration monitoring methods based on wearable devices are not sensitive to the environment, wearing them all the time can cause people to feel uncomfortable or resistant. In addition, there are differences in physiological baseline levels between different individuals, which requires complex calibration. Compared with other sensor-based systems, face recognition technology has the advantages of low equipment cost, low interference, and good recognition effect, and therefore has broad application prospects in the field of concentration monitoring. However, traditional vision-based concentration monitoring has the risk of infringing personal privacy information, such as bank card numbers, mobile phone chat interfaces, etc. The feasibility of using these monitoring systems in a home environment is questionable.

[0004] Therefore, in order to solve these problems, the present invention proposes a novel concentration monitoring method with visual privacy protection function to protect personal privacy while monitoring students' concentration. Summary of the Invention

[0005] The purpose of this invention is to solve the problem of privacy infringement in the existing online classroom concentration monitoring system. A style transfer method is proposed for visual privacy protection, and a statistical model of the association between the level of visual privacy protection and concentration recognition is further established.

[0006] In order to solve the above technical problems, the present invention discloses an online classroom concentration recognition method with a visual privacy protection function, comprising the following steps:

[0007] S1. Perform style transfer processing on online classroom video data to achieve visual privacy protection;

[0008] S2. Extracting image color features and salient structural features of the video frame sequence after style transfer processing;

[0009] S3. Mapping image color features and salient structural features to image visual privacy protection scores;

[0010] S4. Obtaining the recognition rate of focus recognition from the video frame sequence after style transfer processing;

[0011] S5. Establish a binary statistical model between the visual privacy protection score and the online class concentration recognition rate.

[0012] Among them, in step S1, the style transfer processing method is a convolutional neural network, which transfers the style of one image to another image through a convolutional neural network. The content information of the image is captured by the deeper layers of the network, while the style information of the image is captured by the shallow and intermediate layers of the network. By adjusting the intensity of style transfer and the target style, effective extraction of target content and protection of privacy content can be achieved under different privacy protection requirements and application scenarios. VGG-Network is selected to extract target features, and by performing gradient descent optimization on different layers of the network, starting from the initial white noise image, an image that matches the feature response of the original image is gradually generated to reconstruct the encoded content of a given layer;

[0013] Define the content loss function for:

[0014]

[0015] Among them, I C is an image containing modeling content, and its response matrix is I G To generate an image, the response matrix is

[0016] The gradient is calculated by backpropagation with the standard error, using the derivative of the loss:

[0017]

[0018] The style representation of an image at a given network layer is obtained by calculating the correlation between the feature responses of that layer. Specifically, the correlation is represented by a l Gram matrix G of rows and columns l Definition, where G l (i, j) is the inner product of the flattened vectors of feature responses i and j. To find a style representation that matches the input image, the algorithm first initializes a random image and performs gradient descent optimization on the style representation;

[0019] Define style loss function is the loss E of each layer of the network l The weighting is:

[0020]

[0021] Among them, I s Represents the style image, I G Represents the generated image. The minimization process of this loss function will make the generated image match the style image as closely as possible in the style representation of a given network layer;

[0022]

[0023] Where, is the Gram matrix of the style image, is the Gram matrix of the generated image, n l is the number of feature maps, m l is the total dimension of each feature map. For the activation degree E l The derivative of is:

[0024]

[0025] In order to generate an image that mixes the content of one image with the style of another image, the neural art algorithm initializes a white noise image and simultaneously minimizes the content loss at one level and the style loss across multiple levels. Its total loss function is:

[0026]

[0027] In step S2, the image color features are obtained by dividing the video frame after style transfer processing, converting it into the LAB domain, and calculating the color histogram. The color histogram CHF is expressed as follows:

[0028]

[0029] Where B is the number of image blocks, I blockis the image block in LAB space, G σ is a rotationally symmetric Gaussian low-pass filter generated by the standard deviation, * is the convolution operation of the matrix, Hist(·) is a histogram function containing k bins, and F τ (·) is a normalization function with a threshold τ.

[0030] Among them, in step S2, the specific steps of extracting significant structural features are:

[0031] First, the image is preprocessed by local contrast normalization:

[0032]

[0033] Among them, I block is an image block of the same size as the GCS-LBP window, I block (i, j) represents the center pixel of the image block, const is a very small positive number set to avoid the denominator being zero, μ(i, j) and σ(i, j) refer to the mean and standard deviation of an image block in the image, respectively;

[0034] Image patch after contrast normalization Perform GCS-LBP encoding:

[0035]

[0036] in, and They are two pixels symmetrical about the central pixel of the GCS-LBP window with a radius of R and containing P pixels. is the assignment function with threshold T:

[0037]

[0038] Use the Itti-Korch saliency model to extract image saliency features:

[0039]

[0040] I, C, and O are the brightness, color, and direction integrated saliency maps, respectively. Saliency is the final fused saliency map.

[0041] Combining the Itti-Korch saliency model with the GCS-LBP operator, a saliency generalized centrosymmetric local binary pattern operator IGCS-LBP is proposed to extract saliency structural features for image visual privacy protection evaluation. The calculation formula is as follows:

[0042]

[0043] Among them, IGL Represents the feature map derived from GCS-LBP, I GB Indicates the significant value from the Itti-Korch significance model;

[0044] In step S3, the image color features and salient structural features are mapped to the image visual privacy protection score through the support vector regression model. The support vector regression (SVR) equation is as follows:

[0045]

[0046] where n is the number of support vectors, and the Lagrange multiplier α is i and And the parameter b needs to be determined when training the SVR model. K(x, x i ) is the kernel function used to test the input vectors x and x i Using RBF as the SVR kernel function can more accurately predict the VPP level, which is expressed as:

[0047] K(u,v)=exp(-γ||uv|| 2 );

[0048] The features of the training set and the corresponding difference-of-means (DMOS) values ​​are used as input to train the SVR model. Finally, the features of the test set are used as input to the trained model to obtain the predicted style transfer visual privacy protection score.

[0049] In step S4, the face target area is detected in the video frame sequence after style transfer processing using the IntraFace pre-trained model, the face target is input into the improved CNN network to extract facial features and the focus classification is performed using the Sigmoid classifier.

[0050] In step S5, a correlation statistical model between the visual privacy protection scores of video frames encoded with different style transfer rates and the corresponding online class concentration recognition rates of the video frames is obtained;

[0051] Preferably, the curve fitting method in step S5 includes linear fitting, Gaussian fitting, exponential fitting and polynomial fitting;

[0052] More preferably, the curve fitting method in step S5 is polynomial fitting.

[0053] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the online classroom concentration identification method with visual privacy protection function of the present invention.

[0054] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the steps of the online classroom concentration identification method with visual privacy protection function of the present invention are implemented.

[0055] Beneficial effects: The present invention addresses the risk of privacy leakage in traditional videos at the visual level, and uses the style transfer imaging mechanism to improve imaging, thereby achieving the purpose of visual privacy protection. The present invention addresses the problem of characterizing the degree of privacy protection of style transfer result images, and uses color features and significant structural features to objectively evaluate the degree of visual privacy protection. The present invention addresses the problem of intelligent application of style transfer result images, extracts the target area of ​​the face, and then realizes concentration recognition by improving the CNN network method. The present invention establishes a binary statistical model between the visual privacy protection degree and the concentration recognition rate of the style transfer result image, and seeks a balance between the visual privacy protection score and the online classroom recognition rate, that is, ensuring a high recognition rate while maintaining high visual privacy protection, providing value for its practical application. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more apparent.

[0057] Figure 1 Schematic diagram of the process of the present invention;

[0058] Figure 2 This is a partial example diagram of an online classroom dataset constructed in an embodiment of the present invention;

[0059] Figure 3 Schematic diagram of image visual privacy protection under different style transfer rates in an embodiment of the present invention;

[0060] Figure 4 This is a line chart showing the concentration level of videos with different style transfer rates according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further explained below with reference to the accompanying drawings and specific embodiments.

[0062] Example 1:

[0063] See Figure 1 The embodiment of the present invention provides an online classroom concentration recognition method with a visual privacy protection function, comprising the following steps:

[0064] Step 1: We constructed an online classroom focus dataset, selected effective video clips, and performed style transfer on the video data using several effective neural styles to achieve visual privacy protection.

[0065] This technology uses convolutional neural networks (CNNs) to transfer the style of one image to another. The deeper layers of the network capture the image's content information, while the shallower and intermediate layers capture the image's style information. By adjusting the strength of the style transfer and the target style, it can effectively extract the target content and protect the privacy of the content under different privacy protection requirements and application scenarios. The VGG-Network is used to extract target features. By performing gradient descent optimization on the different layers of the network, starting from an initial white noise image, an image that matches the feature response of the original image is gradually generated to reconstruct the encoded content at a given level.

[0066] 1.1 Content Loss Function Defined as:

[0067]

[0068] Among them, I C is an image containing modeling content, and its response matrix is I G To generate an image, the response matrix is The gradient is calculated by backpropagation with the standard error, using the derivative of the loss:

[0069]

[0070] The style representation of an image at a given network layer is obtained by calculating the correlation between the feature responses of that layer. Specifically, the correlation is represented by a l Gram matrix G of rows and columns l Definition, where G l (i, j) is the inner product of the flattened vectors of feature responses i and j. To find a style representation that matches the input image, the algorithm first initializes a random image and performs gradient descent optimization on the style representation.

[0071] 1.2 Style Loss Function Defined as the loss E of each layer of the network l Weighted:

[0072]

[0073] Among them, I s Represents the style image, I G Represents the generated image. The minimization process of this loss function will make the generated image match the style image as closely as possible based on the style representation of the given network layer.

[0074]

[0075] Where, is the Gram matrix of the style image, is the Gram matrix of the generated image, n l is the number of feature maps, m l is the total dimension of each feature map. For the activation degree E l The derivative of is:

[0076]

[0077] 1.3 In order to generate an image that mixes the content of one image with the style of another image, the neural art algorithm initializes a white noise image and simultaneously minimizes the content loss at one level and the style loss across multiple levels. Its total loss function is:

[0078]

[0079] Among them, α is the weight parameter of the content loss, which controls the content image I C To generate image I G β is the weight parameter of style loss. It controls the style image I S To generate image I G They determine the optimal weight combination through continuous learning of the deep learning network.

[0080] Step 2: For the video frame sequence with style transfer visual privacy protection, extract image color and salient structure features respectively:

[0081] Step 2.1: Convert the style transfer video frame into LAB domain and calculate its color histogram CHF:

[0082] First, the image is divided into M×M blocks. After each block is converted to the LAB model, it is downsampled in the L, A, and B channels and convolved with a Gaussian kernel. The convolution result is then linearly interpolated and its histogram is calculated. The result after histogram processing is normalized as the color histogram of the image block. Finally, the average color histogram of all image blocks is calculated as the color histogram of the entire image. The specific form is as follows:

[0083]

[0084] Where B is the number of image blocks, I block is the image block in LAB space, G σis a rotationally symmetric Gaussian low-pass filter generated by the standard deviation, * is the convolution operation of the matrix, Hist(·) is a histogram function containing k bins, and F τ (·) is a normalization function with a threshold τ.

[0085] Step 2.2: Calculate the salient structural features of the style-transferred video frame:

[0086] This case uses the GCS-LBP operator to extract the structural features of the image. First, the image is preprocessed by local contrast normalization:

[0087]

[0088] Among them, I block is an image block of the same size as the GCS-LBP window, I block (i, j) represents the center pixel of the image block, const is a very small positive number set to avoid the denominator being zero, μ(i, j) and σ(i, j) refer to the mean and standard deviation of an image block in the image, respectively.

[0089] Image patch after contrast normalization Perform GCS-LBP encoding:

[0090]

[0091] in, and They are two pixels symmetrical about the central pixel of the GCS-LBP window with a radius of R and containing P pixels. is the assignment function with threshold T:

[0092]

[0093] Use the Itti-Korch saliency model to extract image saliency features:

[0094]

[0095] I, C, and O are the brightness, color, and direction integrated saliency maps, respectively, and Saliency is the final fused saliency map.

[0096] In this embodiment, the Itti-Korch saliency model is combined with the GCS-LBP operator to propose a saliency generalized centrosymmetric local binary pattern operator IGCS-LBP to extract saliency structural features for image visual privacy protection evaluation. The calculation formula is as follows:

[0097] IGCS-LBP=∑ i ∑ j IGL (i, j)·Δ(I GB (i, j, k) (12)

[0098]

[0099] Among them, I GL Represents the feature map derived from GCS-LBP, I GB Indicates the significant value from the Itti-Korch significance model;

[0100] Step 3: Input the two features into the SVR model to obtain the visual privacy protection of the video frame:

[0101] The color feature CHF and salient structure feature IGCS-LBP of the style-transferred video frame are input into the SVR model to obtain the visual privacy protection degree of the video frame. The SVR regression equation is as follows:

[0102]

[0103] where n is the number of support vectors, and the Lagrange multiplier α is i and And the parameter b needs to be determined when training the SVR model. K(x, x i ) is the kernel function used to test the input vectors x and x i Using RBF as the SVR kernel function can more accurately predict the VPP level, which is expressed as:

[0104] K(u,v)=exp(-γ||uv|| 2 ) (15)

[0105] The features of the training set and the corresponding difference-of-means (DMOS) values ​​are used as input to train the SVR model. Finally, the features of the test set are used as input to the trained model to obtain the predicted style transfer visual privacy protection score.

[0106] Step 4: For the style transfer visual preservation video frame sequence, use the IntraFace pre-trained model to detect the face target area, input the face target into the improved CNN network to extract facial features and perform focus classification using the Sigmoid classifier:

[0107] Step 4.1: Crop the face region based on the landmarks detected by the IntraFace pre-processing model. These labels can be used to extract relevant information about the face. A larger crop size can retain more information, while a smaller crop size can reduce noise from the background. In the implementation process, the cropped image size S is given by S = α × max (d υ , d h) is determined, where α is a scalar that controls the size of the face area, d υ is the distance between the uppermost landmark point and the lowermost landmark point, d h is the horizontal distance between the leftmost landmark point and the rightmost ground point. After determining the cropping size, we set the nose as the center position of the cropped face to obtain a moderate face image for model training.

[0108] Step 4.2: The network architecture of this example consists of two convolution and pooling blocks, followed by two fully connected layers. We use ReLU as the activation function for each convolution layer. To prevent overfitting, Dropout is also used after the fully connected layer. The CNN network structure in this embodiment is simple and can achieve good results in the focus recognition task. In order to further improve the discriminative ability of the model, two consecutive convolution layers are stacked before the maximum pooling. Larger convolution filters are used, which gives the neurons in the model a larger receptive field. After the modification, the receptive field of each neuron in the first fully connected layer becomes 12×12, which is about 6% of the input 48×48 image.

[0109] Step 5: Establish a binary statistical model between the visual privacy protection score and the focus recognition rate to balance the relationship between the two:

[0110] Step 5.1: To further quantify the correlation between visual privacy protection and online classroom focus recognition results, curve fitting is performed between the visual privacy protection of video frames encoded with different style transfer rates and their corresponding focus recognition rates. The fitting methods selected in this implementation case are: linear fitting, Gaussian fitting, exponential fitting, and polynomial fitting.

[0111] Step 5.2: Select the best fitting curve using the sum of squared errors (SSE), coefficient of determination (R-square), and root mean square (RMSE). The fitting effect is better when the SSE and RMSE are close to 0 and the R-square is close to 1. In this embodiment, the present invention selects polynomial fitting from various fitting results as the statistical model for the association between visual privacy protection and recognition rate.

[0112] The specific model is:

[0113] 1. Input:

[0114] Visual privacy-preserving video frames Here, m, n are the number of focused and non-focused video frames, respectively, and R∈{18%, 24%, 30%, 36%, 42%}.

[0115] 2. Calculation I R Visual privacy protection level:

[0116] 2.1 Extraction of CHF:

[0117]

[0118] 2.2 Extracting IGCS-LBP:

[0119]

[0120] 2.3 Input the feature F = {CHF, IGCS-LBP} into the SVR model to obtain the visual privacy protection level. The SVR model is:

[0121]

[0122] 3. Calculate the attention recognition rate of style transfer visual privacy protection videos with different style transfer rates.

[0123] 4. Output: Statistical model of the correlation between visual privacy protection level and attention recognition rate:

[0124] The Gaussian fitting formula is:

[0125] Where a is the height of the peak, b is the location of the peak (mean), and C is the standard deviation, which determines the width of the curve.

[0126] To balance visual privacy protection in style transfer and attentiveness recognition, this implementation establishes a binary statistical model between the two. While achieving visual privacy protection for video data, it is still desirable to ensure the usefulness of the privacy-protected video, that is, to ensure its recognition effect in intelligent applications. When the visual privacy protection score of an image is low, the visual information contained in the image does not meet the privacy protection requirements. When the visual privacy protection score of an image is high, the recognition rate of the image will be greatly reduced, making the image in the visual privacy protection state lose its application value. Therefore, it is desirable to strike a balance between the visual privacy protection score and the online classroom recognition rate, that is, to ensure a high recognition rate while maintaining high visual privacy protection, and to establish a quantitative correlation statistical model between visual security and attentiveness recognition rate.

[0127] Example 2:

[0128] The computer-readable storage medium of this embodiment stores a computer program, which, when executed by a processor, implements the steps of the online classroom concentration identification method with visual privacy protection function of Example 1.

[0129] The computer-readable storage medium of this embodiment may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal; the computer-readable storage medium of this embodiment may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc. equipped on the terminal; further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device.

[0130] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0131] Example 3:

[0132] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the online classroom concentration identification method with visual privacy protection function of Example 1 are implemented.

[0133] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The memory can include read-only memory and random access memory, and provide instructions and data to the processor. A part of the memory can also include non-volatile random access memory. For example, the memory can also store information about the device type.

[0134] Those skilled in the art will appreciate that the disclosed contents of the embodiments may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to the flowcharts and / or block diagrams of the methods and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as the combination of the processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions; these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0138] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0139] The present invention provides a method for identifying online classroom concentration with visual privacy protection. There are many methods and approaches for implementing this technical solution. The above is only a preferred embodiment of the present invention. It should be noted that those skilled in the art may make various improvements and modifications without departing from the principles of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention. Any components not specified in this embodiment may be implemented using existing technologies.

Claims

1. A method for identifying online classroom concentration with visual privacy protection function, characterized in that: The following steps are involved: S1. Perform style transfer processing on online classroom video data to achieve visual privacy protection; S2. Extracting image color features and salient structural features of the video frame sequence after style transfer processing; Among them, in step S2, the specific steps of extracting significant structural features are: First, the image is preprocessed by local contrast normalization: ； in, is an image block of the same size as the GCS-LBP window, Indicates the center pixel of the image block. const is a very small positive number set to avoid the denominator being zero. and Refers to the mean and standard deviation of an image block in the image respectively; Image patch after contrast normalization Perform GCS-LBP encoding: ； in, and They are two pixels symmetrical about the central pixel of the GCS-LBP window with a radius of R and containing P pixels. is a valued function with a threshold value T: ； Use the Itti-Korch saliency model to extract image saliency features: ； I, C, O are the brightness, color and direction comprehensive saliency maps respectively, and Saliency is the final fusion saliency map; Combining the Itti-Korch saliency model with the GCS-LBP operator, a saliency generalized centrosymmetric local binary pattern operator IGCS-LBP is proposed to extract saliency structural features for image visual privacy protection evaluation. The calculation formula is as follows: ； ； in, represents the feature map, Indicates significant value; S3. Mapping image color features and salient structural features to image visual privacy protection scores; S4. Obtaining the recognition rate of focus recognition from the video frame sequence after style transfer processing; S5. Establish a binary statistical model between the visual privacy protection score and the online class concentration recognition rate.

2. The method according to claim 1, characterized in that In step S1, the style transfer processing method is a convolutional neural network. By performing gradient descent optimization on different layers of the network, starting from an initial white noise image, an image that matches the feature response of the original image is gradually generated. The specific steps are as follows: Define the content loss function : ； in, is an image containing modeling content, and its response matrix is ; To generate an image, the response matrix is ; The gradient is calculated using standard error backpropagation, using the derivative of the loss in the form: ； The style representation of an image at a given network layer is obtained by calculating the correlation between the feature responses of that layer. Specifically, the correlation is represented by a Gram matrix of rows and columns Definition, where is the inner product of the flattened vectors of feature responses i and j; to find a style representation that matches the input image, the algorithm first initializes a random image and performs gradient descent optimization on the style representation; Define style loss function : ； in, represents the style image, Indicates the generated image, is the loss of each layer of the network; Specifically, the minimization process of this loss function will make the generated image match the style image as much as possible in the style representation of a given network layer. The loss function is defined as: ； in, is the Gram matrix of the style image, To generate the Gram matrix of the image, is the number of feature maps, is the total dimension of each feature map; For the activation of layer l The derivative of is: ； To generate an image that mixes the content of one image with the style of another, the Neural Art algorithm initializes a white noise image and simultaneously minimizes the content loss at one level and the style loss across multiple levels; defining the total loss function for: ； in, is the weight parameter of content loss; is the weight parameter of style loss.

3. The method according to claim 1, characterized in that In step S2, the image color features are obtained by dividing the video frame obtained after the style transfer process into blocks, converting it into the LAB domain, and calculating the color histogram.

4. The method according to claim 1, wherein In step S3, the image color features and salient structural features are mapped to the image visual privacy protection score through a support vector regression model.

5. The method according to claim 1, wherein In step S4, the face target area is detected in the video frame sequence after style transfer processing using the IntraFace pre-trained model, the face target is input into the improved CNN network to extract facial features and the focus classification is performed using the Sigmoid classifier.

6. The method according to claim 1, characterized in that In step S5, a correlation statistical model between the visual privacy protection scores and the online class concentration recognition rate is obtained by curve fitting the visual privacy protection scores of the video frames encoded with different style transfer rates and the corresponding online class concentration recognition rates of the video frames.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the online classroom concentration recognition method with visual privacy protection function as described in any one of claims 1 to 6 are implemented.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the online classroom concentration identification method with visual privacy protection function as described in any one of claims 1 to 6 are implemented.

Images