Visual privacy protection method, device, electronic device and storage medium for video data

Abstract

The present disclosure relates to the technical field of computer vision privacy protection, and discloses a visual privacy protection method, device, electronic device, and storage medium for video data, the method comprising: compressing the video data to be processed, and determining the compressed video data including multiple I frames, multiple P frames, and multiple B frames; for any I frame, dividing the I frame into multiple regions according to a preset window scale; determining the privacy metric entropy corresponding to any region in the I frame according to the grayscale histogram corresponding to the region; encrypting the I frame according to the privacy metric entropy corresponding to each region, and determining the encrypted frame corresponding to the I frame; determining the target video data according to the encrypted frame corresponding to each I frame, multiple P frames, and multiple B frames. Through the embodiments of the present disclosure, the complexity of the visual information of the video frame can be quantitatively evaluated, and encryption processing can be carried out in a targeted manner, thereby ensuring the availability of the video data while improving the visual privacy protection effect.

Description

Technical Field

[0001] The present disclosure relates to the technical field of computer vision privacy protection, and in particular to a method, device, electronic device, and storage medium for visual privacy protection of video data. Background Art

[0002] In the prior art, video data is widely used in deep neural network (DNN) related tasks in various scenarios, such as computer vision, autonomous driving, etc. The video data used in these task scenarios may involve the user's visual privacy. During the training and use of DNN, there are many attack methods that can steal video data and obtain the user's visual privacy. Therefore, it is very important to protect the visual privacy of video data while ensuring the availability of video data and not affecting the training and use of DNN. Summary of the invention

[0003] In view of this, the present disclosure proposes a technical solution for a method, device, electronic device, and storage medium for visual privacy protection of video data.

[0004] According to one aspect of the present disclosure, a method for visual privacy protection of video data is provided, comprising: compressing video data to be processed to determine compressed video data, wherein the compressed video data includes multiple I frames, multiple P frames, and multiple B frames; for any I frame, dividing the I frame into multiple regions according to a preset window scale; for any I frame, determining a privacy metric entropy corresponding to any region in the I frame according to a grayscale histogram corresponding to the region, wherein the privacy metric entropy corresponding to any region is used to describe the complexity of visual information corresponding to the region; for any I frame, encrypting the I frame according to the privacy metric entropy corresponding to each region in the I frame, and determining an encrypted frame corresponding to the I frame; determining target video data for visual privacy protection according to the encrypted frame corresponding to each I frame, the multiple P frames, and the multiple B frames.

[0005] In a possible implementation, for any I frame, according to a grayscale histogram corresponding to any area in the I frame, determining the privacy metric entropy corresponding to the area includes: for any I frame, according to a grayscale histogram corresponding to any area in the I frame, determining the frequency of occurrence corresponding to each grayscale value in the area; according to the frequency of occurrence corresponding to each grayscale value in the area, determining the privacy metric entropy corresponding to the area.

[0006] In a possible implementation, for any I frame, the I frame is encrypted according to the privacy metric entropy corresponding to each area in the I frame, and the encrypted frame corresponding to the I frame is determined, including: for any I frame, according to the privacy metric entropy corresponding to each area in the I frame, the standard deviation corresponding to each area is determined; for any I frame, according to the standard deviation corresponding to each area in the I frame, Gaussian blur is performed on each area respectively, to determine the encrypted frame corresponding to the I frame.

[0007] In a possible implementation, for any I frame, the I frame is encrypted according to the privacy measurement entropy corresponding to each area in the I frame, and the encrypted frame corresponding to the I frame is determined, including: for any I frame, according to the privacy measurement entropy corresponding to each area in the I frame, the motion parameters corresponding to each area are determined, wherein the motion parameters corresponding to any area include the simulated motion speed, simulated motion direction and simulated motion time corresponding to the area; for any I frame, according to the motion parameters corresponding to each area in the I frame, each area is motion blurred respectively, and the encrypted frame corresponding to the I frame is determined.

[0008] In a possible implementation, for any I frame, according to the privacy metric entropy corresponding to each region in the I frame, the I frame is encrypted to determine the encrypted frame corresponding to the I frame, including: for any I frame, according to the privacy metric entropy corresponding to each region in the I frame, a pixel interference parameter corresponding to each region is determined, wherein the pixel interference parameter corresponding to any region is used to adjust the position of each pixel in the region in at least one image channel; for any I frame, according to the pixel interference parameter corresponding to each region in the I frame, the I frame is pixel-scrambled to determine the encrypted frame corresponding to the I frame.

[0009] In a possible implementation, compressing the video data to be processed and determining the compressed video data includes: determining a plurality of image frames to be compressed in the video data to be processed, and a reference image frame corresponding to each image frame to be compressed; for any image frame to be compressed, dividing the image frame to be compressed and the reference image frame corresponding to the image frame to be compressed into a plurality of image blocks respectively; for any image frame to be compressed, determining, according to the position of any image block in the image frame to be compressed, a predicted position of a reference image block corresponding to the image block in the reference image frame corresponding to the image frame to be compressed; determining, according to the position of any image block in the image frame to be compressed and the predicted position of the reference image block corresponding to the image block, a coding residual corresponding to the image frame to be compressed, wherein the coding residual is used to perform coding compression on the image frame to be compressed based on the reference image frame corresponding to the image frame to be compressed; and performing video compression on the video data to be processed according to the coding residual corresponding to each image frame to be compressed to determine the compressed video data.

[0010] In one possible implementation, the method further includes: performing a performance test on a target deep neural network corresponding to the target video data based on the target video data to determine a convergence speed corresponding to the target deep neural network; when the convergence speed does not meet preset requirements, adjusting the adaptive moment estimation optimizer of the target deep neural network to obtain an optimized deep neural network.

[0011] According to another aspect of the present disclosure, a visual privacy protection device for video data is provided, comprising: a video compression module, used to compress video data to be processed and determine compressed video data, wherein the compressed video data includes multiple I frames, multiple P frames, and multiple B frames; a frame processing module, used to divide any I frame into multiple regions according to a preset window scale; a privacy measurement module, used to determine, for any I frame, a privacy measurement entropy corresponding to any region in the I frame according to a grayscale histogram corresponding to the region, wherein the privacy measurement entropy corresponding to any region is used to describe the complexity of visual information corresponding to the region; an I frame encryption module, used to encrypt any I frame according to the privacy measurement entropy corresponding to each region in the I frame, and determine an encrypted frame corresponding to the I frame; and a target video data determination module, used to determine target video data for visual privacy protection according to the encrypted frame corresponding to each I frame, the multiple P frames, and the multiple B frames.

[0012] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0013] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0014] In the disclosed embodiment, by compressing the video data to be processed, the compressed video data including multiple I frames, multiple P frames, and multiple B frames can be determined, so that the amount of data to be processed in the encryption process can be reduced by using the inter-frame analysis technology, the encryption efficiency can be improved, and the resource consumption of visual privacy protection can be reduced. For any I frame, the I frame can be divided into multiple regions according to the preset window scale, and then the grayscale histogram corresponding to any region in the I frame can be used to determine the privacy metric entropy corresponding to the region, so as to quantitatively describe the complexity of the visual information corresponding to the region, and provide real and reliable data support for the subsequent encryption process for the I frame, improve the reliability and flexibility of visual information privacy protection, so as to balance the requirements of visual information privacy protection and maintaining the availability of video data. For any I frame, according to the privacy metric entropy corresponding to each region in the I frame, the I frame can be encrypted in a targeted manner to determine the encrypted frame corresponding to the I frame. Compared with the prior art, there is no need to perform high-cost encryption calculations or high-volume noise injection, which can improve encryption efficiency and reduce resource consumption in the encryption process. According to the encrypted frames, multiple P frames, and multiple B frames corresponding to each I frame, the target video data for visual privacy protection can be determined to ensure the data validity of subsequent target video data in the application process, especially in the model training of deep neural networks, to retain the high efficiency and model accuracy of its training performance. On the other hand, the visual privacy protection method provided by the present disclosure does not rely on specific hardware results or task scenarios, has high versatility, and can be widely used in different visual tasks and various stages of visual tasks.

[0015] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0017] Figure 1 A schematic diagram showing a method for stealing video data according to an embodiment of the present disclosure is shown.

[0018] Figure 2 A flowchart of a method for visual privacy protection of video data according to an embodiment of the present disclosure is shown.

[0019] Figure 3A schematic diagram showing an inter-frame analysis technology according to an embodiment of the present disclosure is shown.

[0020] Figure 4 A schematic diagram showing a process of determining entropy of a privacy metric according to an embodiment of the present disclosure is shown.

[0021] Figure 5 A block diagram of a visual privacy protection device for video data according to an embodiment of the present disclosure is shown.

[0022] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0024] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0025] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0026] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0027] In the existing technology, video data is widely used in deep neural network (DNN) related tasks in various scenarios, such as computer vision, autonomous driving, etc. The video data used in these task scenarios may involve the user's visual privacy, and during the training and use of DNN, there are many attack methods that can steal video data and obtain the user's visual privacy.

[0028] Figure 1A schematic diagram of a method for stealing video data according to an embodiment of the present disclosure is shown. Figure 1 As shown, in common DNN usage scenarios, the ways to steal video data include: directly stealing uploaded video data when the video data is uploaded from the client to the server where the DNN is deployed; reconstructing video data when the DNN uses the video data for training or reasoning; and reconstructing video data using DNN model weights, gradients and other information.

[0029] In the existing technology, commonly used privacy protection technologies mainly include: Differential Privacy (DP) technology, Fully Homomorphic Encryption (FHE) technology, Trusted Execution Environment (TEE) technology, and technology combining Federated Learning (FL) and DP, etc. However, these privacy protection technologies have obvious limitations when applied to visual privacy protection of video data.

[0030] Specifically, DP technology can ensure privacy leakage by adding noise to the data, and is often used to protect the privacy of members of a data set. However, DP technology usually applies uniform noise processing to all parts of the data set, lacks refined processing capabilities, and is less applicable to video data; and when observing video data through the human eye, the high-frequency noise added by DP technology can be filtered, so that some sensitive visual information in the video data is still visible, resulting in poor privacy protection of video data by DP technology. In order to enhance the effect of privacy protection, it is necessary to increase the amount of noise added to the video data by DP technology, which will affect the performance and accuracy of DNN.

[0031] FHE technology refers to an encryption algorithm that satisfies the homomorphic operation property of ciphertext, that is, after the data is encrypted on the same platform, a specific calculation is performed on the ciphertext, and the ciphertext calculation result is equivalent to directly performing the same calculation on the plaintext data after the corresponding homomorphic decryption, realizing the "calculable but invisible" of data. The defect of FHE technology is that its computational complexity is high. When encrypting video data, it consumes more computing resources and there is a serious ciphertext expansion problem, which makes the amount of ciphertext data obtained by encryption processing larger than the amount of plaintext video data, thereby increasing the burden on storage and data transmission resources and reducing the processing speed of DNN using ciphertext data for training or reasoning.

[0032] TEE technology can achieve data privacy protection by executing DNN training or reasoning processes in a trusted hardware environment. However, since TEE technology relies on dedicated hardware, its application cost is high and deployment is difficult. In addition, TEE technology has limited processing power and high resource consumption, making it difficult to expand to large-scale video data processing tasks and prone to performance bottlenecks.

[0033] FL technology can achieve data privacy protection by training models in different local devices and sending model updates to the server for aggregation without sharing the original data. It can be used in combination with DP technology. However, the privacy protection technology combining FL and DP will increase the training complexity of DNN, reduce the training and communication efficiency of DNN, and thus affect the model accuracy of DNN.

[0034] In view of this, the present disclosure provides a visual privacy protection method for video data, which can be applied to various stages of visual tasks, and can quantitatively describe the complexity of visual information of each I frame in the compressed video data by calculating the privacy metric entropy corresponding to each I frame in the compressed video data, and can specifically encrypt the compressed video data, reduce the consumption of computing resources, improve the efficiency and reliability of visual privacy protection, and improve the data availability of the processed target video data, reduce the impact of data encryption on subsequent application processes, and maintain the model accuracy of DNN. The visual privacy protection method for video data provided by the present disclosure is described in detail below.

[0035] Figure 2 A flowchart of a method for visual privacy protection of video data according to an embodiment of the present disclosure is shown. The method for visual privacy protection of video data can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method for visual privacy protection of video data can be implemented by a processor calling computer-readable instructions stored in a memory. Alternatively, the method for visual privacy protection of video data can be executed by a server. Figure 2 As shown, the visual privacy protection method of the video data includes:

[0036] In step S201, the video data to be processed is compressed to determine compressed video data, wherein the compressed video data includes a plurality of I frames, a plurality of P frames, and a plurality of B frames.

[0037] The video data to be processed here may refer to video data required for training or reasoning of at least one deep neural network. Its specific form may be flexibly set according to actual usage requirements, and the present disclosure does not make any specific limitations on this.

[0038] By using inter-frame analysis technology, the video data to be processed can be compressed to determine the compressed video data. Since only the I frame retains the complete visual information in the compressed video data, and any P frame or B frame only records the difference information between it and the corresponding I frame, the I frame that retains the complete visual information can be encrypted in a targeted manner later, thereby reducing the amount of data that needs to be processed during the encryption process, improving encryption efficiency, and reducing resource consumption for visual privacy protection. Among them, the specific method of compressing the video data to be processed can refer to the implementation method in the relevant technology, and the present disclosure does not make specific limitations on this.

[0039] The process of compressing the video data to be processed and determining the compressed video data will be described in detail later in conjunction with possible implementation methods of the present disclosure, and will not be elaborated here.

[0040] In step S202, for any I frame, the I frame is divided into a plurality of regions according to a preset window scale.

[0041] For any I frame in the compressed video data, different parts in the I frame usually contain different amounts of visual information. Therefore, in order to improve the encryption strength for the part rich in visual information while maintaining the availability of the video data, the I frame can be divided into multiple regions according to a preset window scale, so that the visual information can be analyzed and encrypted at the regional scale later. The specific value of the window scale can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations on this.

[0042] It should be noted that the window scale here is not limited to a single value, but may include multiple different values, so that any I frame can be divided into multiple regions of different sizes according to the window scales of different values, thereby improving the flexibility of region division, improving the privacy protection strength of visual information, and allowing any encrypted I frame to retain more structure and texture information, thereby ensuring that the encrypted target video data maintains data validity.

[0043] In step S203, for any I frame, the privacy metric entropy corresponding to any region in the I frame is determined according to the grayscale histogram corresponding to the region, wherein the privacy metric entropy corresponding to any region is used to describe the complexity of the visual information corresponding to the region.

[0044] For any I frame in the compressed video data, the I frame can be processed to determine the grayscale histogram corresponding to each area in the I frame; the specific method for determining the grayscale histogram here, as well as the specific form of the grayscale histogram, can refer to the implementation method in the relevant technology, and the present disclosure does not make specific limitations on this.

[0045] For any area in the I frame, the privacy measure entropy (PME) corresponding to the area can be calculated based on the grayscale histogram corresponding to the I frame to quantitatively describe the complexity of the visual information in the area, thereby providing real and reliable data support for the subsequent encryption process of the I frame and improving the reliability and flexibility of visual information privacy protection. Specifically, for any area in the I frame, the larger the privacy measure entropy corresponding to the area, the higher the complexity of the visual information in the area, which means that the visual information in the area is easy to identify and extract, and may involve more user privacy information, and it is necessary to improve the intensity of visual information privacy protection for the area.

[0046] The following text will describe in detail the process of determining the privacy metric entropy corresponding to any area in any I frame based on the grayscale histogram corresponding to the area in the I frame in combination with possible implementation methods of the present disclosure, which will not be described in detail here.

[0047] In step S204, for any I frame, the I frame is encrypted according to the privacy metric entropy corresponding to each region in the I frame to determine the encrypted frame corresponding to the I frame.

[0048] For any I frame in the compressed video data, the privacy metric entropy corresponding to each area in the I frame can be used to quantitatively analyze the distribution state of the visual information in the I frame, so as to encrypt the I frame in a targeted manner, flexibly and dynamically adjust the encryption strategy, and determine the encrypted frame corresponding to the I frame, so as to achieve the goal of improving the strength of visual information privacy protection while ensuring the data availability of the encrypted frame corresponding to the I frame.

[0049] The following text will describe in detail the process of encrypting any I frame according to the privacy metric entropy corresponding to each area in the I frame and determining the encrypted frame corresponding to the I frame in combination with possible implementation methods of the present invention, which will not be described in detail here.

[0050] In step S205, target video data for visual privacy protection is determined according to the encrypted frame, multiple P frames, and multiple B frames corresponding to each I frame.

[0051] Since any P frame or B frame in the compressed video data only records the difference information between it and the corresponding I frame, after encrypting each I frame in the compressed video data using the above method, since the visual information in the encrypted frame corresponding to each I frame is protected, indirect privacy protection of each P frame and B frame is achieved. Using the encrypted frame corresponding to each I frame, multiple P frames, and multiple B frames, the target video data with visual privacy protection can be obtained.

[0052] In the disclosed embodiment, by compressing the video data to be processed, the compressed video data including multiple I frames, multiple P frames, and multiple B frames can be determined, so that the amount of data to be processed in the encryption process can be reduced by using the inter-frame analysis technology, the encryption efficiency can be improved, and the resource consumption of visual privacy protection can be reduced. For any I frame, the I frame can be divided into multiple regions according to the preset window scale, and then the grayscale histogram corresponding to any region in the I frame can be used to determine the privacy metric entropy corresponding to the region, so as to quantitatively describe the complexity of the visual information corresponding to the region, and provide real and reliable data support for the subsequent encryption process for the I frame, improve the reliability and flexibility of visual information privacy protection, so as to balance the requirements of visual information privacy protection and maintaining the availability of video data. For any I frame, according to the privacy metric entropy corresponding to each region in the I frame, the I frame can be encrypted in a targeted manner to determine the encrypted frame corresponding to the I frame. Compared with the prior art, there is no need to perform high-cost encryption calculations or high-volume noise injection, which can improve encryption efficiency and reduce resource consumption in the encryption process. According to the encrypted frames, multiple P frames, and multiple B frames corresponding to each I frame, the target video data for visual privacy protection can be determined to ensure the data validity of subsequent target video data in the application process, especially in the model training of deep neural networks, to retain the high efficiency and model accuracy of its training performance. On the other hand, the visual privacy protection method provided by the present disclosure does not rely on specific hardware results or task scenarios, has high versatility, and can be widely used in different visual tasks and various stages of visual tasks.

[0053] In a possible implementation, video data to be processed is compressed to determine compressed video data, including: determining multiple image frames to be compressed in the video data to be processed, and a reference image frame corresponding to each image frame to be compressed; for any image frame to be compressed, dividing the image frame to be compressed and the reference image frame corresponding to the image frame to be compressed into multiple image blocks respectively; for any image frame to be compressed, determining a predicted position of a reference image block corresponding to the image block in the reference image frame corresponding to the image frame to be compressed according to the position of any image block in the image frame to be compressed; determining a coding residual corresponding to the image frame to be compressed according to the position of any image block in the image frame to be compressed and the predicted position of the reference image block corresponding to the image block, wherein the coding residual is used to perform coding compression on the image frame to be compressed based on the reference image frame corresponding to the image frame to be compressed; and performing video compression on the video data to be processed according to the coding residual corresponding to each image frame to be compressed to determine compressed video data.

[0054] Figure 3 FIG. 2 is a schematic diagram showing an inter-frame analysis technique according to an embodiment of the present disclosure. Figure 3 As shown, for any image frame I(t) to be compressed, its adjacent image frame I(t-1) can be determined as the reference image frame corresponding to the image frame I(t) to be compressed. Based on the preset segmentation parameter N, the image frame I(t) to be compressed and the reference image frame I(t-1) can be divided into N×N image blocks respectively. The specific value of the segmentation parameter N can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations on this.

[0055] For any image block Bcurrent in the image frame to be compressed I(t), there is a corresponding reference image block Breference in the reference image frame I(t-1). Through motion estimation and motion compensation for the image block Bcurrent and the reference image block Breference, the temporal redundancy between the image frame to be compressed I(t) and the reference image frame I(t-1) can be utilized, and the predicted position of the reference image block Breference in the reference image frame I(t-1) can be determined according to the position of the image block Bcurrent in the image frame to be compressed I(t), and then the coding residual corresponding to the image frame to be compressed I(t) is determined, and the image frame to be compressed I(t) is encoded as a P frame or a B frame, and the reference image frame I(t-1) is determined as an I frame.

[0056] Specifically, the motion estimation between the image block Bcurrent and the reference image block Breference can be expressed as formula (1):

[0057]

[0058] Among them, E(dx,dy) represents the motion estimation between the image block Bcurrent and the reference image block Breference; (dx,dy) represents the motion vector; (x,y) represents the position of the image block Bcurrent in the image frame to be compressed I(t); i represents the horizontal coordinate of any pixel point in the image block Bcurrent or the reference image block Breference; j represents the vertical coordinate of any pixel point in the image block Bcurrent or the reference image block Breference.

[0059] By minimizing the function E(dx,dy), the position change of the reference image block Breference relative to the image block Bcurrent can be simulated. The position change of the reference image block Breference relative to the image block Bcurrent can be expressed as formula (2):

[0060]

[0061] Among them, (dx * ,dy * ) represents the position change of the reference image block Breference relative to the image block Bcurrent.

[0062] According to the motion vector of the reference image block Breference relative to the image block Bcurrent, motion compensation can be performed based on the position of the image block Bcurrent in the image frame to be compressed I(t), thereby determining the predicted position of the reference image block Breference in the reference image frame I(t-1). The predicted position of the reference image block Breference in the reference image frame I(t-1) can be expressed as formula (3):

[0063]

[0064] Among them, B predicted (x, y) represents the predicted position of the reference image block Breference in the reference image frame I(t-1).

[0065] According to the position of the image block Bcurrent in the image frame to be compressed I(t) and the predicted position of the reference image block Breference in the reference image frame I(t-1), the coding residual corresponding to the image frame to be compressed I(t) can be determined. The coding residual corresponding to the image frame to be compressed I(t) can be expressed by formula (4):

[0066]

[0067] Wherein, R(x,y) represents the coding residual corresponding to the image frame I(t) to be compressed.

[0068] The coding residual corresponding to the image frame to be compressed I(t) can be used to describe the difference information between the image frame to be compressed I(t) and the reference image frame I(t-1), so that the image frame to be compressed I(t) can be encoded as a P frame or a B frame in combination with the specific position of the image frame to be compressed I(t) in the video to be compressed, and the reference image frame I(t-1) can be determined as an I frame. Accordingly, according to the coding residual corresponding to each image frame to be compressed, the video data to be processed can be compressed to determine the compressed video data. Furthermore, the coding residual corresponding to each image frame to be compressed obtained through the above process can be stored in a trusted storage environment to ensure the security of the compressed video data.

[0069] In a possible implementation, for any I frame, the privacy metric entropy corresponding to any area in the I frame is determined according to the grayscale histogram corresponding to the area, including: for any I frame, according to the grayscale histogram corresponding to any area in the I frame, determining the frequency of occurrence corresponding to each grayscale value in the area; determining the privacy metric entropy corresponding to the area according to the frequency of occurrence corresponding to each grayscale value in the area.

[0070] Figure 4 A schematic diagram showing a process of determining a privacy metric entropy according to an embodiment of the present disclosure is shown. Figure 4 As shown, each I frame of the compressed video data can be grayed out and converted into a grayscale image. The specific method of graying out any I frame can refer to the implementation in the related art, and the present disclosure does not make any specific limitation on this.

[0071] In one example, for any I frame, each pixel in the I frame can be converted from RGB color to grayscale. The specific operation process can be expressed as formula (5):

[0072]

[0073] Among them, (x, y) represents any pixel in the I frame; Gray(x, y) represents the gray value corresponding to the pixel (x, y); R(x, y) represents the value of the pixel (x, y) in the red channel (Red); G(x, y) represents the value of the pixel (x, y) in the green channel (Green); B(x, y) represents the value of the pixel (x, y) in the blue channel (Blue).

[0074] For any I frame converted into a grayscale image, a grayscale histogram corresponding to any region in the I frame can be determined, and the frequency of occurrence of each grayscale value in the region can be counted based on the grayscale histogram corresponding to any region in the I frame. For a specific method of determining the frequency of occurrence of each grayscale value in the region based on the grayscale histogram corresponding to any region in the I frame, reference can be made to the implementation in the related art, and the present disclosure does not specifically limit this.

[0075] In one example, for any region in any I frame, the occurrence frequency of any gray value in the region can be expressed as formula (6):

[0076]

[0077] Among them, g i Represents any gray value; h i Represents the gray value g i The number of statistics; N represents the total number of statistics of all different gray values; p i Represents the gray value g i The corresponding frequency of occurrence in this area.

[0078] The frequency of occurrence of each gray value in the area is used to analyze the complexity of the visual information in the area and determine the privacy metric entropy corresponding to the area. For any area in any I frame, its corresponding privacy metric entropy can be expressed as formula (7):

[0079]

[0080] Among them, R j Represents any region in any I frame; H(R j ) represents region R j The corresponding privacy metric entropy; i represents any gray value; p i (j) Indicates that the gray value i is in region R j The frequency of occurrence within.

[0081] Through the above process, the privacy metric entropy of any area in any I frame can be calculated to quantitatively analyze the complexity of the visual information of different parts in the I frame, thereby providing real and reliable data support for the subsequent encryption process of the I frame. Compared with the encryption method of performing encryption calculation on the image or video data as a whole, or indiscriminate noise injection in the prior art, on the one hand, it can reduce the resource consumption of the encryption process and improve the encryption efficiency; on the other hand, it can improve the degree of retention of available visual information in the encrypted frame, meet the two requirements of visual information privacy protection and maintaining the availability of video data, and improve the reliability and flexibility of visual information privacy protection.

[0082] In a possible implementation, for any I frame, the I frame is encrypted according to the privacy metric entropy corresponding to each area in the I frame, and the encrypted frame corresponding to the I frame is determined, including: for any I frame, according to the privacy metric entropy corresponding to each area in the I frame, the standard deviation corresponding to each area is determined; for any I frame, according to the standard deviation corresponding to each area in the I frame, Gaussian blur is performed on each area respectively, and the encrypted frame corresponding to the I frame is determined.

[0083] For any I frame in the compressed video data, according to the privacy metric entropy corresponding to each region in the I frame, a corresponding standard deviation can be set for each region. The specific method for determining the standard deviation corresponding to any region according to the privacy metric entropy corresponding to the region can be flexibly set according to actual usage requirements, and this disclosure does not make specific limitations on this.

[0084] In one example, a mapping relationship between privacy metric entropy and standard deviation can be preset, and based on the mapping relationship, the standard deviation corresponding to any region in any I frame is determined according to the privacy metric entropy corresponding to the region. The specific form of the mapping relationship here can be flexibly set according to actual usage requirements, for example, it can be a linear correlation relationship, etc., and this disclosure does not make specific limitations on this.

[0085] For any region in any I frame, the standard deviation corresponding to the region can be used to perform Gaussian blur on the region, that is, the pixels around each pixel in the region are weighted averaged through the Gaussian function to achieve the effect of smoothing the region and reduce the details and noise of the visual information in the region. The Gaussian function can be expressed as formula (8):

[0086]

[0087] Among them, (x, y) represents any pixel in the area; σ represents the standard deviation corresponding to the area.

[0088] According to formula (8), the strength of Gaussian blurring for the area can be determined by the standard deviation corresponding to the area. The larger the standard deviation, the higher the strength of Gaussian blurring, and the more reliable the visual privacy protection in the area.

[0089] By performing Gaussian blurring of different strengths on each region in any I frame, the I frame can be encrypted and the encrypted frame corresponding to the I frame can be determined.

[0090] In a possible implementation, for any I frame, the I frame is encrypted according to the privacy metric entropy corresponding to each area in the I frame, and the encrypted frame corresponding to the I frame is determined, including: for any I frame, the motion parameters corresponding to each area in the I frame are determined according to the privacy metric entropy corresponding to each area, wherein the motion parameters corresponding to any area include the simulated motion speed, simulated motion direction and simulated motion time corresponding to the area; for any I frame, motion blur is performed on each area according to the motion parameters corresponding to each area in the I frame, and the encrypted frame corresponding to the I frame is determined.

[0091] Motion blur can simulate the blur effect caused by the motion of the image acquisition device or the acquisition object during the image acquisition process. For any I frame in the compressed video data, each area in the I frame can be controlled to simulate motion blur to achieve encryption processing of the I frame.

[0092] Specifically, for any I frame in the compressed video data, according to the privacy metric entropy corresponding to each region in the I frame, corresponding motion parameters can be set for each region respectively, so that each region simulates the motion effect and generates motion blur. Among them, according to the privacy metric entropy corresponding to any region, the specific method of determining the motion parameter corresponding to the region can be flexibly set according to actual usage requirements, and this disclosure does not make specific limitations on this.

[0093] For any area in any I frame, the specific method of performing motion blur on the area according to the motion parameters corresponding to the area can be referred to the implementation method in the relevant technology, and this paper does not make any specific limitation on this.

[0094] In one example, for any region in any I frame, a motion blur model can be constructed according to the motion parameters corresponding to the region to integrate the region and implement motion blur for the region. When the simulated motion direction corresponding to any region is horizontal, the motion blur model corresponding to the region can be expressed as formula (9):

[0095]

[0096] Among them, I blur (x, y) represents any area after motion blur; v represents the simulated motion speed corresponding to the area; T represents the simulated motion time corresponding to the area.

[0097] By performing motion blurring of different intensities on each region in any I frame, the I frame can be encrypted and the encrypted frame corresponding to the I frame can be determined.

[0098] In a possible implementation, for any I frame, the I frame is encrypted according to the privacy metric entropy corresponding to each region in the I frame, and the encrypted frame corresponding to the I frame is determined, including: for any I frame, according to the privacy metric entropy corresponding to each region in the I frame, a pixel interference parameter corresponding to each region is determined, wherein the pixel interference parameter corresponding to any region is used to adjust the position of each pixel in the region in at least one image channel; for any I frame, according to the pixel interference parameter corresponding to each region in the I frame, pixels of the I frame are scrambled to determine the encrypted frame corresponding to the I frame.

[0099] Pixel scrambling can mean that the positions of each pixel in the image are scrambled to eliminate the correlation between adjacent pixels in the image, destroy the connection between visual information, and realize image encryption. For any I frame, according to the privacy metric entropy corresponding to each area in the I frame, high-intensity pixel scrambling can be performed on areas with high visual information complexity, and low-intensity pixel scrambling can be performed on areas with low visual information complexity. Compared with the image encryption method of the prior art that scrambles pixels of images or video frames with a single, fixed intensity, this can balance the needs of visual privacy protection and available information retention.

[0100] Specifically, for any I frame, the pixel interference parameter corresponding to each area can be determined according to the privacy metric entropy corresponding to each area in the I frame, so as to adjust the position of each pixel in the area in at least one image channel. Among them, the specific method for determining the pixel interference parameter corresponding to any area according to the privacy metric entropy corresponding to the area can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations on this. The specific content of the pixel interference parameter can be flexibly set according to actual usage requirements, for example, it can include the image channel targeted by the scrambling operation, the number of pixels whose positions need to be adjusted, etc., and the present disclosure does not make specific limitations on this.

[0101] For any area in any I frame, the pixels in the area can be scrambled according to the pixel interference parameters corresponding to the area, so as to achieve encryption of the area; by scrambling the pixels of each area in the I frame, the I frame can be encrypted, the encrypted frame corresponding to the I frame can be determined, and the visual information in the I frame can be protected.

[0102] In one possible implementation, the method also includes: based on the target video data, performing a performance test on a target deep neural network corresponding to the target video data to determine a convergence speed corresponding to the target deep neural network; when the convergence speed does not meet preset requirements, adjusting the adaptive moment estimation optimizer of the target deep neural network to obtain an optimized deep neural network.

[0103] The target deep neural network here can represent any deep neural network that needs to use target video data for model training. Its specific form can be flexibly set according to actual usage requirements, and this disclosure does not make specific limitations on this.

[0104] When using the target video data to train the target deep neural network model, gradient oscillation may occur, causing the deep neural network to converge more slowly or even fail to converge during the training process. This is especially true in specific application scenarios such as small data sets and single models, which can cause the performance of the target deep neural network to decline significantly.

[0105] In order to ensure that the target deep neural network trained with the target video data maintains the expected convergence ability under gradient oscillation, the target deep neural network corresponding to the target video data can be firstly tested for performance based on the target video data to determine the convergence speed corresponding to the target deep neural network. If the convergence speed corresponding to the target deep neural network does not meet the preset requirements, the adaptive moment estimation optimizer (AdaptiveMoment Estimation, Adam) of the target deep neural network is adjusted to improve the stability and convergence speed of the optimized deep neural network by improving the update rule of Adam.

[0106] Specifically, due to the confusion of the visual information in the target video data caused by the above-mentioned encryption process, the image gradient of the target video data may change drastically within a limited range, resulting in a high possibility that the target deep neural network will fall into a local minimum when Adam of the target deep neural network can adaptively adjust the update step, thereby affecting the normal convergence of the target deep neural network.

[0107] Based on this, the disclosed embodiments can adjust the Adam of the target deep neural network to iterate based on a stable update step size, optimize Adam, obtain a stably updated Adam, reduce the possibility of the target deep neural network falling into a local minimum, and improve the stability and convergence speed of the optimized deep neural network.

[0108] Among them, the specific method of adjusting Adam of the target deep neural network can be flexibly set according to the specific form of Adam of the target deep neural network, and the present disclosure does not make specific limitations on this.

[0109] In one example, Adam of any common deep neural network can adjust the update step size in real time according to the increase of time based on the update steps corresponding to formula (10) and formula (11):

[0110]

[0111] Among them, m t represents the momentum corresponding to the loss function; v t represents the adaptive learning rate; β and γ represent hyperparameters; Represents the momentum corresponding to the adjusted loss function; Represents the adjusted adaptive learning rate.

[0112] The method provided in the embodiment of the present disclosure can adjust Adam to dynamically adjust the learning rate and momentum according to the current gradient and the historical gradient at each iteration, and remove the update steps corresponding to formula (10) and formula (11), thereby improving the stability of the update step size.

[0113] Through the above process, it is possible to optimize the application process of the target video data in the deep neural network, adjust the corresponding target deep neural network mode, improve the availability and applicability of the target video data, and reduce the impact of the encryption of visual information in the target video data on the training performance of the target deep neural network.

[0114] In the disclosed embodiment, by compressing the video data to be processed, the compressed video data including multiple I frames, multiple P frames, and multiple B frames can be determined, so that the amount of data to be processed in the encryption process can be reduced by using the inter-frame analysis technology, the encryption efficiency can be improved, and the resource consumption of visual privacy protection can be reduced. For any I frame, the I frame can be divided into multiple regions according to the preset window scale, and then the grayscale histogram corresponding to any region in the I frame can be used to determine the privacy metric entropy corresponding to the region, so as to quantitatively describe the complexity of the visual information corresponding to the region, and provide real and reliable data support for the subsequent encryption process for the I frame, improve the reliability and flexibility of visual information privacy protection, so as to balance the requirements of visual information privacy protection and maintaining the availability of video data. For any I frame, according to the privacy metric entropy corresponding to each region in the I frame, the I frame can be encrypted in a targeted manner to determine the encrypted frame corresponding to the I frame. Compared with the prior art, there is no need to perform high-cost encryption calculations or high-volume noise injection, which can improve encryption efficiency and reduce resource consumption in the encryption process. According to the encrypted frames, multiple P frames, and multiple B frames corresponding to each I frame, the target video data for visual privacy protection can be determined to ensure the data validity of the subsequent target video data in the application process, especially in the model training of the deep neural network, retaining the high efficiency and model accuracy of its training performance; in addition, the visual privacy protection method provided by the present disclosure can also optimize the deep neural network for the application of the target video data in the deep neural network, further improving the availability and scope of application of the target video data. On the other hand, the visual privacy protection method provided by the present disclosure does not rely on specific hardware results or task scenarios, has high versatility, and can be widely used in different visual tasks and various stages of visual tasks.

[0115] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0116] In addition, the present disclosure also provides a visual privacy protection device for video data, an electronic device, and a computer-readable storage medium, all of which can be used to implement any visual privacy protection method for video data provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

[0117] Figure 5 FIG. 1 is a block diagram of a visual privacy protection device for video data according to an embodiment of the present disclosure. Figure 5 As shown, the device 500 includes:

[0118] The video compression module 501 is used to compress the video data to be processed and determine the compressed video data, wherein the compressed video data includes a plurality of I frames, a plurality of P frames, and a plurality of B frames;

[0119] The frame processing module 502 is used to divide any I frame into multiple regions according to a preset window scale;

[0120] The privacy measurement module 503 is used to determine the privacy measurement entropy corresponding to any region in any I frame according to the grayscale histogram corresponding to the region in the I frame, wherein the privacy measurement entropy corresponding to any region is used to describe the complexity of the visual information corresponding to the region;

[0121] An I frame encryption module 504 is used to encrypt any I frame according to the privacy metric entropy corresponding to each region in the I frame, and determine an encrypted frame corresponding to the I frame;

[0122] The target video data determination module 505 is used to determine the target video data for visual privacy protection according to the encrypted frame, multiple P frames, and multiple B frames corresponding to each I frame.

[0123] In one possible implementation, the privacy measurement module 503 is used to: for any I frame, determine the frequency of occurrence of each grayscale value in any area in the I frame based on the grayscale histogram corresponding to the area; determine the privacy measurement entropy corresponding to the area based on the frequency of occurrence of each grayscale value in the area.

[0124] In one possible implementation, the I frame encryption module 504: for any I frame, according to the privacy metric entropy corresponding to each area in the I frame, determines the standard deviation corresponding to each area; for any I frame, according to the standard deviation corresponding to each area in the I frame, Gaussian blurs each area separately to determine the encrypted frame corresponding to the I frame.

[0125] In one possible implementation, the I frame encryption module 504: for any I frame, according to the privacy metric entropy corresponding to each area in the I frame, determines the motion parameters corresponding to each area, wherein the motion parameters corresponding to any area include the simulated motion speed, simulated motion direction and simulated motion time corresponding to the area; for any I frame, according to the motion parameters corresponding to each area in the I frame, motion blurs each area separately to determine the encrypted frame corresponding to the I frame.

[0126] In one possible implementation, the I frame encryption module 504: for any I frame, according to the privacy metric entropy corresponding to each area in the I frame, determines the pixel interference parameter corresponding to each area, wherein the pixel interference parameter corresponding to any area is used to adjust the position of each pixel in the area in at least one image channel; for any I frame, according to the pixel interference parameter corresponding to each area in the I frame, scrambles the pixels of the I frame to determine the encrypted frame corresponding to the I frame.

[0127] In a possible implementation, the video compression module 501: determines multiple image frames to be compressed in the video data to be processed, and a reference image frame corresponding to each image frame to be compressed; for any image frame to be compressed, divides the image frame to be compressed and the reference image frame corresponding to the image frame to be compressed into multiple image blocks respectively; for any image frame to be compressed, determines the predicted position of a reference image block corresponding to the image block in the reference image frame corresponding to the image frame to be compressed according to the position of any image block in the image frame to be compressed; determines the coding residual corresponding to the image frame to be compressed according to the position of any image block in the image frame to be compressed and the predicted position of the reference image block corresponding to the image block, wherein the coding residual is used to encode and compress the image frame to be compressed based on the reference image frame corresponding to the image frame to be compressed; and performs video compression on the video data to be processed according to the coding residual corresponding to each image frame to be compressed to determine the compressed video data.

[0128] In one possible implementation, the device 500 also includes a deep neural network optimization module, which is used to: perform a performance test on a target deep neural network corresponding to the target video data based on the target video data, and determine a convergence speed corresponding to the target deep neural network; when the convergence speed does not meet preset requirements, adjust the adaptive moment estimation optimizer of the target deep neural network to obtain an optimized deep neural network.

[0129] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0130] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0131] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0132] Figure 6 1900 is a block diagram of an electronic device according to an embodiment of the present disclosure. For example, the apparatus 1900 may be provided as a server or a terminal device. Figure 6 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0133] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0134] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0135] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0136] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the above. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0137] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0138] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0139] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0140] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0141] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0142] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0143] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A visual privacy protection method for video data, characterized in that: include: Compressing the video data to be processed to determine compressed video data, wherein the compressed video data includes a plurality of I frames, a plurality of P frames, and a plurality of B frames; For any I frame, the I frame is divided into multiple regions according to a preset window scale; For any I frame, the privacy metric entropy corresponding to any region in the I frame is determined according to the grayscale histogram corresponding to the region, where the privacy metric entropy corresponding to any region is used to describe the complexity of the visual information corresponding to the region; For any I frame, the I frame is encrypted according to the privacy metric entropy corresponding to each region in the I frame, and the encrypted frame corresponding to the I frame is determined; Target video data for visual privacy protection is determined according to the encrypted frame corresponding to each I frame, the multiple P frames, and the multiple B frames.

2. The method according to claim 1, characterized in that The step of determining, for any I frame, a privacy metric entropy corresponding to any region in the I frame according to a grayscale histogram corresponding to the region includes: For any I frame, determine the occurrence frequency of each gray value in any region in the I frame according to the gray histogram corresponding to the region; According to the occurrence frequency corresponding to each gray value in the area, the privacy metric entropy corresponding to the area is determined.

3. The method according to claim 1 or 2, characterized in that: The method of encrypting any I frame according to the privacy metric entropy corresponding to each region in the I frame and determining an encrypted frame corresponding to the I frame includes: For any I frame, determine the standard deviation corresponding to each area according to the privacy metric entropy corresponding to each area in the I frame; For any I frame, Gaussian blur is performed on each region according to the standard deviation corresponding to each region in the I frame to determine the encrypted frame corresponding to the I frame.

4. The method according to claim 1 or 2, characterized in that: The method of encrypting any I frame according to the privacy metric entropy corresponding to each region in the I frame and determining an encrypted frame corresponding to the I frame includes: For any I frame, determine the motion parameters corresponding to each region according to the privacy metric entropy corresponding to each region in the I frame, wherein the motion parameters corresponding to any region include the simulated motion speed, simulated motion direction and simulated motion time corresponding to the region; For any I frame, motion blur is performed on each region according to the motion parameters corresponding to each region in the I frame to determine the encrypted frame corresponding to the I frame.

5. The method according to claim 1 or 2, characterized in that: The method of encrypting any I frame according to the privacy metric entropy corresponding to each region in the I frame and determining an encrypted frame corresponding to the I frame includes: For any I frame, determine a pixel interference parameter corresponding to each region according to the privacy metric entropy corresponding to each region in the I frame, wherein the pixel interference parameter corresponding to any region is used to adjust the position of each pixel in the region in at least one image channel; For any I frame, pixels of the I frame are shuffled according to the pixel interference parameters corresponding to each region in the I frame to determine the encrypted frame corresponding to the I frame.

6. The method according to claim 1 or 2, characterized in that: The step of compressing the video data to be processed and determining the compressed video data includes: Determining a plurality of image frames to be compressed and a reference image frame corresponding to each image frame to be compressed in the video data to be processed; For any image frame to be compressed, the image frame to be compressed and a reference image frame corresponding to the image frame to be compressed are divided into a plurality of image blocks respectively; For any image frame to be compressed, according to the position of any image block in the image frame to be compressed, determine the predicted position of a reference image block corresponding to the image block in the reference image frame corresponding to the image frame to be compressed; Determine, according to the position of any image block in the image frame to be compressed and the predicted position of the reference image block corresponding to the image block, a coding residual corresponding to the image frame to be compressed, wherein the coding residual is used to perform coding compression on the image frame to be compressed based on the reference image frame corresponding to the image frame to be compressed; The video data to be processed is compressed according to the coding residual corresponding to each image frame to be compressed, so as to determine the compressed video data.

7. The method according to claim 1 or 2, characterized in that: The method further comprises: According to the target video data, a performance test is performed on a target deep neural network corresponding to the target video data to determine a convergence speed corresponding to the target deep neural network; When the convergence speed does not meet the preset requirements, the adaptive moment estimation optimizer of the target deep neural network is adjusted to obtain an optimized deep neural network.

8. A visual privacy protection device for video data, characterized in that: include: A video compression module, used to compress the video data to be processed and determine the compressed video data, wherein the compressed video data includes a plurality of I frames, a plurality of P frames, and a plurality of B frames; A frame processing module, for dividing any I frame into multiple regions according to a preset window scale; A privacy measurement module is used to determine the privacy measurement entropy corresponding to any region in any I frame according to the grayscale histogram corresponding to the region in the I frame, wherein the privacy measurement entropy corresponding to any region is used to describe the complexity of the visual information corresponding to the region; An I frame encryption module is used to encrypt any I frame according to the privacy metric entropy corresponding to each area in the I frame, and determine the encrypted frame corresponding to the I frame; The target video data determination module is used to determine the target video data for visual privacy protection according to the encrypted frame corresponding to each I frame, the multiple P frames, and the multiple B frames.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images