Multi-mode large model driven data desensitization and video data protection method and device

Abstract

The invention provides a data desensitization and video data protection method and device driven by a multi-mode large model. The method comprises the following steps: determining target data to be desensitized based on a video to be desensitized; determining a target desensitization strategy corresponding to the target to-be-desensitized data, and desensitizing the target to-be-desensitized data through the multi-modal large model based on the target desensitization strategy to obtain desensitized data; determining desensitization effect parameters based on the target data to be desensitized and the desensitized data; if the desensitization effect parameter meets the desensitization quality condition, generating a desensitized video corresponding to the video to be desensitized based on the desensitized data, and sending the desensitized video; if the desensitization effect parameter does not meet the desensitization quality condition, the target desensitization strategy is adjusted to obtain an adjusted desensitization strategy, the adjusted desensitization strategy serves as the target desensitization strategy, and desensitization processing is conducted on the target data to be desensitized based on the target desensitization strategy to obtain desensitized data. By means of the scheme, the desensitization strategy can be adjusted in a self-adaptive mode so as to improve the desensitization effect.

Description

Technical Field

[0001] The present application relates to the field of data security, and in particular to a multimodal large model driven data desensitization and video data protection method and device. Background Art

[0002] Data desensitization is a data security technology that aims to transform sensitive information contained in raw data through pre-set rules and algorithms to protect private data. The main purpose of data desensitization is to ensure the privacy and security of sensitive information without affecting the use value of the data. By desensitizing the raw data, data security can be protected and the risk of data leakage can be effectively reduced. However, when desensitizing video data to obtain desensitized videos, the quality of the desensitized videos is usually poor, that is, the desensitization effect is poor, and the desensitized videos cannot reflect the original information of the video data. Summary of the invention

[0003] The present application provides a data desensitization and video data protection method driven by a multimodal large model, the method comprising: determining target data to be desensitized based on an acquired video to be desensitized; determining a target desensitization strategy corresponding to the target data to be desensitized, and performing desensitization processing on the target data to be desensitized based on the target desensitization strategy through a multimodal large model to obtain desensitized data; determining desensitization effect parameters based on the target data to be desensitized and the desensitized data; if the desensitization effect parameters have met the desensitization quality conditions, generating a desensitized video corresponding to the video to be desensitized based on the desensitized data, and sending the desensitized video; if the desensitization effect parameters do not meet the desensitization quality conditions, adjusting the target desensitization strategy to obtain an adjusted desensitization strategy, using the adjusted desensitization strategy as the target desensitization strategy, and returning to perform an operation of performing desensitization processing on the target data to be desensitized based on the target desensitization strategy to obtain desensitized data.

[0004] The present application provides a data desensitization and video data protection device driven by a multimodal large model, the device comprising: a determination module, used to determine target data to be desensitized based on an acquired video to be desensitized; determine a target desensitization strategy corresponding to the target data to be desensitized; a processing module, used to perform desensitization processing on the target data to be desensitized through a multimodal large model based on the target desensitization strategy to obtain desensitized data; the determination module is also used to determine a desensitization effect parameter based on the target data to be desensitized and the desensitized data, the desensitization effect parameter being used to reflect the desensitization quality of the desensitized data; a generation module, used to generate a desensitized video corresponding to the video to be desensitized based on the desensitized data if the desensitization effect parameter has met the desensitization quality condition, and send the desensitized video; the processing module, used to adjust the target desensitization strategy to obtain an adjusted desensitization strategy if the desensitization effect parameter does not meet the desensitization quality condition, use the adjusted desensitization strategy as the target desensitization strategy, and perform desensitization processing on the target data to be desensitized based on the target desensitization strategy to obtain desensitized data.

[0005] The present application provides an electronic device, a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the multimodal large model-driven data desensitization and video data protection method of the above-mentioned example of the present application.

[0006] The present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the multimodal large model-driven data desensitization and video data protection method of the above-mentioned example of the present application.

[0007] The present application provides a machine-readable storage medium, which stores machine-executable instructions that can be executed by a processor; wherein the processor is used to execute the machine-executable instructions, and when the machine-executable instructions are executed, the multimodal large model-driven data desensitization and video data protection method of the above-mentioned example of the present application is implemented.

[0008] As can be seen from the above technical solutions, in the embodiment of the present application, after desensitizing the target data to be desensitized to obtain the desensitized data, the desensitization effect parameters are determined based on the target data to be desensitized and the desensitized data. If the desensitization effect parameters do not meet the desensitization quality conditions, the target desensitization strategy is adjusted to obtain the adjusted desensitization strategy, and the target data to be desensitized is desensitized again based on the adjusted desensitization strategy until the desensitization effect parameters meet the desensitization quality conditions. When the desensitization effect parameters meet the desensitization quality conditions, the desensitized video will be generated, so that the quality of the desensitized video is better, the desensitization effect is better, and the desensitized video can reflect the original information of the video data. In this way, the desensitization effect of the desensitized video can be evaluated, and the desensitization strategy can be adaptively adjusted to improve the desensitization effect, so that the desensitized video has availability. While improving the data security and availability of the desensitized video, the image quality assessment can be more accurately adapted to the dynamic changes of the video data, and the desensitization operation can be evaluated to determine whether it affects the value of the video data in subsequent applications, and the desensitization strategy can be automatically adjusted to achieve the best balance between security and availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a flowchart of the multimodal large model driven data desensitization and video data protection method in this application; Figure 2 It is a flowchart of the multimodal large model driven data desensitization and video data protection method in this application; Figure 3 is a schematic diagram of a sensitive information identification process in one embodiment of the present application; Figure 4 It is a structural schematic diagram of a multimodal large model driven data desensitization and video data protection device in this application; Figure 5 It is a hardware structure diagram of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0010] In the embodiment of the present application, a multi-modal large model driven data desensitization and video data protection method is proposed, which can be applied to electronic devices, see Figure 1 As shown, it is a flow chart of the data desensitization and video data protection method driven by the multimodal large model, and the method may include: Step 101: determine target data to be desensitized based on the acquired videos to be desensitized.

[0011] Step 102: determine the target desensitization strategy corresponding to the target data to be desensitized, and perform desensitization processing on the target data to be desensitized through a multimodal large model based on the target desensitization strategy to obtain desensitized data.

[0012] Step 103: Determine desensitization effect parameters based on the target data to be desensitized and the desensitized data.

[0013] Step 104: Determine whether the desensitization effect parameters have met the desensitization quality conditions.

[0014] If so, that is, the desensitization effect parameter has met the desensitization quality condition, then step 105 is executed.

[0015] If not, that is, the desensitization effect parameter does not meet the desensitization quality condition, step 106 is executed.

[0016] Step 105: Generate a desensitized video corresponding to the video to be desensitized based on the desensitized data, and send the desensitized video, that is, send the desensitized video to the receiving end of the video to be desensitized.

[0017] Step 106, adjust the target desensitization strategy to obtain an adjusted desensitization strategy, use the adjusted desensitization strategy as the target desensitization strategy, and return to execute step 102, that is, re-desensitize the target data to be desensitized based on the target desensitization strategy (that is, the adjusted desensitization strategy) to obtain desensitized data.

[0018] Exemplarily, the target data to be desensitized may include at least two of the image data to be desensitized, the audio data to be desensitized, and the text data to be desensitized. If the target data to be desensitized includes the image data to be desensitized, the audio data to be desensitized, and the text data to be desensitized, the desensitized data includes the desensitized image data, the desensitized audio data, and the desensitized text data. Based on this, the desensitization effect parameters are determined based on the target data to be desensitized and the desensitized data, which may include but are not limited to: based on the image matching degree between the desensitized image data and the image data to be desensitized, the audio matching degree between the desensitized audio data and the audio data to be desensitized, and the text matching degree between the desensitized text data and the text data to be desensitized, determine the feature matching degree. Determine the image quality based on the desensitized image data and the image data to be desensitized. Obtain the desensitization effect parameters, which may include the feature matching degree and the image quality. For example, for step 104, when determining whether the desensitization effect parameters have met the desensitization quality conditions, if the feature matching degree is greater than the first threshold and the image quality is greater than the second threshold, the desensitization effect parameters have met the desensitization quality conditions; if the feature matching degree is not greater than the first threshold, and / or the image quality is not greater than the second threshold, the desensitization effect parameters do not meet the desensitization quality conditions.

[0019] Exemplarily, determining the image quality based on the desensitized image data and the image data to be desensitized may include but is not limited to: determining a first similarity value based on a target brightness weight and a brightness parameter value, determining a second similarity value based on a target contrast weight and a contrast parameter value, and determining a third similarity value based on a target structure weight and a structure parameter value; determining the image quality based on the first similarity value, the second similarity value, and the third similarity value. The brightness parameter value is determined based on the mean pixel value of the desensitized image data and the mean pixel value of the image data to be desensitized, the contrast parameter value is determined based on the pixel value variance of the desensitized image data and the pixel value variance of the image data to be desensitized, and the structure parameter value is determined based on the pixel value variance of the desensitized image data, the pixel value variance of the image data to be desensitized, and the covariance of the pixel values ​​of the desensitized image data and the image data to be desensitized.

[0020] The process of determining the target brightness weight may include: determining an initial brightness weight corresponding to the frame rate of the video to be desensitized; and determining the target brightness weight based on the initial brightness weight. Wherein, if the frame rate is less than the first value, the initial brightness weight is the first brightness weight value; if the frame rate is greater than the second value, the initial brightness weight is the second brightness weight value; if the frame rate is not less than the first value and not greater than the second value, the initial brightness weight is between the second brightness weight value and the first brightness weight value, and the initial brightness weight is inversely proportional to the frame rate.

[0021] The process of determining the target contrast weight may include: determining an initial contrast weight corresponding to the frame rate of the video to be desensitized; and determining the target contrast weight based on the initial contrast weight. Wherein, if the frame rate is less than a third value, the initial contrast weight may be a first contrast weight value; if the frame rate is greater than a fourth value, the initial contrast weight may be a second contrast weight value; if the frame rate is not less than the third value and not greater than the fourth value, the initial contrast weight may be between the second contrast weight value and the first contrast weight value, and the initial contrast weight may be inversely proportional to the frame rate.

[0022] The process of determining the target structure weight may include: determining an initial structure weight corresponding to the frame rate; and determining the target structure weight based on the initial structure weight. Wherein, if the frame rate is less than the fifth value, the initial structure weight is the first structure weight value; if the frame rate is greater than the sixth value, the initial structure weight is the second structure weight value; if the frame rate is not less than the fifth value and not greater than the sixth value, the initial structure weight is between the first structure weight value and the second structure weight value, and the initial structure weight is proportional to the frame rate.

[0023] Exemplarily, determining the target brightness weight based on the initial brightness weight includes: determining the initial brightness weight as the target brightness weight; or adjusting the initial brightness weight based on the degree of motion of the video to be desensitized, and determining the adjusted brightness weight as the target brightness weight; wherein the degree of motion is determined based on the motion vector between adjacent frames in the video to be desensitized; if the degree of motion is less than the first motion threshold, the initial brightness weight is increased, and if the degree of motion is greater than the second motion threshold, the initial brightness weight remains unchanged. Determining the target contrast weight based on the initial contrast weight includes: determining the initial contrast weight as the target contrast weight; or adjusting the initial contrast weight based on the degree of motion, and determining the adjusted contrast weight as the target contrast weight; if the degree of motion is less than the first motion threshold, the initial contrast weight remains unchanged; if the degree of motion is greater than the second motion threshold, the initial contrast weight is increased. Determining the target structure weight based on the initial structure weight may include: determining the initial structure weight as the target structure weight; or adjusting the initial structure weight based on the degree of motion, and determining the adjusted structure weight as the target structure weight; wherein, if the degree of motion is less than the first motion threshold, the initial structure weight remains unchanged; if the degree of motion is greater than the second motion threshold, the initial structure weight is increased.

[0024] Exemplarily, if the degree of motion is not less than a first motion threshold and not greater than a second motion threshold, the initial brightness weight, initial contrast weight, and initial structure weight may be adjusted so that the difference between the adjusted brightness weight, adjusted contrast weight, and adjusted structure weight is not greater than the threshold.

[0025] Exemplarily, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, desensitizing the target data to be desensitized based on the target desensitization strategy to obtain desensitized data may include but is not limited to: extracting features from the image data to be desensitized through a multimodal large model to obtain image features, extracting features from the audio data to be desensitized through a multimodal large model to obtain audio features, extracting features from the text data to be desensitized through a multimodal large model to obtain text features, and fusing image features, audio features, and text features through a multimodal large model to obtain multimodal features; based on a multimodal large model, The modal features determine the image sensitive positions of the image data to be desensitized, the audio sensitive positions of the audio data to be desensitized, and the text sensitive positions of the text data to be desensitized through a multimodal large model; based on the image desensitization strategy corresponding to the image data to be desensitized, the image sensitive positions of the image data to be desensitized are desensitized to obtain desensitized image data; based on the audio desensitization strategy corresponding to the audio data to be desensitized, the audio sensitive positions of the audio data to be desensitized are desensitized to obtain desensitized audio data; based on the text desensitization strategy corresponding to the text data to be desensitized, the text sensitive positions of the text data to be desensitized are desensitized to obtain desensitized text data.

[0026] Exemplarily, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, the target desensitization strategy includes an image desensitization strategy, an audio desensitization strategy, and a text desensitization strategy. Determining the target desensitization strategy corresponding to the target data to be desensitized may include, but is not limited to: querying the configured desensitization rule library through the multimodal attributes corresponding to the video to be desensitized, and obtaining the image desensitization strategy corresponding to the multimodal attributes; wherein the multimodal attributes may include at least one of the image attributes, the audio attributes, and the text attributes, and the desensitization rule library may include the correspondence between the multimodal attributes and the image desensitization strategy; wherein the image desensitization strategy includes an image desensitization network model. Determine the audio desensitization strategy based on the audio attributes corresponding to the video to be desensitized, and the audio desensitization strategy may include an audio desensitization network model and a vocoder, the audio desensitization network model is used to convert the audio data to be desensitized into acoustic features, and the vocoder is used to convert the acoustic features into audio waveforms. Determine the configured text desensitization strategy. The text desensitization strategy includes a text encryption algorithm or a text replacement algorithm. The text encryption algorithm is used to encrypt the desensitized text data, and the text replacement algorithm is used to replace or obfuscate the desensitized text data.

[0027] Among them, when the target desensitization strategy is adjusted to obtain the adjusted desensitization strategy, if the image desensitization strategy needs to be adjusted, the network parameters in the image desensitization network model are adjusted; if the audio desensitization strategy needs to be adjusted, the network parameters in the audio desensitization network model are adjusted; if the text desensitization strategy needs to be adjusted, the algorithm type adopted by the text encryption algorithm is adjusted, or the algorithm type adopted by the text replacement algorithm can be adjusted.

[0028] As can be seen from the above technical solutions, in the embodiment of the present application, after desensitizing the target data to be desensitized to obtain the desensitized data, the desensitization effect parameters are determined based on the target data to be desensitized and the desensitized data. If the desensitization effect parameters do not meet the desensitization quality conditions, the target desensitization strategy is adjusted to obtain the adjusted desensitization strategy, and the target data to be desensitized is desensitized again based on the adjusted desensitization strategy until the desensitization effect parameters meet the desensitization quality conditions. When the desensitization effect parameters meet the desensitization quality conditions, the desensitized video will be generated, so that the quality of the desensitized video is better, the desensitization effect is better, and the desensitized video can reflect the original information of the video data. In this way, the desensitization effect of the desensitized video can be evaluated, and the desensitization strategy can be adaptively adjusted to improve the desensitization effect, so that the desensitized video has availability. While improving the data security and availability of the desensitized video, the image quality assessment can be more accurately adapted to the dynamic changes of the video data, and the desensitization operation can be evaluated to determine whether it affects the value of the video data in subsequent applications, and the desensitization strategy can be automatically adjusted to achieve the best balance between security and availability.

[0029] The above technical solutions of the embodiments of the present application are described below in combination with specific application scenarios.

[0030] In an embodiment of the present application, a multimodal large model-driven data desensitization and video data protection method is proposed, which can be applied to an electronic device. The electronic device can be the video transmitter itself, or it can be a device between the video transmitter and the video receiver (such as a network device, a security device, etc.). There is no restriction on the type of the electronic device.

[0031] See also Figure 2 As shown, it is a flowchart of a multimodal large model driven data desensitization and video data protection method, which may include: Step 201: determine target data to be desensitized based on the acquired video to be desensitized.

[0032] Exemplarily, the target data to be desensitized may include at least two of image data to be desensitized, audio data to be desensitized, and text data to be desensitized. For example, the image data to be desensitized may be an image to be desensitized, the audio data to be desensitized may be audio to be desensitized, and the text data to be desensitized may be text to be desensitized.

[0033] After obtaining the video to be desensitized, since the video to be desensitized is composed of a large number of images, all the images of the video to be desensitized can be used as the image data to be desensitized, or part of the images of the video to be desensitized can be used as the image data to be desensitized. In this way, the image data to be desensitized can be obtained from the video to be desensitized.

[0034] After obtaining the video to be desensitized, since the video to be desensitized includes sound data, and the sound data is the audio data to be desensitized, the audio data to be desensitized can be obtained from the video to be desensitized.

[0035] After obtaining the video to be desensitized, since the video to be desensitized includes text data (such as subtitle information, barrage information, text description information, etc., and the type of this text data is not limited), and the text data is the text data to be desensitized, the text data to be desensitized can be obtained from the video to be desensitized. For example, natural language processing technology can be used to extract the text data to be desensitized from the video to be desensitized.

[0036] In summary, it can be seen that the image data to be desensitized, the audio data to be desensitized, and the text data to be desensitized can be obtained from the video to be desensitized, and the target data to be desensitized can include at least two of the image data to be desensitized, the audio data to be desensitized, and the text data to be desensitized. For example, the target data to be desensitized can include the image data to be desensitized, the audio data to be desensitized, and the text data to be desensitized at the same time.

[0037] Step 202: sensitive information is identified based on the target data to be desensitized, and the sensitive position of the target data to be desensitized is obtained. For example, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, sensitive information is identified based on the image data to be desensitized, audio data to be desensitized, and text data to be desensitized, and the sensitive position of the image data to be desensitized, the sensitive position of the audio data to be desensitized, and the sensitive position of the text data to be desensitized are obtained. If the target data to be desensitized includes image data to be desensitized and audio data to be desensitized, sensitive information is identified based on the image data to be desensitized and audio data to be desensitized, and the sensitive position of the image data to be desensitized and the sensitive position of the audio data to be desensitized are obtained, and so on.

[0038] For the convenience of description, the following example takes the target data to be desensitized as image data to be desensitized, audio data to be desensitized, and text data to be desensitized as an example. In this way, the sensitive information identification process may include the following steps: Step S11, extracting features of the image data to be desensitized to obtain image features.

[0039] For example, the image data to be desensitized may include multiple images to be desensitized. The image data to be desensitized may be input into the image feature extraction network of the multimodal large model. The image feature extraction network is used to extract features from the image data to be desensitized to obtain image features. The image features may include but are not limited to texture features, shape features, color features, and spatial layout features. In this embodiment, there is no restriction on the type of image features.

[0040] For example, the image feature extraction network can be a ResNet (Residual Network) network, or other types of networks can be used as long as they have the image feature extraction function. ResNet is used as an example for explanation. ResNet can use a custom activation function (such as LeakyReLU) and add a Batch Normalization layer to accelerate network convergence and improve generalization ability. In the network structure of ResNet, the convolution kernel size and step size of some convolution layers are adjusted to adapt to the resolution and detail features of different video frames. In the network structure of ResNet, through multi-layer convolution operations, convolution kernels of different sizes (such as 3*3, 5*5, etc.) are used to extract multi-scale features of the image, and then through pooling operations (such as maximum pooling or average pooling, etc.) and residual connections, the image features of the video frame are effectively extracted. In this way, the image data to be desensitized can be feature extracted to obtain image features. In the network structure of ResNet, for high-resolution videos, the depth of the convolution layer can be increased, and a smaller convolution kernel can be used to better capture the subtle texture of the image data to be desensitized. For low-resolution videos, the convolution kernel and step size can be appropriately increased to improve the feature extraction efficiency of the number of images to be desensitized.

[0041] Step S12: extracting features from the audio data to be desensitized to obtain audio features.

[0042] For example, the audio data to be desensitized can be input into the audio feature extraction network of the multimodal large model, and the audio feature extraction network can be used to extract features from the audio data to be desensitized to obtain audio features (also called audio feature vectors). The audio features may include but are not limited to frequency features, amplitude features, harmonic structure features, and speech rhythm features. In this embodiment, there is no restriction on the type of audio features.

[0043] For example, the audio feature extraction network uses MFCC (Mel Frequency Cepstrum Coefficient) network and DNN (Deep Neural Networks), or other types of networks with audio feature extraction function. MFCC network and DNN network are used as examples for explanation. Based on this, the audio data to be desensitized can be input into the MFCC network, and the audio data to be desensitized (i.e., audio signal) can be decomposed into multiple Mel frequency bands through the MFCC network to obtain frequency domain features. Then, the frequency domain features are input into the DNN network, and the DNN network extracts features based on the frequency domain features to obtain audio features, thereby converting the audio signal into audio features with semantic and structural information. In the network structure of the DNN network, 5 fully connected layers can be used, and the number of nodes in each fully connected layer is 256, 128, 64, 32, and 16 respectively. For the training process of the DNN network, the stochastic gradient descent algorithm can be used to train the DNN network, and the Dropout technology can be used to prevent overfitting. Through the training of the DNN network, the DNN network can capture the frequency characteristics, amplitude characteristics, harmonic structure characteristics and speech rhythm characteristics of the audio, so that feature extraction can be performed to obtain audio features. When extracting features from the audio data to be desensitized through the audio feature extraction network to obtain audio features, for audio data to be desensitized in different audio encoding formats (such as MP3, WAV, etc.), the audio data to be desensitized can be decoded into a unified audio signal format, and then feature extraction can be performed through the audio feature extraction network. Before the audio data to be desensitized is input into the audio feature extraction network, a noise reduction algorithm (such as Wiener filtering) can also be used to process the noise interference of the audio data to be desensitized to ensure the stability of feature extraction.

[0044] Step S13: extracting features from the text data to be desensitized to obtain text features.

[0045] For example, the text data to be desensitized can be input into the text feature extraction network of the multimodal large model, and the text feature extraction network can be used to extract features from the text data to be desensitized to obtain text features (also called text feature vectors). The text features can be high-dimensional semantic features, and there is no restriction on the type of text features.

[0046] For example, the text feature extraction network can use the Transformer network (such as the Transformer-based encoder). The core of the Transformer network is the multi-head attention mechanism. Other types of networks can also be used as long as they have the function of text feature extraction. The Transformer network is used as an example for explanation. When the Transformer network is used to process the desensitized text data, the multi-head attention mechanism divides the desensitized text data into multiple heads, and each head focuses on different semantic dimensions. For example, one head focuses on lexical semantics, and the other head processes the syntactic structure. Through parallel processing and aggregation, deep semantic encoding of the text is achieved. At the same time, position encoding is used to process the sequential information of the text, convert the text information into word vector representation, and mine the multi-dimensional information of the text such as lexical semantics, syntactic structure, semantic relationship, and paragraph semantics, and finally convert the text into a high-dimensional semantic feature representation, that is, text features. When the Transformer network is used to process the desensitized text data, for the desensitized text data in different languages, pre-trained word embedding (such as Word2Vec, etc.) can also be used to adjust the embedding matrix according to the language characteristics. In addition, for long text data to be desensitized, methods such as truncation or truncation plus padding can be used to ensure that the input length of the text data to be desensitized remains consistent.

[0047] Step S14: Fusing the image feature, the audio feature and the text feature to obtain a multimodal feature, and determining the image sensitive position (i.e., sensitive area) of the image data to be desensitized, the audio sensitive position of the audio data to be desensitized, and the text sensitive position of the text data to be desensitized based on the multimodal feature.

[0048] For example, image features, audio features, and text features are input into a multimodal large model (such as a fusion network of a multimodal large model, which may also be called a fusion layer of a multimodal large model), and the multimodal large model is used to fuse the image features, audio features, and text features to obtain multimodal features. The multimodal large model is used to determine the image sensitive positions of the image data to be desensitized, the audio sensitive positions of the audio data to be desensitized, and the text sensitive positions of the text data to be desensitized based on the multimodal features.

[0049] For example, a large multimodal model may include a fusion layer. The fusion layer of the large multimodal model adopts an Attention network with an attention mechanism (such as an Attention network with a multi-head cross-modal attention mechanism). Image features, audio features, and text features can be input into the fusion layer, and these features are fused through the fusion layer to obtain multimodal features. Multimodal features are also called multimodal joint features or multimodal fusion features.

[0050] For the fusion layer of the multimodal large model, by calculating the similarity matrix between different modal features (such as image features, audio features, and text features), dynamically assigning weights according to the similarity matrix, and adaptively learning the dynamic association weights between different modal features in a data-driven manner. Then, based on the dynamic association weights between different modal features, image features, audio features, and text features can be fused.

[0051] In the process of processing the fusion layer of the multimodal large model, the following formula can also be used for processing: In the above formula, Q is the query vector, K is the key vector, and V is the value vector. is the dimension of the key vector. In this embodiment, there is no restriction on the processing method of the fusion layer, as long as the fusion layer can fuse the image features, audio features and text features.

[0052] For example, the multimodal large model may include a decision layer (also called the decision network of the multimodal large model), and the multimodal features may be input into the decision layer of the multimodal large model, so as to accurately locate the image sensitive positions of the image data to be desensitized, the audio sensitive positions of the audio data to be desensitized, and the text sensitive positions of the text data to be desensitized.

[0053] For example, the decision layer of the multimodal large model can use a target detection network (such as the Faster R-CNN network based on deep learning), a speech recognition network (such as an end-to-end speech recognition network based on deep learning), and a text classification network (such as a text classifier based on a convolutional neural network). The input data of the target detection network is a multimodal feature. The target detection network screens and refines the candidate areas of the image data to be desensitized (i.e., video images) based on the multimodal features, thereby accurately locating the sensitive information area in the image data to be desensitized, and the sensitive information area is the image sensitive position of the image data to be desensitized. In this embodiment, there is no restriction on the processing process of the target detection network.

[0054] The input data of the speech recognition network is a multimodal feature. The speech recognition network identifies the sensitive speech content in the audio data to be desensitized (i.e., audio) based on the multimodal feature, thereby accurately locating the sensitive speech content in the audio data to be desensitized, and the location of the sensitive speech content is the audio sensitive position of the audio data to be desensitized. In this embodiment, there is no restriction on the processing process of the speech recognition network. Based on the audio sensitive position, the target detection network can also locate the image position corresponding to the audio sensitive position from the image data to be desensitized, and this image position is also the image sensitive position of the image data to be desensitized.

[0055] The input data of the text classification network is a multimodal feature. The text classification network identifies sensitive information in the text data to be desensitized (i.e., text) based on the multimodal feature, thereby accurately locating sensitive information in the text data to be desensitized, and the location of the sensitive information is the text sensitive location of the text data to be desensitized. In this embodiment, there is no restriction on the processing process of the text classification network. Based on the text sensitive location, the target detection network can also locate the image location corresponding to the text sensitive location from the image data to be desensitized, and this image location is also the image sensitive location of the image data to be desensitized.

[0056] For example, when it is recognized that specific sensitive words are mentioned in the conversation between characters in the video, the image sensitive positions of the image data to be desensitized, the audio sensitive positions of the audio data to be desensitized, and the text sensitive positions of the text data to be desensitized are comprehensively determined based on the facial expressions of the characters in the image and the voice intonation of the audio.

[0057] In one possible implementation, see Figure 3 The figure shows a schematic diagram of the sensitive information identification process. The image data to be desensitized is input into the image processing module (image feature extraction network), and the image processing module extracts features from the image data to be desensitized to obtain image features. The audio data to be desensitized is input into the audio processing module (audio feature extraction network), and the audio processing module extracts features from the audio data to be desensitized to obtain audio features. The text data to be desensitized is input into the text processing module (text feature extraction network), and the text processing module extracts features from the text data to be desensitized to obtain text features.

[0058] Then, the image features, audio features and text features are input into the fusion layer of the multimodal large model, and the image features, audio features and text features are fused through the fusion layer to obtain multimodal features.

[0059] Then, the multimodal features are input into the decision layer of the multimodal large model, and the sensitive positions (such as image sensitive positions of image data to be desensitized, audio sensitive positions of audio data to be desensitized, and text sensitive positions of text data to be desensitized) are located through the decision layer, and the sensitive positions are used for subsequent desensitization processing.

[0060] At this point, step 202 is completed, and the sensitive location of the target data to be desensitized is obtained.

[0061] Step 203: determine the target desensitization strategy corresponding to the target data to be desensitized, and perform desensitization processing on the target data to be desensitized based on the target desensitization strategy to obtain desensitized data.

[0062] For example, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, then the image desensitization strategy corresponding to the image data to be desensitized, the audio desensitization strategy corresponding to the audio data to be desensitized, and the text desensitization strategy corresponding to the text data to be desensitized can be determined. Based on this, the image data to be desensitized is desensitized based on the image desensitization strategy to obtain desensitized image data. Based on the audio desensitization strategy, the audio data to be desensitized is desensitized to obtain desensitized audio data. Based on the text desensitization strategy, the text data to be desensitized is desensitized to obtain desensitized text data.

[0063] If the target data to be desensitized includes image data to be desensitized and audio data to be desensitized, then determine the image desensitization strategy corresponding to the image data to be desensitized and the audio desensitization strategy corresponding to the audio data to be desensitized. Desensitize the image data to be desensitized based on the image desensitization strategy to obtain desensitized image data. Desensitize the audio data to be desensitized based on the audio desensitization strategy to obtain desensitized audio data, and so on.

[0064] For the convenience of description, the following example takes the target data to be desensitized as image data to be desensitized, audio data to be desensitized, and text data to be desensitized as an example. In this way, the desensitization process may include the following steps: Step S21, determining the image desensitization strategy corresponding to the image data to be desensitized.

[0065] Exemplarily, an image desensitization strategy may be preconfigured, and after the image data to be desensitized is obtained, the configured image desensitization strategy may be determined, that is, all image data to be desensitized correspond to the same image desensitization strategy.

[0066] Exemplarily, a desensitization rule library may be preconfigured, and the desensitization rule library may include a correspondence between multimodal attributes and image desensitization strategies. See Table 1 for an example of a desensitization rule library.

[0067] Table 1

[0068] Regarding the correspondence between the multimodal attributes and the image desensitization strategy, the correspondence can be configured based on experience, or a certain algorithm can be used to determine the correspondence, and there is no restriction on this. After the desensitization rule base is configured, the desensitization rule base can remain unchanged, and the desensitization rule base can also be updated regularly.

[0069] After obtaining the image data to be desensitized, the desensitization rule library shown in Table 1 can be queried through the multimodal attribute corresponding to the video to be desensitized to obtain the image desensitization strategy corresponding to the multimodal attribute. For example, if the multimodal attribute corresponding to the video to be desensitized is the multimodal attribute a1, then by querying the desensitization rule library shown in Table 1, the image desensitization strategy corresponding to the multimodal attribute a1 is the image desensitization strategy b1.

[0070] For the multimodal attributes corresponding to the video to be desensitized, the multimodal attributes may include but are not limited to at least one of image attributes, audio attributes and text attributes. For example, image attributes are used to describe the attributes of the image data to be desensitized in the video to be desensitized, such as image resolution, color depth, video frame rate, etc., and there is no restriction on the type of this image attribute. Audio attributes are used to describe the attributes of the audio data to be desensitized in the video to be desensitized, such as audio sampling rate, etc., and there is no restriction on the type of this audio attribute. Text attributes are used to describe the attributes of the text data to be desensitized in the video to be desensitized, and there is no restriction on the type of this text attribute.

[0071] Based on the multimodal attributes corresponding to the video to be desensitized, the desensitization rule library can be queried to obtain the image desensitization strategy corresponding to the image data to be desensitized, thereby generating a precisely adapted image desensitization strategy.

[0072] Exemplarily, the image desensitization strategy may include an image desensitization network model (such as using a replacement model based on deep learning as the image desensitization network model, which can realize the region replacement function, that is, replacing a sensitive area in the image to realize the desensitization function), that is, using a certain network model as an image desensitization strategy. Since the network model is used to realize image desensitization, it can also be called an image desensitization network model. Of course, the image desensitization strategy can also be other types of desensitization strategies, which are not limited to this.

[0073] For example, for multimodal attribute a1, type 1 network model can be used as image desensitization strategy b1, for multimodal attribute a2, type 2 network model can be used as image desensitization strategy b2, and so on. On this basis, for each multimodal attribute, a certain type of network model can be configured as an image desensitization strategy, and different multimodal attributes can correspond to the same or different types of network models.

[0074] For example, if it is determined based on multimodal attributes that the image resolution is higher than 1080p and the background of the facial area is simple, then the image desensitization strategy (i.e., image desensitization network model) corresponding to the multimodal attributes can be the StyleGAN model in the Generative Adversarial Network (GAN) technology. Of course, the StyleGAN model is just an example.

[0075] Step S22: after obtaining the image desensitization strategy corresponding to the image data to be desensitized, desensitize the image sensitive positions of the image data to be desensitized based on the image desensitization strategy to obtain desensitized image data.

[0076] For example, the sensitive positions in the image data to be desensitized can be called image sensitive positions, and the image sensitive positions can be sensitive areas, such as rectangular areas, circular areas, irregular areas, etc. On this basis, the image sensitive positions can be desensitized, such as generating an image for the image sensitive positions (the image desensitization network model corresponding to the image data to be desensitized is used to generate this image), so that the image can be used to replace the image sensitive positions of the image data to be desensitized, thereby obtaining the desensitized image data. For example, taking the image desensitization network model as the StyleGAN model, the StyleGAN model can generate an image for the image sensitive positions, thereby using the image to replace the image sensitive positions of the image data to be desensitized.

[0077] For example, when using the StyleGAN model, images are generated for sensitive locations of images by screening and matching from a pre-trained rich database based on features such as age, gender, and expression. First, the cosine similarity of the feature vectors of the image sensitive location is calculated with each feature vector in the database, and a set of feature vectors with the highest similarity is selected. Then, based on the feature vector with the highest similarity, a new image is generated using the generator, which is used to replace the image sensitive location of the image data to be desensitized. During the replacement process, the Poisson fusion algorithm or multi-band fusion algorithm in image fusion technology is used to adjust the details such as lighting and shadows of the replaced image. The Poisson fusion algorithm solves the gradient field according to the Laplace equation to ensure a smooth transition of the fusion boundary. The multi-band fusion algorithm decomposes the image into bands of different frequencies and fuses them according to the weights of different frequencies, so that the replaced image blends naturally and smoothly with the background to achieve a desensitization effect.

[0078] Step S23: determine the audio desensitization strategy corresponding to the audio data to be desensitized.

[0079] Exemplarily, an audio desensitization strategy may be preconfigured, and after obtaining the audio data to be desensitized, the configured audio desensitization strategy may be determined, that is, all audio data to be desensitized correspond to the same audio desensitization strategy.

[0080] Exemplarily, a desensitization database may be preconfigured, and the desensitization database may include a correspondence between audio attributes and audio desensitization strategies. See Table 2 for an example of a desensitization database.

[0081] Table 2

[0082] Regarding the correspondence between the audio attribute and the audio desensitization strategy, the correspondence can be configured based on experience, or a certain algorithm can be used to determine the correspondence, and there is no restriction on this. After the desensitization database is configured, the desensitization database can remain unchanged, and the desensitization database can also be updated regularly.

[0083] After obtaining the audio data to be desensitized, the audio desensitization strategy can be determined based on the audio attributes corresponding to the video to be desensitized (i.e., the audio attributes corresponding to the audio data to be desensitized), such as querying the desensitization database shown in Table 2 through the audio attributes corresponding to the audio data to be desensitized, and obtaining the audio desensitization strategy corresponding to the audio attribute. For example, if the audio attribute corresponding to the audio data to be desensitized is the audio attribute c1, then by querying the desensitization database shown in Table 2, the audio desensitization strategy corresponding to the audio attribute c1 is the audio desensitization strategy d1. For the audio attributes corresponding to the audio data to be desensitized, the audio attributes are used to describe the attributes of the audio data to be desensitized in the video to be desensitized, for example, the audio attribute is used to indicate that the audio is a single voice and the speaking speed is slow, the audio attribute is used to indicate that the audio is a single voice and the speaking speed is fast, the audio attribute is used to indicate that the audio is multiple voices and the speaking speed is slow, and the audio attribute is used to indicate that the audio is multiple voices and the speaking speed is fast.

[0084] Exemplarily, the audio desensitization strategy may include an audio desensitization network model and a vocoder, the audio desensitization network model is used to convert the audio data to be desensitized into acoustic features, and the vocoder is used to convert the acoustic features into an audio waveform, thereby desensitizing the audio data to be desensitized through the audio waveform. Based on this, a certain network model and vocoder can be used as an audio desensitization strategy. Since the network model is used to achieve audio desensitization, it can also be called an audio desensitization network model. For example, for audio attribute c1, a type 3 network model (audio desensitization network model) and a vocoder are used as an audio desensitization strategy d1, and for audio attribute c2, a type 4 network model and a vocoder are used as an audio desensitization strategy d2, and so on. On this basis, for each audio attribute, a certain type of network model (audio desensitization network model) and a vocoder can be configured as an audio desensitization strategy, and different audio attributes correspond to the same or different types of network models.

[0085] For example, if the audio (audio data to be desensitized) is determined to be a single voice with a slow speaking speed based on audio attributes, the audio desensitization strategy corresponding to the audio attributes can be the Tacotron model and WaveNet vocoder based on deep learning, and the Tacotron model can represent the audio desensitization network model.

[0086] Step S24: After obtaining the audio desensitization strategy corresponding to the audio data to be desensitized, desensitize the audio sensitive position of the audio data to be desensitized based on the audio desensitization strategy to obtain the desensitized audio data.

[0087] For example, the sensitive position in the audio data to be desensitized can be called an audio sensitive position. On this basis, the audio sensitive position can be desensitized, such as synthesizing an audio waveform for the audio sensitive position (the audio desensitization network model and the vocoder are used to generate this audio waveform), so that the audio waveform can be used to replace the audio sensitive position of the audio data to be desensitized, thereby obtaining the desensitized audio data.

[0088] For example, taking the audio desensitization strategy as the Tacotron model and the WaveNet vocoder as an example, the Tacotron model is used to convert the audio data to be desensitized into acoustic features. The audio features of the audio data to be desensitized (see step S12) and the text features of the text data to be desensitized (see step S13) are used as input data of the Tacotron model. The Tacotron model processes the audio features and text features through an encoder, a decoder, and an attention mechanism network to obtain the acoustic features corresponding to the audio data to be desensitized, and inputs the acoustic features to the WaveNet vocoder, which converts the acoustic features into an audio waveform. Then, the audio waveform can be used to replace the audio sensitive position of the audio data to be desensitized to obtain the desensitized audio data.

[0089] During the processing of the Tacotron model, the timbre, intonation, speech rate and other key parameters of the synthesized audio (i.e., audio waveform) can be precisely controlled through parameter adjustment and optimization of the Tacotron model according to the audio features of the audio data to be desensitized (such as fundamental frequency, harmonic structure and speech rhythm). The mean square error (MSE) loss function and perceptual loss function can be used to optimize the Tacotron model to ensure that the synthesized audio is highly coordinated and consistent with the video image and text information in terms of semantics, emotion and time axis, so as to achieve seamless connection and natural transition. For different languages ​​and accents, the parameters and structure of the Tacotron model can be adjusted by adding training data of specific languages ​​and accents to improve the accuracy of speech synthesis.

[0090] Step S25: Determine the text desensitization strategy corresponding to the text data to be desensitized.

[0091] Exemplarily, a text desensitization strategy may be pre-configured, and after obtaining the text data to be desensitized, the configured text desensitization strategy may be determined, that is, all the text data to be desensitized correspond to the same text desensitization strategy.

[0092] For example, the text desensitization strategy can be a text encryption algorithm, and the text encryption algorithm is used to perform encryption operations on the text data to be desensitized. For example, the text encryption algorithm is configured as the text desensitization strategy, so that the text desensitization strategy corresponding to the text data to be desensitized is determined to be the text encryption algorithm. The algorithm type of the text encryption algorithm can be the block encryption mode of the AES-256 encryption algorithm (such as CBC mode), the padding mode of the AES-256 encryption algorithm (such as PKCS7 mode), the DES encryption algorithm, etc., without restriction.

[0093] For example, the text desensitization strategy may be a text replacement algorithm, and the text replacement algorithm is used to perform a replacement operation or a fuzzification operation on the text data to be desensitized. For example, a text replacement algorithm is configured as a text desensitization strategy, so that the text desensitization strategy corresponding to the text data to be desensitized is determined to be a text replacement algorithm. The algorithm type of the text replacement algorithm may be a specific symbol sequence replacement algorithm, a fuzzification replacement algorithm, and the like.

[0094] Step S26: After obtaining the text desensitization strategy corresponding to the text data to be desensitized, desensitize the text sensitive positions of the text data to be desensitized based on the text desensitization strategy to obtain desensitized text data.

[0095] For example, the sensitive positions in the text data to be desensitized can be called text sensitive positions, and the text sensitive positions can be desensitized. If the text desensitization strategy is a text encryption algorithm, the text encryption algorithm can be used to encrypt the text at the text sensitive position to obtain the encrypted content, and the encrypted content is used to replace the text at the text sensitive position to obtain desensitized text data. Alternatively, if the text desensitization strategy is a text replacement algorithm, a replacement content can be generated based on the text replacement algorithm, and the replacement content is used to replace the text at the text sensitive position to obtain desensitized text data.

[0096] For example, taking the block encryption mode of the AES-256 encryption algorithm as an example, a hardware random number generator can be used to generate the initial key, and then the final 256-bit key can be generated through the hash function and key expansion algorithm to ensure the randomness, uniqueness and security of the key. During the encryption process, the text in the text-sensitive position is encrypted using the block encryption mode based on the 256-bit key. For example, taking the text replacement algorithm (rule-based replacement algorithm) as an example, the text in the text-sensitive position can be replaced with a specific symbol sequence or fuzzy processing. The replacement rules are dynamically and intelligently adjusted according to the video scene where the text is located and the degree of association with other modal information. For example, in videos involving the medical field, for sensitive medical terms, based on the medical knowledge graph and multimodal information, field-specific synonym replacement or fuzzification strategies are used to retain the text semantic information and context relevance to the greatest extent.

[0097] At this point, step 203 is completed, and the target data to be desensitized can be desensitized to obtain desensitized data.

[0098] Step 204: Determine desensitization effect parameters based on the target data to be desensitized and the desensitized data.

[0099] For example, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, the desensitization effect parameters are determined based on the image data to be desensitized and the desensitized image data, the audio data to be desensitized and the desensitized audio data, the text data to be desensitized and the desensitized text data. If the target data to be desensitized includes image data to be desensitized and audio data to be desensitized, the desensitization effect parameters are determined based on the image data to be desensitized and the desensitized image data, the audio data to be desensitized and the desensitized audio data, and so on. For the convenience of description, taking the target data to be desensitized including image data to be desensitized, audio data to be desensitized, and text data to be desensitized as an example, the desensitization effect parameter determination process may include: Step S31, determining the image matching degree between the desensitized image data and the image data to be desensitized.

[0100] Exemplarily, the image matching degree indicates the similarity between the desensitized image data and the image data to be desensitized. The more similar the desensitized image data and the image data to be desensitized are, the greater the image matching degree is. Based on this, an image matching degree determination algorithm is used to determine the image matching degree between the desensitized image data and the image data to be desensitized. For example, the desensitized image data and the image data to be desensitized are input into the image matching degree determination algorithm, and the image matching degree determination algorithm outputs the image matching degree.

[0101] For the image matching degree determination algorithm, an image feature matching algorithm based on deep learning, such as the SIFT (Scale Invariant Feature Transform) algorithm, can be used. Other types of algorithms can also be used, and there is no restriction on this. The SIFT algorithm is taken as an example. Based on the SIFT algorithm, the image feature 1 of the desensitized image data and the image feature 2 of the image data to be desensitized are input into the SIFT algorithm. The desensitized image data and the image data to be desensitized are matched based on the image feature 1 and the image feature 2 by the SIFT algorithm to obtain the image matching degree of the desensitized image data and the image data to be desensitized.

[0102] Step S32: Determine the audio matching degree between the desensitized audio data and the audio data to be desensitized.

[0103] Exemplarily, the audio matching degree indicates the similarity between the desensitized audio data and the audio data to be desensitized. The more similar the desensitized audio data is to the audio data to be desensitized, the greater the audio matching degree is. Based on this, an audio matching degree determination algorithm is used to determine the audio matching degree between the desensitized audio data and the audio data to be desensitized. For example, the desensitized audio data and the audio data to be desensitized are input into the audio matching degree determination algorithm, and the audio matching degree determination algorithm outputs the audio matching degree.

[0104] For the audio matching degree determination algorithm, an image feature matching algorithm based on deep learning, such as the SIFT algorithm, can also be used. That is to say, the audio matching degree can be determined by using the SIFT algorithm. Other types of algorithms can also be used, and there is no limitation on this. The SIFT algorithm is taken as an example. Based on the SIFT algorithm, the desensitized audio data can be matched with the audio data to be desensitized to obtain the audio matching degree.

[0105] In order to determine the audio matching degree by using the SIFT algorithm, the audio feature 1 of the desensitized audio data and the audio feature 2 of the audio data to be desensitized can be determined, and the audio feature 1 is mapped to the vector space of the same dimension as the image feature descriptor to obtain the audio feature 3, and the audio feature 2 is mapped to the vector space of the same dimension as the image feature descriptor to obtain the audio feature 4. For example, if the input feature of the SIFT algorithm is a feature of the A*B*C dimension (that is, the vector space of the image feature descriptor is a feature of the A*B*C dimension), A can represent the feature width, B can represent the feature height, and C can represent the number of feature channels, then the audio feature 1 can be mapped to the audio feature 3 of the A*B*C dimension, and the audio feature 2 can be mapped to the audio feature 4 of the A*B*C dimension. On this basis, the audio feature 3 and the audio feature 4 can be input into the SIFT algorithm, and the desensitized audio data and the audio data to be desensitized are matched by the SIFT algorithm based on the audio feature 3 and the audio feature 4 to obtain the audio matching degree of the desensitized audio data and the audio data to be desensitized.

[0106] For example, regarding how to map audio feature 1 (audio feature 2) to audio feature 3 (audio feature 4), a feature mapping method based on deep learning can be used. This feature mapping method is used to map features of one dimension to features of another dimension, and there is no restriction on this feature mapping method.

[0107] Step S33: Determine the text matching degree between the desensitized text data and the text data to be desensitized.

[0108] Exemplarily, the text matching degree indicates the similarity between the desensitized text data and the text data to be desensitized. The more similar the desensitized text data and the text data to be desensitized are, the greater the text matching degree is. Based on this, a text matching degree determination algorithm can be used to determine the text matching degree between the desensitized text data and the text data to be desensitized. For example, the desensitized text data and the text data to be desensitized can be input into the text matching degree determination algorithm, and the text matching degree determination algorithm outputs the text matching degree.

[0109] For the text matching algorithm, an image feature matching algorithm based on deep learning, such as the SIFT algorithm, can also be used. That is to say, the text matching can be determined by using the SIFT algorithm. Other types of algorithms can also be used, and there is no limitation on this. The SIFT algorithm is taken as an example. Based on the SIFT algorithm, the desensitized text data can be matched with the text data to be desensitized to obtain the text matching degree.

[0110] In order to determine the text matching degree by using the SIFT algorithm, the text feature 1 of the desensitized text data and the text feature 2 of the text data to be desensitized can be determined. The text feature 1 is converted into a feature vector 1 by word embedding and semantic coding technology, and the feature vector 1 is mapped to a vector space of the same dimension as the image feature descriptor (i.e., the SIFT feature descriptor) by linear transformation to obtain text feature 3. The text feature 2 is converted into a feature vector 2 by word embedding and semantic coding technology, and the feature vector 2 is mapped to a vector space of the same dimension as the image feature descriptor by linear transformation to obtain text feature 4. For example, if the input feature of the SIFT algorithm is a feature of the A*B*C dimension, then the text feature 3 and the text feature 4 are of the A*B*C dimension. On this basis, the text feature 3 and the text feature 4 can be input into the SIFT algorithm, and the desensitized text data and the text data to be desensitized are matched by the SIFT algorithm based on the text feature 3 and the text feature 4 to obtain the text matching degree of the desensitized text data and the text data to be desensitized.

[0111] Step S34: Determine the feature matching degree based on the image matching degree, the audio matching degree and the text matching degree. For example, perform a weighted operation on the image matching degree, the audio matching degree and the text matching degree to obtain the feature matching degree.

[0112] For example, the feature matching degree can be determined using the following formula (1): Formula (1) In formula (1), SIFT new It can represent the feature matching degree, SIFT imageIt can represent the image matching degree. Here, the weighted coefficient of the image matching degree is 1 for explanation. The weighted coefficient of the image matching degree can also be other values ​​and can be configured according to experience. There is no limitation on this. SIFT audio It can represent the audio matching degree, and the weighted coefficient of the audio matching degree is , weighting coefficient It can be configured based on experience, or determined based on the importance and relevance of the audio matching degree, and there is no limitation on this. SIFT text It can represent the text matching degree, and the weighted coefficient of the text matching degree is , weighting coefficient It can be configured based on experience, or determined based on the importance and relevance of the audio matching degree, and there is no limitation on this.

[0113] From formula (1), it can be seen that feature matching can focus on audio features (audio matching) and text features (text matching) on ​​the basis of focusing on local features of the image (image matching), and can comprehensively consider the impact of desensitization on key information from multiple dimensions, and integrate multimodal information into SIFT feature descriptors (feature matching), that is, SIFT feature descriptors comprehensively consider local features of the image, audio descriptions, and text information. By taking feature matching as a desensitization effect parameter, feature matching can be used as the degree of retention of key information, and the retention and removal effects of key information can be used as desensitization effect evaluation indicators.

[0114] Weighting coefficient and weighting coefficient The weight is determined according to the importance and relevance of the modality. By performing weighted operations on image matching, audio matching, and text matching, the fusion of multimodal information at the feature descriptor level is achieved, so that the feature descriptor contains not only the local information of the image, but also the audio information and text information related to the image, reflecting the key information more comprehensively.

[0115] Step S35: determining the image quality based on the desensitized image data and the image data to be desensitized.

[0116] Exemplarily, the image quality represents the image quality of the desensitized image data. The more similar the desensitized image data is to the image data to be desensitized, the better the image quality of the desensitized image data is. Based on this, an image quality assessment algorithm can be used to determine the image quality of the desensitized image data. For example, the desensitized image data and the image data to be desensitized can be input into the image quality assessment algorithm, and the image quality assessment algorithm determines the image quality based on the desensitized image data and the image data to be desensitized, and outputs the image quality.

[0117] For the image quality assessment algorithm, the SSIM (Structural Similarity) algorithm can be used. The SSIM algorithm is an indicator for measuring the similarity between two images. The SSIM algorithm can compare images in terms of brightness, contrast, and structural information. Of course, the image quality assessment algorithm can also use other types of algorithms, which are not limited to this. The SSIM algorithm is used as an example for explanation.

[0118] For example, the image quality of desensitized image data can be determined using the following formula (2): Formula (2) In formula (2), SSIM improved Indicates the image quality of the desensitized image data, x Indicates the image data to be desensitized. y Indicates desensitized image data. fps Indicates the frame rate of the video to be desensitized, that is, it can be based on the image data to be desensitized x , desensitized image data y and frame rate fps Determining image quality SSIM improved .

[0119] (fps) Represents the target brightness weight, i.e. frame rate fps The corresponding target brightness weight, represents the brightness parameter value, and the product value of the target brightness weight and the brightness parameter value represents the first similarity value. In the brightness parameter value, μ x Represents the mean pixel value of the image data to be desensitized (i.e. the average value of the pixel values ​​of all pixels), μ y Represents the mean pixel value of the desensitized image data, c 1 Represents a configured constant value, which is used to avoid the numerator or denominator of the brightness parameter value being 0. Obviously, the brightness parameter value is based on the mean pixel value of the desensitized image data. μ y And the mean pixel value of the image data to be desensitized μ x Sure.

[0120] (fps) Indicates frame rate fps The corresponding target contrast weight, represents the contrast parameter value, and the product value of the target contrast weight and the contrast parameter value represents the second similarity value. In the contrast parameter value, x Represents the pixel value variance of the image data to be desensitized (i.e. the variance of the pixel values ​​of all pixels), y Represents the pixel value variance of the desensitized image data, c 2 Indicates a configured constant value, which is used to prevent the numerator or denominator of the contrast parameter value from being 0. Obviously, the contrast parameter value is determined based on the pixel value variance of the desensitized image data and the pixel value variance of the image data to be desensitized.

[0121] (fps) Indicates frame rate fps The corresponding target structure weight, represents the structural parameter value, and the product of the target structural weight and the structural parameter value represents the third similarity value. In the structural parameter value, x Represents the pixel value variance of the image data to be desensitized, y Represents the pixel value variance of the desensitized image data, xy Represents the covariance of the pixel values ​​of the desensitized image data and the image data to be desensitized, c 2 Represents a configured constant value. Obviously, the structure parameter value is determined based on the pixel value variance of the desensitized image data, the pixel value variance of the image data to be desensitized, and the covariance of the pixel values ​​of the desensitized image data and the image data to be desensitized.

[0122] It can be seen from formula (2) that the image quality can be determined based on the first similarity value, the second similarity value and the third similarity value, such as the image quality is the sum of the first similarity value, the second similarity value and the third similarity value.

[0123] In a possible implementation, in order to obtain accurate target brightness weights, target contrast weights, and target structure weights, the videos to be desensitized can be divided into different categories according to their frame rates. For example, videos to be desensitized whose frame rates are less than a first value (such as 15fps) are defined as low frame rate videos, videos to be desensitized whose frame rates are greater than a second value (such as 30fps) are defined as high frame rate videos, and videos to be desensitized whose frame rates are not less than the first value and not greater than the second value are defined as medium frame rate videos. For videos with different frame rates, initial weights are set for each component of the SSIM algorithm (brightness, contrast, structure).

[0124] Since high frame rate videos contain more dynamic detail information, they can be given a higher weight in terms of structural similarity to highlight the consideration of image details and dynamic changes. Medium frame rate videos are relatively balanced, and low frame rate videos focus more on overall perceived quality. On this basis, the initial weights can be set as follows: For the initial brightness weight, if the frame rate is less than the first value, the initial brightness weight is the first brightness weight value, and the first brightness weight value can be configured based on experience, such as 0.4. If the frame rate is greater than the second value, the initial brightness weight is the second brightness weight value, and the second brightness weight value can be configured based on experience, such as 0.2, and the second brightness weight value needs to be less than the first brightness weight value. If the frame rate is not less than the first value and not greater than the second value, the initial brightness weight is between the second brightness weight value and the first brightness weight value, and the initial brightness weight is inversely proportional to the frame rate. For example, the initial brightness weight is determined based on the second brightness weight value, the first brightness weight value and the frame rate, as long as the initial brightness weight is between the second brightness weight value and the first brightness weight value. See formula (3) to determine the initial brightness weight. (fps) .

[0125] Formula (3) For the initial contrast weight, if the frame rate is less than the third value (the third value can be the same as or different from the first value, taking 15 as an example), the initial contrast weight is the first contrast weight value, which can be configured based on experience, such as 0.4. If the frame rate is greater than the fourth value (the fourth value can be the same as or different from the second value, taking 30 as an example), the initial contrast weight is the second contrast weight value, which can be configured based on experience, such as 0.2, and the second contrast weight value needs to be less than the first contrast weight value. If the frame rate is not less than the third value and not greater than the fourth value, the initial contrast weight is between the second contrast weight value and the first contrast weight value, and the initial contrast weight is inversely proportional to the frame rate. For example, the initial contrast weight is determined based on the second contrast weight value, the first contrast weight value and the frame rate, as long as the initial contrast weight is between the second contrast weight value and the first contrast weight value. As shown in formula (4), to determine the initial contrast weight (fps) .

[0126] Formula (4) For the initial structural weight, if the frame rate is less than the fifth value (the fifth value is the same as or different from the first value, taking 15 as an example), the initial structural weight is the first structural weight value, and the first structural weight value is configured based on experience, such as 0.2. If the frame rate is greater than the sixth value (the sixth value is the same as or different from the second value, taking 30 as an example), the initial structural weight is the second structural weight value, and the second structural weight value is configured based on experience, such as 0.6, and the second structural weight value needs to be greater than the first structural weight value. If the frame rate is not less than the fifth value and not greater than the sixth value, the initial structural weight is between the first structural weight value and the second structural weight value, and the initial structural weight is proportional to the frame rate. For example, the initial structural weight is determined based on the second structural weight value, the first structural weight value and the frame rate, as long as the initial structural weight is between the first structural weight value and the second structural weight value. See formula (5) to determine the initial structural weight (fps) .

[0127] Formula (5) After obtaining the initial brightness weight, the initial brightness weight can be determined as the target brightness weight. After obtaining the initial contrast weight, the initial contrast weight can be determined as the target contrast weight. After obtaining the initial structure weight, the initial structure weight can be determined as the target structure weight. In summary, the target brightness weight, target contrast weight, and target structure weight can be dynamically adjusted according to the frame rate of the video to be desensitized, so that the image quality of the desensitized image data can be measured more accurately.

[0128] In a possible implementation, after obtaining the initial brightness weight, initial contrast weight, and initial structure weight, the initial brightness weight, initial contrast weight, and initial structure weight can be dynamically adjusted based on the degree of motion of the video to be desensitized, so as to obtain more accurate target brightness weight, target contrast weight, and target structure weight, which can improve the accuracy of image quality assessment based on the motion state.

[0129] Exemplarily, the initial brightness weight is adjusted based on the degree of motion of the video to be desensitized, and the adjusted brightness weight is determined as the target brightness weight. The initial contrast weight is adjusted based on the degree of motion of the video to be desensitized, and the adjusted contrast weight is determined as the target contrast weight. The initial structure weight is adjusted based on the degree of motion of the video to be desensitized, and the adjusted structure weight is determined as the target structure weight.

[0130] The degree of motion of the video to be desensitized is determined based on the motion vectors between adjacent frames in the video to be desensitized. For example, the motion vectors between adjacent frames of the video to be desensitized can be calculated using an optical flow method (such as the Lucas-Kanade optical flow method or the Farneback optical flow method). For example, a reference frame can be selected from the video to be desensitized (the reference frame can be one or more frames), and for each pixel of the reference frame, the motion vector of the pixel (that is, the displacement vector of the pixel between the reference frame and the adjacent frame) is calculated by the optical flow method, and the motion vector of the pixel can reflect the motion of the pixel. The degree of motion of the video to be desensitized is quantified by calculating the average or variance of the motion vectors of all pixels.

[0131] For example, the following formula (6) can be used to determine the degree of motion of the video to be desensitized: Formula (6) In formula (6), Motion intensity represents the degree of motion of the video to be desensitized, N represents the total number of pixels in the reference frame, Represents the motion vector of the i-th pixel, where i is 1-N. In this way, the average value of the motion vectors of all pixels can be used as a measure of the degree of motion of the video to be desensitized.

[0132] If the motion level of the video to be desensitized is less than the first motion threshold (the first motion threshold can be configured based on experience and is not limited), it means that the details and structural information of the image are more important under low motion intensity. Based on this, the initial brightness weight can be increased, and the increased brightness weight is used as the target brightness weight. The initial contrast weight can remain unchanged, that is, the initial contrast weight is used as the target contrast weight. The initial structure weight can remain unchanged, that is, the initial structure weight is used as the target structure weight.

[0133] If the motion level of the video to be desensitized is greater than the second motion threshold (the second motion threshold can be configured based on experience and is not limited), it means that the video is in a state of high motion intensity, and more emphasis is placed on the overall coherence and smoothness of the image. Based on this, the initial brightness weight can be kept unchanged, that is, the initial brightness weight is used as the target brightness weight. The initial contrast weight can be increased, and the increased contrast weight is used as the target contrast weight. The initial structure weight can be increased, and the increased structure weight is used as the target structure weight.

[0134] If the degree of motion of the video to be desensitized is not less than the first motion threshold and not greater than the second motion threshold, it means that it is in a medium motion intensity, and more attention is paid to maintaining relative balance. Based on this, the initial brightness weight, the initial contrast weight, and the initial structure weight can be adjusted so that the difference between the adjusted brightness weight, the adjusted contrast weight, and the adjusted structure weight is not greater than the threshold. The adjusted brightness weight can be used as the target brightness weight, the adjusted contrast weight can be used as the target contrast weight, and the adjusted structure weight can be used as the target structure weight. Among them, when the difference between the adjusted brightness weight and the adjusted contrast weight is not greater than the threshold, the adjusted brightness weight is the same or approximately the same as the adjusted contrast weight. When the difference between the adjusted brightness weight and the adjusted structure weight is not greater than the threshold, the adjusted brightness weight is the same or approximately the same as the adjusted structure weight. When the difference between the adjusted contrast weight and the adjusted structure weight is not greater than the threshold, the adjusted contrast weight is the same or approximately the same as the adjusted structure weight.

[0135] In summary, the target brightness weight, target contrast weight, and target structure weight can be dynamically adjusted according to the degree of motion of the video to be desensitized, which can more accurately measure the image quality of the desensitized image data. By improving the SSIM algorithm based on the inter-frame weighting method based on frame rate and degree of motion, the image quality assessment can be more accurately adapted to the dynamic changes of video data, further improving the desensitization effect of video data.

[0136] In one possible implementation, in the process of desensitizing video data based on the desensitization strategy, the effect of the desensitization operation can be monitored in real time. For example, the hash value of the original data and the desensitized data is calculated using a hash function, and the difference in the hash value is compared as an integrity indicator. The accuracy of the desensitized data is evaluated by combining manual labeling and automatic evaluation, such as checking whether sensitive information is completely desensitized, and using timestamps and operation logs to ensure the consistency of the desensitization process. According to the evaluation results, a reinforcement learning algorithm (such as Q-learning) is used to adjust the desensitization strategy to ensure the stable and reliable operation of the desensitization process.

[0137] Step S36, obtaining desensitization effect parameters, which include feature matching degree and image quality.

[0138] At this point, step 204 is completed, and the desensitization effect parameter can be obtained.

[0139] Step 205: Determine whether the desensitization effect parameters have met the desensitization quality conditions.

[0140] If so, that is, the desensitization effect parameter has satisfied the desensitization quality condition, then step 206 is executed.

[0141] If not, that is, the desensitization effect parameter does not meet the desensitization quality condition, step 207 is executed.

[0142] Exemplarily, the desensitization effect parameters may include feature matching and image quality. If the feature matching is greater than a first threshold, and the image quality is greater than a second threshold, the desensitization effect parameters have met the desensitization quality condition. If the feature matching is not greater than the first threshold, and / or the image quality is not greater than the second threshold, the desensitization effect parameters do not meet the desensitization quality condition. For example, the first threshold can be configured based on experience to represent the threshold of feature matching, such as 0.8, 0.85, etc., without limitation, and the second threshold can be configured based on experience to represent the threshold of image quality, such as 0.8, 0.85, etc., without limitation.

[0143] Step 206: Generate a desensitized video corresponding to the video to be desensitized based on the desensitized data, and send the desensitized video, that is, send the desensitized video to the receiving end of the video to be desensitized.

[0144] For example, a desensitized video corresponding to the video to be desensitized can be generated based on the desensitized image data, the desensitized audio data and the desensitized text data. There is no restriction on the method of generating the desensitized video. The desensitized video can include the desensitized image data, the desensitized audio data and the desensitized text data.

[0145] Step 207, adjust the target desensitization strategy to obtain an adjusted desensitization strategy, use the adjusted desensitization strategy as the target desensitization strategy, and return to execute step 203, that is, based on the target desensitization strategy (that is, the adjusted desensitization strategy), desensitize the target data to be desensitized to obtain desensitized data.

[0146] For example, if the target desensitization strategy includes an image desensitization strategy, an audio desensitization strategy, and a text desensitization strategy, the image desensitization strategy, the audio desensitization strategy, and the text desensitization strategy are adjusted at the same time to obtain an adjusted image desensitization strategy, an adjusted audio desensitization strategy, and an adjusted text desensitization strategy. Alternatively, the image desensitization strategy is adjusted to obtain an adjusted image desensitization strategy, while the audio desensitization strategy and the text desensitization strategy remain unchanged. Alternatively, the audio desensitization strategy is adjusted to obtain an adjusted audio desensitization strategy, while the image desensitization strategy and the text desensitization strategy remain unchanged. Alternatively, the text desensitization strategy is adjusted to obtain an adjusted text desensitization strategy, while the image desensitization strategy and the audio desensitization strategy remain unchanged.

[0147] In a possible implementation, the adjustment priority of the image desensitization policy is higher than the adjustment priority of the audio desensitization policy, and the adjustment priority of the audio desensitization policy is higher than the adjustment priority of the text desensitization policy.

[0148] During the first desensitization strategy adjustment process, the image desensitization strategy is adjusted to obtain the adjusted image desensitization strategy, and the audio desensitization strategy and the text desensitization strategy are kept unchanged. Then, steps 203 to 205 are repeated. If the desensitization effect parameters have met the desensitization quality conditions, the adjustment process is terminated. If the desensitization effect parameters do not meet the desensitization quality conditions, the second desensitization strategy adjustment process is performed.

[0149] In the second desensitization strategy adjustment process, the audio desensitization strategy is adjusted to obtain the adjusted audio desensitization strategy, and the image desensitization strategy and the text desensitization strategy are kept unchanged. Steps 203 to 205 are repeated. If the desensitization effect parameters do not meet the desensitization quality conditions, the third desensitization strategy adjustment process is performed.

[0150] In the third desensitization strategy adjustment process, the text desensitization strategy is adjusted to obtain an adjusted text desensitization strategy, and the image desensitization strategy and the audio desensitization strategy are kept unchanged. Steps 203 to 205 are repeated. If the desensitization effect parameters do not meet the desensitization quality conditions, the fourth desensitization strategy adjustment process is performed.

[0151] During the fourth desensitization strategy adjustment process, the image desensitization strategy is adjusted, and so on.

[0152] Exemplarily, when the image desensitization strategy is adjusted to obtain the adjusted image desensitization strategy, if the image desensitization strategy is an image desensitization network model, the network parameters in the image desensitization network model can be adjusted. Of course, this is just an example of adjustment, as long as the image desensitization strategy can be adjusted.

[0153] For example, when the image desensitization network model (StyleGAN model) is used to desensitize the image sensitive positions of the image data to be desensitized, the desensitization effect depends on the network parameters in the image desensitization network model. Therefore, by adjusting the network parameters in the image desensitization network model, the desensitization effect can be optimized so that the desensitization effect parameters can meet the desensitization quality conditions. In addition, regarding the adjustment method of the network parameters, there is no restriction in this embodiment, and the network parameters can be adjusted.

[0154] Exemplarily, when the audio desensitization strategy is adjusted to obtain the adjusted audio desensitization strategy, if the audio desensitization strategy includes an audio desensitization network model (such as a Tacotron model) and a vocoder (such as a WaveNet vocoder), the network parameters in the audio desensitization network model can be adjusted. For example, when the audio sensitive position of the audio data to be desensitized is desensitized using the audio desensitization network model, the desensitization effect depends on the network parameters in the audio desensitization network model. By adjusting the network parameters in the audio desensitization network model, the desensitization effect can be optimized so that the desensitization effect parameters meet the desensitization quality conditions.

[0155] Exemplarily, when adjusting the text desensitization strategy to obtain the adjusted text desensitization strategy, if the text desensitization strategy is a text encryption algorithm, the algorithm type used by the text encryption algorithm can be adjusted. For example, if the algorithm type of the text encryption algorithm is the block encryption mode of the AES-256 encryption algorithm, it can be adjusted to the filling mode of the AES-256 encryption algorithm or the DES encryption algorithm. Alternatively, if the text desensitization strategy is a text replacement algorithm, the algorithm type used by the text replacement algorithm can be adjusted.

[0156] It can be seen from the above technical solutions that in the embodiment of the present application, a video data desensitization method driven by a multimodal large model is provided, and the multimodal large model is used to intelligently desensitize the video data, giving full play to the advantages of the multimodal large model. When the desensitization effect parameter does not meet the desensitization quality condition, the target desensitization strategy can be adjusted until the desensitization effect parameter meets the desensitization quality condition, so that the quality of the desensitized video is better, the desensitization effect is better, and the original information of the video data can be reflected. The desensitization effect of the desensitized video can be evaluated, and the desensitization strategy can be adaptively adjusted to improve the desensitization effect, so that the desensitized video has availability. While improving the data security and availability of the desensitized video, the image quality assessment can be more accurately adapted to the dynamic changes of the video data. In the process of processing video data with the multimodal large model, the integrity and availability of the desensitized data are verified. The multimodal large model is used to compare and analyze the video data before and after desensitization. It is evaluated from multiple modes (such as image quality, degree of retention of key information, etc., where the degree of retention of key information can be the above-mentioned feature matching degree) whether the desensitization operation affects the value of the video data, and the desensitization strategy can be automatically adjusted to achieve the best balance between security and availability.

[0157] Based on the same application concept as the above method, a multi-modal large model driven data desensitization and video data protection device is proposed in the embodiment of the present application, see Figure 4 FIG. 1 is a schematic diagram of the structure of the device, and the device may include: The determination module 41 is used to determine the target data to be desensitized based on the acquired video to be desensitized; determine the target desensitization strategy corresponding to the target data to be desensitized; the processing module 42 is used to perform desensitization processing on the target data to be desensitized through a multimodal large model based on the target desensitization strategy to obtain desensitized data; the determination module 41 is also used to determine the desensitization effect parameter based on the target data to be desensitized and the desensitized data; wherein the desensitization effect parameter is used to reflect the desensitization quality of the desensitized data; the generation module 43 is used to generate a desensitized video corresponding to the video to be desensitized based on the desensitized data if the desensitization effect parameter meets the desensitization quality condition, and send the desensitized video; the processing module 42 is also used to adjust the target desensitization strategy to obtain an adjusted desensitization strategy if the desensitization effect parameter does not meet the desensitization quality condition, use the adjusted desensitization strategy as the target desensitization strategy, and perform desensitization processing on the target data to be desensitized based on the target desensitization strategy to obtain desensitized data.

[0158] Exemplarily, the target data to be desensitized includes at least two of the image data to be desensitized, the audio data to be desensitized, and the text data to be desensitized; if the target data to be desensitized includes the image data to be desensitized, the audio data to be desensitized, and the text data to be desensitized, the desensitized data includes the desensitized image data, the desensitized audio data, and the desensitized text data; the determination module 41 determines the desensitization effect parameters based on the target data to be desensitized and the desensitized data, and is specifically used for: determining the desensitization effect parameters based on the image matching degree between the desensitized image data and the image data to be desensitized, the image matching degree between the desensitized audio data and the audio data to be desensitized, and the image matching degree between the desensitized audio data and the audio data to be desensitized. The audio matching degree between the desensitized text data and the text data to be desensitized is used to determine the feature matching degree; the image quality is determined based on the desensitized image data and the image data to be desensitized; a desensitization effect parameter is obtained, and the desensitization effect parameter includes the feature matching degree and the image quality; if the feature matching degree is greater than a first threshold and the image quality is greater than a second threshold, the desensitization effect parameter has met the desensitization quality condition; if the feature matching degree is not greater than the first threshold and / or the image quality is not greater than the second threshold, the desensitization effect parameter does not meet the desensitization quality condition.

[0159] Exemplarily, when determining the image quality based on the desensitized image data and the image data to be desensitized, the determination module 41 is specifically used to: determine a first similarity value based on a target brightness weight and a brightness parameter value, determine a second similarity value based on a target contrast weight and a contrast parameter value, and determine a third similarity value based on a target structure weight and a structure parameter value; the brightness parameter value is determined based on the mean pixel value of the desensitized image data and the mean pixel value of the image data to be desensitized, the contrast parameter value is determined based on the pixel value variance of the desensitized image data and the pixel value variance of the image data to be desensitized, and the structure parameter value is determined based on the pixel value variance of the desensitized image data, the pixel value variance of the image data to be desensitized, and the covariance of the pixel values ​​of the desensitized image data and the image data to be desensitized; the image quality is determined based on the first similarity value, the second similarity value, and the third similarity value.

[0160] Exemplarily, when determining the target brightness weight, the determination module 41 is specifically used to: determine an initial brightness weight corresponding to the frame rate of the video to be desensitized; determine the target brightness weight based on the initial brightness weight; wherein, if the frame rate is less than a first value, the initial brightness weight is a first brightness weight value; if the frame rate is greater than a second value, the initial brightness weight is a second brightness weight value; if the frame rate is not less than the first value and not greater than the second value, the initial brightness weight is between the second brightness weight value and the first brightness weight value, and the initial brightness weight is inversely proportional to the frame rate.

[0161] Exemplarily, when determining the target contrast weight, the determination module 41 is specifically used to: determine an initial contrast weight corresponding to the frame rate; determine the target contrast weight based on the initial contrast weight; wherein, if the frame rate is less than a third value, the initial contrast weight is a first contrast weight value; if the frame rate is greater than a fourth value, the initial contrast weight is a second contrast weight value; if the frame rate is not less than the third value and not greater than the fourth value, the initial contrast weight is between the second contrast weight value and the first contrast weight value, and the initial contrast weight is inversely proportional to the frame rate.

[0162] Exemplarily, when determining the target structure weight, the determination module 41 is specifically used to: determine an initial structure weight corresponding to the frame rate; determine the target structure weight based on the initial structure weight; wherein, if the frame rate is less than a fifth value, the initial structure weight is a first structure weight value; if the frame rate is greater than a sixth value, the initial structure weight is a second structure weight value; if the frame rate is not less than the fifth value and not greater than the sixth value, the initial structure weight is between the first structure weight value and the second structure weight value, and the initial structure weight is proportional to the frame rate.

[0163] Exemplarily, when the determination module 41 determines the target brightness weight based on the initial brightness weight, it is specifically used to: determine the initial brightness weight as the target brightness weight; or, adjust the initial brightness weight based on the degree of motion of the video to be desensitized, and determine the adjusted brightness weight as the target brightness weight; wherein the degree of motion is determined based on the motion vector between adjacent frames in the video to be desensitized; if the degree of motion is less than a first motion threshold, increase the initial brightness weight, and if the degree of motion is greater than a second motion threshold, the initial brightness weight remains unchanged.

[0164] Exemplarily, when determining the target contrast weight based on the initial contrast weight, the determination module 41 is specifically used to: determine the initial contrast weight as the target contrast weight; or, adjust the initial contrast weight based on the motion degree, and determine the adjusted contrast weight as the target contrast weight; if the motion degree is less than a first motion threshold, the initial contrast weight remains unchanged; if the motion degree is greater than a second motion threshold, increase the initial contrast weight.

[0165] Exemplarily, when determining the target structure weight based on the initial structure weight, the determination module 41 is specifically used to: determine the initial structure weight as the target structure weight; or, adjust the initial structure weight based on the degree of movement, and determine the adjusted structure weight as the target structure weight; wherein, if the degree of movement is less than a first movement threshold, the initial structure weight remains unchanged; if the degree of movement is greater than a second movement threshold, the initial structure weight is increased.

[0166] Exemplarily, if the degree of motion is not less than a first motion threshold and not greater than a second motion threshold, the initial brightness weight, the initial contrast weight, and the initial structure weight are adjusted so that the difference between the adjusted brightness weight, the adjusted contrast weight, and the adjusted structure weight is not greater than a threshold.

[0167] Exemplarily, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, the processing module 42 performs desensitization processing on the target data to be desensitized based on the target desensitization strategy to obtain desensitized data, and is specifically used for: extracting features from the image data to be desensitized by a multimodal large model to obtain image features, extracting features from the audio data to be desensitized by a multimodal large model to obtain audio features, extracting features from the text data to be desensitized by a multimodal large model to obtain text features, and performing feature fusion of the image features, the audio features, and the text features by a multimodal large model to obtain multimodal features; based on the multimodal The modal features determine the image sensitive positions of the image data to be desensitized, the audio sensitive positions of the audio data to be desensitized, and the text sensitive positions of the text data to be desensitized through a multimodal large model; based on the image desensitization strategy corresponding to the image data to be desensitized, the image sensitive positions of the image data to be desensitized are desensitized to obtain desensitized image data; based on the audio desensitization strategy corresponding to the audio data to be desensitized, the audio sensitive positions of the audio data to be desensitized are desensitized to obtain desensitized audio data; based on the text desensitization strategy corresponding to the text data to be desensitized, the text sensitive positions of the text data to be desensitized are desensitized to obtain desensitized text data.

[0168] Exemplarily, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, the target desensitization strategy includes an image desensitization strategy, an audio desensitization strategy, and a text desensitization strategy; the determination module 41 determines the target desensitization strategy corresponding to the target data to be desensitized by: querying the configured desensitization rule library through the multimodal attributes corresponding to the video to be desensitized, and obtaining the image desensitization strategy corresponding to the multimodal attributes; wherein the multimodal attributes include at least one of image attributes, audio attributes, and text attributes, and the desensitization rule library includes the correspondence between the multimodal attributes and the image desensitization strategy; the image desensitization strategy includes an image desensitization network model; Determining an audio desensitization strategy based on the audio attributes corresponding to the video to be desensitized, the audio desensitization strategy comprising an audio desensitization network model and a vocoder, the audio desensitization network model is used to convert the audio data to be desensitized into acoustic features, and the vocoder is used to convert the acoustic features into an audio waveform; Determine a configured text desensitization strategy, wherein the text desensitization strategy includes a text encryption algorithm or a text replacement algorithm, wherein the text encryption algorithm is used to perform an encryption operation on the desensitized text data, and the text replacement algorithm is used to perform a replacement operation or a fuzzification operation on the desensitized text data.

[0169] Exemplarily, when the processing module 42 adjusts the target desensitization strategy to obtain the adjusted desensitization strategy, if the image desensitization strategy needs to be adjusted, the network parameters in the image desensitization network model are adjusted; if the audio desensitization strategy needs to be adjusted, the network parameters in the audio desensitization network model are adjusted; if the text desensitization strategy needs to be adjusted, the algorithm type adopted by the text encryption algorithm is adjusted, or the algorithm type adopted by the text replacement algorithm is adjusted.

[0170] Based on the same application concept as the above method, an electronic device is proposed in the embodiment of the present application, see Figure 5 As shown, the electronic device includes: a processor 51 and a machine-readable storage medium 52, the machine-readable storage medium 52 storing machine-executable instructions that can be executed by the processor 51; the processor 51 is used to execute the machine-executable instructions to implement the multimodal large model-driven data desensitization and video data protection method disclosed in the above example of this application.

[0171] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the multimodal large model-driven data desensitization and video data protection method disclosed in the above example of the present application can be implemented.

[0172] The above-mentioned machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0173] Based on the same application concept as the above method, an embodiment of the present application also provides a computer program product, which may include a computer program, and when the computer program is executed by a processor, it implements the multimodal large model-driven data desensitization and video data protection method disclosed in the above example of the present application.

[0174] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0175] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A multimodal large model driven data desensitization and video data protection method, characterized in that: The method comprises: Determine the target data to be desensitized based on the acquired videos to be desensitized; Determine a target desensitization strategy corresponding to the target data to be desensitized, and perform desensitization processing on the target data to be desensitized by using a multimodal large model based on the target desensitization strategy to obtain desensitized data; Determine a desensitization effect parameter based on the target data to be desensitized and the desensitized data; If the desensitization effect parameter satisfies the desensitization quality condition, a desensitized video corresponding to the video to be desensitized is generated based on the desensitized data, and the desensitized video is sent; If the desensitization effect parameter does not meet the desensitization quality condition, the target desensitization strategy is adjusted to obtain an adjusted desensitization strategy, and the adjusted desensitization strategy is used as the target desensitization strategy, and the operation of desensitizing the target data to be desensitized based on the target desensitization strategy to obtain desensitized data is returned to be executed.

2. The method according to claim 1, characterized in that The target data to be desensitized includes at least two of image data to be desensitized, audio data to be desensitized, and text data to be desensitized; If the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, the desensitized data includes desensitized image data, desensitized audio data, and desensitized text data; The determining of the desensitization effect parameter based on the target data to be desensitized and the desensitized data includes: Determine a feature matching degree based on an image matching degree between the desensitized image data and the image data to be desensitized, an audio matching degree between the desensitized audio data and the audio data to be desensitized, and a text matching degree between the desensitized text data and the text data to be desensitized; Determining image quality based on the desensitized image data and the image data to be desensitized; Obtain desensitization effect parameters, wherein the desensitization effect parameters include the feature matching degree and the image quality; if the feature matching degree is greater than a first threshold and the image quality is greater than a second threshold, the desensitization effect parameters have met the desensitization quality condition; if the feature matching degree is not greater than the first threshold and / or the image quality is not greater than the second threshold, the desensitization effect parameters do not meet the desensitization quality condition.

3. The method according to claim 2, characterized in that The determining of the image quality based on the desensitized image data and the image data to be desensitized includes: A first similarity value is determined based on a target brightness weight and a brightness parameter value, a second similarity value is determined based on a target contrast weight and a contrast parameter value, and a third similarity value is determined based on a target structure weight and a structure parameter value; wherein the brightness parameter value is determined based on a mean pixel value of the desensitized image data and a mean pixel value of the image data to be desensitized, the contrast parameter value is determined based on a pixel value variance of the desensitized image data and a pixel value variance of the image data to be desensitized, and the structure parameter value is determined based on a pixel value variance of the desensitized image data, a pixel value variance of the image data to be desensitized, and a covariance of pixel values ​​of the desensitized image data and the image data to be desensitized; Image quality is determined based on the first similarity value, the second similarity value, and the third similarity value.

4. The method according to claim 3, characterized in that: The process of determining the target brightness weight includes: determining an initial brightness weight corresponding to the frame rate of the video to be desensitized; wherein, if the frame rate is less than a first value, the initial brightness weight is a first brightness weight value; if the frame rate is greater than a second value, the initial brightness weight is a second brightness weight value; if the frame rate is not less than the first value and not greater than the second value, the initial brightness weight is between the second brightness weight value and the first brightness weight value, and the initial brightness weight is inversely proportional to the frame rate; Determining the target brightness weight based on the initial brightness weight; The process of determining the target contrast weight includes: determining an initial contrast weight corresponding to the frame rate; wherein, if the frame rate is less than a third value, the initial contrast weight is a first contrast weight value; if the frame rate is greater than a fourth value, the initial contrast weight is a second contrast weight value; if the frame rate is not less than the third value and not greater than the fourth value, the initial contrast weight is between the second contrast weight value and the first contrast weight value, and the initial contrast weight is inversely proportional to the frame rate; determining the target contrast weight based on the initial contrast weight; The process of determining the target structure weight includes: determining an initial structure weight corresponding to the frame rate; wherein, if the frame rate is less than a fifth value, the initial structure weight is a first structure weight value; if the frame rate is greater than a sixth value, the initial structure weight is a second structure weight value; if the frame rate is not less than the fifth value and not greater than the sixth value, the initial structure weight is between the first structure weight value and the second structure weight value, and the initial structure weight is proportional to the frame rate; The target structure weight is determined based on the initial structure weight.

5. The method according to claim 4, characterized in that The determining the target brightness weight based on the initial brightness weight comprises: Determine the initial brightness weight as the target brightness weight; or, adjust the initial brightness weight based on the degree of motion of the video to be desensitized, and determine the adjusted brightness weight as the target brightness weight; wherein the degree of motion is determined based on the motion vector between adjacent frames in the video to be desensitized; wherein, if the degree of motion is less than a first motion threshold, increase the initial brightness weight, and if the degree of motion is greater than a second motion threshold, the initial brightness weight remains unchanged; The determining the target contrast weight based on the initial contrast weight comprises: Determine the initial contrast weight as the target contrast weight; or, adjust the initial contrast weight based on the motion degree, and determine the adjusted contrast weight as the target contrast weight; wherein, if the motion degree is less than a first motion threshold, the initial contrast weight remains unchanged; if the motion degree is greater than a second motion threshold, the initial contrast weight is increased; The step of determining the target structure weight based on the initial structure weight comprises: Determine the initial structure weight as the target structure weight; or, adjust the initial structure weight based on the degree of motion, and determine the adjusted structure weight as the target structure weight; wherein, if the degree of motion is less than a first motion threshold, the initial structure weight remains unchanged; if the degree of motion is greater than a second motion threshold, the initial structure weight is increased; Among them, if the degree of motion is not less than the first motion threshold and not greater than the second motion threshold, the initial brightness weight, the initial contrast weight and the initial structure weight are adjusted so that the difference between the adjusted brightness weight, the adjusted contrast weight and the adjusted structure weight is not greater than the threshold.

6. The method according to claim 1, characterized in that If the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, desensitizing the target data to be desensitized by using a multimodal large model based on the target desensitization strategy to obtain desensitized data, including: Extracting features from the desensitized image data through a multimodal large model to obtain image features, extracting features from the desensitized audio data through a multimodal large model to obtain audio features, extracting features from the desensitized text data through a multimodal large model to obtain text features, and fusing the image features, the audio features, and the text features through a multimodal large model to obtain multimodal features; Determine, by means of a multimodal large model, an image sensitive position of the image data to be desensitized, an audio sensitive position of the audio data to be desensitized, and a text sensitive position of the text data to be desensitized based on the multimodal features; Based on the image desensitization strategy corresponding to the image data to be desensitized, the image sensitive positions of the image data to be desensitized are desensitized to obtain desensitized image data; based on the audio desensitization strategy corresponding to the audio data to be desensitized, the audio sensitive positions of the audio data to be desensitized are desensitized to obtain desensitized audio data; based on the text desensitization strategy corresponding to the text data to be desensitized, the text sensitive positions of the text data to be desensitized are desensitized to obtain desensitized text data.

7. The method according to claim 1, characterized in that If the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, the target desensitization strategy includes an image desensitization strategy, an audio desensitization strategy, and a text desensitization strategy; The determining of the target desensitization strategy corresponding to the target data to be desensitized includes: The configured desensitization rule base is queried through the multimodal attribute corresponding to the video to be desensitized, and the image desensitization strategy corresponding to the multimodal attribute is obtained; wherein the multimodal attribute includes at least one of an image attribute, an audio attribute, and a text attribute, and the desensitization rule base includes a correspondence between the multimodal attribute and the image desensitization strategy; wherein the image desensitization strategy includes an image desensitization network model; Determining an audio desensitization strategy based on the audio attributes corresponding to the video to be desensitized, the audio desensitization strategy comprising an audio desensitization network model and a vocoder, the audio desensitization network model is used to convert the audio data to be desensitized into acoustic features, and the vocoder is used to convert the acoustic features into an audio waveform; Determine a configured text desensitization strategy, wherein the text desensitization strategy includes a text encryption algorithm or a text replacement algorithm, wherein the text encryption algorithm is used to perform an encryption operation on the desensitized text data, and the text replacement algorithm is used to perform a replacement operation or a fuzzification operation on the desensitized text data; Among them, when the target desensitization strategy is adjusted to obtain the adjusted desensitization strategy, if the image desensitization strategy needs to be adjusted, the network parameters in the image desensitization network model are adjusted; if the audio desensitization strategy needs to be adjusted, the network parameters in the audio desensitization network model are adjusted; if the text desensitization strategy needs to be adjusted, the algorithm type adopted by the text encryption algorithm is adjusted, or the algorithm type adopted by the text replacement algorithm is adjusted.

8. A multi-modal large model driven data desensitization and video data protection device, characterized in that: The device comprises: A determination module, used to determine the target data to be desensitized based on the acquired video to be desensitized; determine the target desensitization strategy corresponding to the target data to be desensitized; a processing module, used to perform desensitization processing on the target data to be desensitized through a multimodal large model based on the target desensitization strategy to obtain desensitized data; The determination module is further used to determine a desensitization effect parameter based on the target data to be desensitized and the desensitized data, wherein the desensitization effect parameter is used to reflect the desensitization quality of the desensitized data; A generating module, configured to generate a desensitized video corresponding to the video to be desensitized based on the desensitized data if the desensitization effect parameter meets the desensitization quality condition, and send the desensitized video; The processing module is used to adjust the target desensitization strategy to obtain an adjusted desensitization strategy if the desensitization effect parameter does not meet the desensitization quality condition, use the adjusted desensitization strategy as the target desensitization strategy, and perform desensitization processing on the target data to be desensitized based on the target desensitization strategy to obtain desensitized data.

9. The device according to claim 8, characterized in that in, The target data to be desensitized includes at least two of image data to be desensitized, audio data to be desensitized, and text data to be desensitized; If the target data to be desensitized includes image data to be desensitized, audio data to be desensitized, and text data to be desensitized, the desensitized data includes desensitized image data, desensitized audio data, and desensitized text data; When the determination module determines the desensitization effect parameter based on the target data to be desensitized and the desensitized data, it is specifically used to: determine the feature matching degree based on the image matching degree between the desensitized image data and the image data to be desensitized, the audio matching degree between the desensitized audio data and the audio data to be desensitized, and the text matching degree between the desensitized text data and the text data to be desensitized; Determining image quality based on the desensitized image data and the image data to be desensitized; Obtaining a desensitization effect parameter, wherein the desensitization effect parameter includes the feature matching degree and the image quality; if the feature matching degree is greater than a first threshold and the image quality is greater than a second threshold, the desensitization effect parameter has satisfied a desensitization quality condition; If the feature matching degree is not greater than the first threshold, and / or the image quality is not greater than the second threshold, the desensitization effect parameter does not meet the desensitization quality condition; Wherein, when the determination module determines the image quality based on the desensitized image data and the image data to be desensitized, it is specifically used to: determine a first similarity value based on a target brightness weight and a brightness parameter value, determine a second similarity value based on a target contrast weight and a contrast parameter value, and determine a third similarity value based on a target structure weight and a structure parameter value; the brightness parameter value is determined based on the mean pixel value of the desensitized image data and the mean pixel value of the image data to be desensitized, the contrast parameter value is determined based on the pixel value variance of the desensitized image data and the pixel value variance of the image data to be desensitized, and the structure parameter value is determined based on the pixel value variance of the desensitized image data, the pixel value variance of the image data to be desensitized, and the covariance of the pixel values ​​of the desensitized image data and the image data to be desensitized; the image quality is determined based on the first similarity value, the second similarity value, and the third similarity value; Wherein, when the determination module determines the target brightness weight, it is specifically used to: determine the initial brightness weight corresponding to the frame rate of the video to be desensitized; determine the target brightness weight based on the initial brightness weight; wherein, if the frame rate is less than a first value, the initial brightness weight is a first brightness weight value; if the frame rate is greater than a second value, the initial brightness weight is a second brightness weight value; if the frame rate is not less than the first value and not greater than the second value, the initial brightness weight is between the second brightness weight value and the first brightness weight value, and the initial brightness weight is inversely proportional to the frame rate; Wherein, when determining the target contrast weight, the determination module is specifically used to: determine an initial contrast weight corresponding to the frame rate; determine the target contrast weight based on the initial contrast weight; wherein, if the frame rate is less than a third value, the initial contrast weight is a first contrast weight value; if the frame rate is greater than a fourth value, the initial contrast weight is a second contrast weight value; if the frame rate is not less than the third value and not greater than the fourth value, the initial contrast weight is between the second contrast weight value and the first contrast weight value, and the initial contrast weight is inversely proportional to the frame rate; Wherein, when determining the target structure weight, the determination module is specifically used to: determine an initial structure weight corresponding to the frame rate; determine the target structure weight based on the initial structure weight; wherein, if the frame rate is less than a fifth value, the initial structure weight is a first structure weight value; if the frame rate is greater than a sixth value, the initial structure weight is a second structure weight value; if the frame rate is not less than the fifth value and not greater than the sixth value, the initial structure weight is between the first structure weight value and the second structure weight value, and the initial structure weight is proportional to the frame rate; Wherein, when the determination module determines the target brightness weight based on the initial brightness weight, it is specifically used to: determine the initial brightness weight as the target brightness weight; or, adjust the initial brightness weight based on the degree of motion of the video to be desensitized, and determine the adjusted brightness weight as the target brightness weight; wherein, the degree of motion is determined based on the motion vector between adjacent frames in the video to be desensitized; wherein, if the degree of motion is less than a first motion threshold, the initial brightness weight is increased, and if the degree of motion is greater than a second motion threshold, the initial brightness weight remains unchanged; Wherein, when the determination module determines the target contrast weight based on the initial contrast weight, it is specifically used to: determine the initial contrast weight as the target contrast weight; or adjust the initial contrast weight based on the motion degree, and determine the adjusted contrast weight as the target contrast weight; if the motion degree is less than a first motion threshold, the initial contrast weight remains unchanged; if the motion degree is greater than a second motion threshold, the initial contrast weight is increased; Wherein, when the determination module determines the target structure weight based on the initial structure weight, it is specifically used to: determine the initial structure weight as the target structure weight; or, adjust the initial structure weight based on the movement degree, and determine the adjusted structure weight as the target structure weight; wherein, if the movement degree is less than a first movement threshold, the initial structure weight remains unchanged; if the movement degree is greater than a second movement threshold, the initial structure weight is increased; If the motion degree is not less than a first motion threshold and not greater than a second motion threshold, the initial brightness weight, the initial contrast weight, and the initial structure weight are adjusted so that the difference between the adjusted brightness weight, the adjusted contrast weight, and the adjusted structure weight is not greater than a threshold; Among them, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized and text data to be desensitized, the processing module performs desensitization processing on the target data to be desensitized through a multimodal large model based on the target desensitization strategy to obtain desensitized data, and is specifically used for: extracting features from the image data to be desensitized through a multimodal large model to obtain image features, extracting features from the audio data to be desensitized through a multimodal large model to obtain audio features, extracting features from the text data to be desensitized through a multimodal large model to obtain text features, and performing feature fusion on the image features, the audio features and the text features through a multimodal large model to obtain multimodal features; The modal big model determines the image sensitive position of the image data to be desensitized, the audio sensitive position of the audio data to be desensitized, and the text sensitive position of the text data to be desensitized based on the multimodal features; based on the image desensitization strategy corresponding to the image data to be desensitized, the image sensitive position of the image data to be desensitized is desensitized to obtain the desensitized image data; based on the audio desensitization strategy corresponding to the audio data to be desensitized, the audio sensitive position of the audio data to be desensitized is desensitized to obtain the desensitized audio data; based on the text desensitization strategy corresponding to the text data to be desensitized, the text sensitive position of the text data to be desensitized is desensitized to obtain the desensitized text data; Wherein, if the target data to be desensitized includes image data to be desensitized, audio data to be desensitized and text data to be desensitized, the target desensitization strategy includes image desensitization strategy, audio desensitization strategy and text desensitization strategy; when the determination module determines the target desensitization strategy corresponding to the target data to be desensitized, it is specifically used to: query the configured desensitization rule library through the multimodal attributes corresponding to the video to be desensitized, and obtain the image desensitization strategy corresponding to the multimodal attribute; wherein the multimodal attribute includes at least one of an image attribute, an audio attribute and a text attribute, and the desensitization rule library includes the corresponding relationship between the multimodal attribute and the image desensitization strategy; the image desensitization strategy includes an image desensitization network model; Determining an audio desensitization strategy based on the audio attributes corresponding to the video to be desensitized, the audio desensitization strategy comprising an audio desensitization network model and a vocoder, the audio desensitization network model is used to convert the audio data to be desensitized into acoustic features, and the vocoder is used to convert the acoustic features into an audio waveform; Determine a configured text desensitization strategy, wherein the text desensitization strategy includes a text encryption algorithm or a text replacement algorithm, wherein the text encryption algorithm is used to perform an encryption operation on the desensitized text data, and the text replacement algorithm is used to perform a replacement operation or a fuzzification operation on the desensitized text data; Among them, when the processing module adjusts the target desensitization strategy to obtain the adjusted desensitization strategy, if the image desensitization strategy needs to be adjusted, the network parameters in the image desensitization network model are adjusted; if the audio desensitization strategy needs to be adjusted, the network parameters in the audio desensitization network model are adjusted; if the text desensitization strategy needs to be adjusted, the algorithm type adopted by the text encryption algorithm is adjusted, or the algorithm type adopted by the text replacement algorithm is adjusted.

10. An electronic device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the method described in any one of claims 1-7.

Images