Deployment methods, devices, equipment, and media for image and video desensitization models

Abstract

The application relates to a desensitization model deployment method and device for images and videos, equipment and medium. The method comprises the following steps: obtaining desensitization verification data and desensitization training data; inputting the desensitization verification data into a desensitization pre-training model for model verification to obtain a desensitization verification model; when the accuracy of the desensitization verification model meets a first preset standard, converting the desensitization model into a desensitization fixed-point model, and performing model deployment based on the desensitization fixed-point model, otherwise, performing quantization-aware training on the desensitization verification model based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard. Therefore, by using the quantization-aware training technology in the model training process, the accuracy loss of the quantized model can be further reduced, the model can be quickly and efficiently deployed to other equipment, the problem that the accuracy of the model is lost during model quantization and fixed-point conversion of the pre-trained model in the prior art is solved, the desensitization algorithm accuracy is improved, and the desensitization algorithm accuracy is improved.

Description

Technical Field

[0001] This application relates to the fields of intelligent information processing and information security technology, and in particular to a method, apparatus, device and medium for deploying image and video desensitization models. Background Technology

[0002] With the continuous advancement of network communication technology, the networking level of vehicles is also constantly improving. People have gained more control over their vehicles through various network communication technologies, such as remote vehicle monitoring and remote parking. These functions of intelligent connected vehicles have brought new convenience and experiences to people's lives. However, cars also collect more information, which may involve sensitive data such as faces and license plates outside the vehicle. To protect personal privacy, the state has formulated relevant standards to regulate the data collected by vehicles. If the videos or pictures outside the vehicle collected by the car contain identifiable faces and license plates, the data needs to be de-identified before the vehicle leaves the vehicle to protect personal privacy information.

[0003] The relevant technologies are prone to accuracy loss when converting models from quantization to fixed points, which leads to poor model deployment results and urgently needs to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for deploying image and video desensitization models to solve the problem that existing well-trained models suffer from accuracy loss during model quantization and conversion to fixed points, resulting in poor model deployment performance, thereby improving the accuracy of desensitization algorithms.

[0005] To achieve the above objectives, the first aspect of this application proposes a method for deploying image and video desensitization models, comprising the following steps:

[0006] Obtain desensitization verification data and desensitization training data;

[0007] The desensitization verification data is input into the desensitization pre-training model for model verification to obtain the desensitization verification model.

[0008] Determine whether the accuracy of the desensitization verification model meets the first preset standard;

[0009] If the accuracy of the desensitization verification model meets the first preset standard, the desensitization model is converted into a desensitization fixed-point model, and the model is deployed based on the desensitization fixed-point model; otherwise, the desensitization verification model is subjected to quantitative perception training based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard.

[0010] According to one embodiment of this application, before deploying the model based on the desensitized point model, the method further includes:

[0011] Determine whether the accuracy of the desensitization point model meets the second preset standard;

[0012] If the accuracy of the desensitized fixed-point model meets the second preset standard, then the model is deployed based on the desensitized fixed-point model; otherwise, the step of performing quantitative perception training on the desensitized verification model based on the desensitized training data is repeated until the accuracy of the desensitized verification model meets the first preset standard.

[0013] According to one embodiment of this application, before inputting the de-identification verification data into the de-identification pre-trained model for model verification, the method further includes:

[0014] Obtain image and video data to be de-identified;

[0015] The image data and video data to be de-identified are preprocessed, and the preprocessed image data and video data to be de-identified are divided into a training set and a validation set.

[0016] Based on the preset YOLOFP algorithm and the preset desensitization algorithm, the preset neural network is trained using the training set to obtain an initial neural network model, and the initial neural network model is verified using the validation set until the initial neural network model meets the preset conditions. Then, the iterative training of the preset neural network ends to obtain the desensitized pre-trained model. Otherwise, the training set of the preset neural network is adjusted and iterative training continues.

[0017] According to one embodiment of this application, the step of performing quantitative perception training on the desensitization verification model based on the desensitization training data includes:

[0018] The desensitization training data is used to train at a preset precision to obtain a basic desensitization model;

[0019] By inserting pseudo-quantization nodes into the aforementioned desensitization base model, a quantization perception model is obtained.

[0020] The quantization perception model is optimized and retrained using the desensitized training data, and the quantization parameters during the optimization and retraining process are calculated.

[0021] The quantization parameters are used to perform a quantization operation on the quantization perception model to obtain a preset quantization model.

[0022] According to one embodiment of this application, the model deployment based on the desensitization-targeted model includes:

[0023] Check the accuracy of the structure and parameters of the described desensitization site model;

[0024] Based on the preset compilation tools, the checked de-identified fixed-point model is converted into an executable de-identified fixed-point model;

[0025] The performance of the runnable desensitization site-specific model is evaluated, and the parameters of the runnable model and / or the model structure are adjusted and optimized based on the evaluation results.

[0026] Based on the preset visualization tools, the evaluated and operational desensitization and point-of-care model is visually tested, and corresponding repairs and optimizations are made based on the visualization results.

[0027] According to the image and video desensitization model deployment method proposed in this application, desensitization verification data is input into the desensitization pre-trained model for model verification to obtain a desensitization verification model. When the accuracy of the desensitization verification model meets a first preset standard, the desensitization model is converted into a desensitization fixed-point model, and the model is deployed based on the desensitization fixed-point model. Otherwise, the desensitization verification model is subjected to quantization-aware training based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard. Therefore, by using quantization-aware training technology during model training, the accuracy loss of the model after quantization can be further reduced, and the model can be deployed to other devices quickly and efficiently. This solves the problem in existing technologies where well-pre-trained models suffer accuracy loss during model quantization and conversion to fixed-point models, resulting in poor model deployment performance, and improves the accuracy of the desensitization algorithm.

[0028] To achieve the above objectives, a second aspect of this application provides a deployment apparatus for an image and video desensitization model, comprising:

[0029] The acquisition module is used to acquire de-identification verification data and de-identification training data;

[0030] The verification module is used to input the de-identification verification data into the de-identification pre-training model for model verification, and obtain the de-identification verification model.

[0031] The judgment module is used to determine whether the accuracy of the desensitization verification model meets the first preset standard;

[0032] The processing module is configured to convert the desensitization model into a desensitization fixed-point model and deploy the model based on the desensitization fixed-point model when the accuracy of the desensitization verification model meets the first preset standard; otherwise, it performs quantitative perception training on the desensitization verification model based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard.

[0033] According to one embodiment of this application, before deploying the model based on the desensitization and localization model, the processing module is further configured to:

[0034] Determine whether the accuracy of the desensitization point model meets the second preset standard;

[0035] If the accuracy of the desensitized fixed-point model meets the second preset standard, then the model is deployed based on the desensitized fixed-point model; otherwise, the step of performing quantitative perception training on the desensitized verification model based on the desensitized training data is repeated until the accuracy of the desensitized verification model meets the first preset standard.

[0036] According to one embodiment of this application, before inputting the de-identification verification data into the de-identification pre-trained model for model verification, the verification module is further configured to:

[0037] Obtain image and video data to be de-identified;

[0038] The image data and video data to be de-identified are preprocessed, and the preprocessed image data and video data to be de-identified are divided into a training set and a validation set.

[0039] Based on the preset YOLOFP algorithm and the preset desensitization algorithm, the preset neural network is trained using the training set to obtain an initial neural network model, and the initial neural network model is verified using the validation set until the initial neural network model meets the preset conditions. Then, the iterative training of the preset neural network ends to obtain the desensitized pre-trained model. Otherwise, the training set of the preset neural network is adjusted and iterative training continues.

[0040] According to one embodiment of this application, the processing module is specifically used for:

[0041] The desensitization training data is used to train at a preset precision to obtain a basic desensitization model;

[0042] By inserting pseudo-quantization nodes into the aforementioned desensitization base model, a quantization perception model is obtained.

[0043] The quantization perception model is optimized and retrained using the desensitized training data, and the quantization parameters during the optimization and retraining process are calculated.

[0044] The quantization parameters are used to perform a quantization operation on the quantization perception model to obtain a preset quantization model.

[0045] According to one embodiment of this application, the processing module is specifically used for:

[0046] Check the accuracy of the structure and parameters of the described desensitization site model;

[0047] Based on the preset compilation tools, the checked de-identified fixed-point model is converted into an executable de-identified fixed-point model;

[0048] The performance of the runnable desensitization site-specific model is evaluated, and the parameters of the runnable model and / or the model structure are adjusted and optimized based on the evaluation results.

[0049] Based on the preset visualization tools, the evaluated and operational desensitization and point-of-care model is visually tested, and corresponding repairs and optimizations are made based on the visualization results.

[0050] According to the image and video desensitization model deployment apparatus proposed in this application, a desensitization verification model can be obtained by inputting desensitization verification data into a desensitization pre-trained model for model verification. When the accuracy of the desensitization verification model meets a first preset standard, the desensitization model is converted into a desensitization fixed-point model, and the model is deployed based on the desensitization fixed-point model. Otherwise, the desensitization verification model is subjected to quantization-aware training based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard. Therefore, by using quantization-aware training technology during model training, the accuracy loss of the model after quantization can be further reduced, and the model can be deployed to other devices quickly and efficiently. This solves the problem in existing technologies where well-pre-trained models suffer accuracy loss during model quantization and conversion to fixed-point models, resulting in poor model deployment performance, and improves the accuracy of the desensitization algorithm.

[0051] To achieve the above objectives, a third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for deploying image and video desensitization models as described in the above embodiments.

[0052] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the method for deploying image and video desensitization models as described in the above embodiments.

[0053] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0054] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0055] Figure 1 This is a flowchart illustrating a method for deploying an image and video desensitization model according to an embodiment of this application.

[0056] Figure 2 This is a schematic diagram of floating-point to fixed-point quantization according to an embodiment of this application;

[0057] Figure 3 This is a flowchart of a quantitative perception training process according to an embodiment of this application;

[0058] Figure 4 This is a schematic diagram illustrating the deployment of a desensitization point-based model according to an embodiment of this application;

[0059] Figure 5 This is an overall schematic diagram of the deployment of an image and video desensitization model according to an embodiment of this application;

[0060] Figure 6 This is a schematic diagram of the simulation results of a desensitization point-based model deployment according to an embodiment of this application;

[0061] Figure 7 This is a schematic diagram of the core architecture of a desensitized pre-trained model according to an embodiment of this application;

[0062] Figure 8 This is a schematic diagram of the network architecture of a preset YOLOFP algorithm according to an embodiment of this application;

[0063] Figure 9 This is a flowchart illustrating a deployment method for another image and video desensitization model provided according to an embodiment of this application;

[0064] Figure 10 This is a block diagram of a deployment apparatus for an image and video desensitization model provided according to an embodiment of this application;

[0065] Figure 11 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0066] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0067] The following describes, with reference to the accompanying drawings, a method, apparatus, device, and medium for deploying image and video desensitization models according to embodiments of this application. First, the method for deploying image and video desensitization models according to embodiments of this application will be described with reference to the accompanying drawings.

[0068] Figure 1 This is a flowchart of a method for deploying an image and video desensitization model according to an embodiment of this application.

[0069] For example, such as Figure 1 As shown, the deployment method of this image and video desensitization model includes the following steps:

[0070] In step S101, desensitization verification data and desensitization training data are obtained.

[0071] Understandably, desensitization refers to transforming sensitive information to protect the privacy and security of the original data while ensuring its availability. Therefore, it is necessary to first obtain desensitized verification data and desensitized training data that have undergone desensitization processing.

[0072] In step S102, the desensitization verification data is input into the desensitization pre-training model for model verification to obtain the desensitization verification model.

[0073] The desensitization pre-training model is a model that has been pre-trained for further optimization. In this embodiment, under the premise of designing a desensitization algorithm, floating-point numbers (a computer data type used to represent real numbers, with high precision and range) are used for initial training of the model to obtain a floating-point model with better training results, which is then used as the desensitization pre-training model.

[0074] In other words, by inputting the desensitized verification data into the desensitized pre-trained model for model verification, and by adjusting the model parameters to optimize the model performance, a desensitized verification model can be obtained.

[0075] In step S103, it is determined whether the accuracy of the desensitization verification model meets the first preset standard.

[0076] Understandably, accuracy is an indicator of model performance. The first preset standard can be a model accuracy threshold pre-set according to specific application scenarios and requirements. If the accuracy of the desensitization verification model is greater than or equal to this accuracy threshold, it is considered that the accuracy of the desensitization verification model meets the first preset standard.

[0077] In step S104, if the accuracy of the desensitization verification model meets the first preset standard, the desensitization model is converted into a desensitization fixed-point model, and the model is deployed based on the desensitization fixed-point model; otherwise, the desensitization verification model is subjected to quantitative perception training based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard.

[0078] Among them, desensitized fixed-point models are applicable to specific hardware or deployment scenarios. Converting floating-point models into fixed-point (fixed-precision) models can improve the model's running efficiency. Quantization-aware training (QAT) is a method to reduce the size of deep learning models, improve running efficiency, and minimize the accuracy loss caused by quantization.

[0079] In other words, when the accuracy of the desensitization verification model meets the first preset standard, the desensitization model can be converted into a desensitization fixed-point model and deployed to other low-power edge devices for practical application; if the accuracy of the desensitization verification model does not meet the first preset standard, the desensitization training data can be used to perform quantization perception training on the desensitization verification model, that is, the desensitization verification model (floating-point model) is quantized and then retrained until the accuracy of the desensitization verification model reaches the requirement, that is, meets the first preset standard.

[0080] To facilitate understanding, the following details how to perform quantitative perception training on the desensitization verification model based on desensitized training data.

[0081] As one possible approach, in some embodiments, the desensitization verification model is subjected to quantization-aware training based on desensitized training data, including: training the desensitized training data with a preset precision to obtain a desensitized base model; inserting pseudo-quantization nodes into the desensitized base model to obtain a quantization-aware model; optimizing and retraining the quantization-aware model using the desensitized training data, and calculating the quantization parameters during the optimization and retraining process; and performing quantization operations on the quantization-aware model using the quantization parameters to obtain a preset quantization model.

[0082] Understandably, quantization is a model compression technique. This application maps the parameters of the desensitized pre-trained model from the floating-point (Float32) range to the discrete integer (INT8) range, such as... Figure 2 As shown, in the INT8 model quantization of this application embodiment, the original floating-point number r is quantized to the integer range of [-128, 127].

[0083] The quantitative calculation formula is shown in equation (1):

[0084] r = S(qZ); (1)

[0085] Where r is the real value of Float32, q is the quantization value of INT8, S is the scale factor, and Z is the zero point.

[0086] Furthermore, the calculations of S and Z are shown in equations (2) and (3), respectively:

[0087]

[0088] Where, r max r is the maximum value of a Float32 floating-point number. min q is the minimum value of a Float32 floating-point number. max q is the maximum value after quantization. min The minimum value after quantization (e.g.) Figure 6 As shown, q max =127, q min =-128), Z is the quantization value corresponding to the floating-point number r=0 of Float32. Thus, the quantization formula of the embodiment of this application can be expressed as:

[0089]

[0090] Specifically, such as Figure 3 As shown, the model is trained using desensitized training data at a preset precision (e.g., Float32) to obtain a desensitized base model. Then, pseudo-quantization nodes are inserted into the desensitized base model to obtain a quantization-aware model. The quantization-aware model is optimized and retrained using the desensitized training data. The pseudo-quantization nodes can simulate the quantization process and save the quantization parameters calculated during the optimization and retraining process. The quantization parameters are used to perform quantization operations on the quantization-aware model to obtain the preset quantization model (i.e., the INT8 model).

[0091] It is understandable that pseudo-quantization is actually the error caused by simulated quantization rounding, and the calculation formula is as follows:

[0092]

[0093] Where r is the real number to be quantized, [a, b] is the numerical range to be quantized, n is the fixed number of bits for quantization, and s(a, b, n) is the quantization scaling factor. In this embodiment, n = 8 is INT8 quantization.

[0094] Therefore, quantization-aware training technology introduces pseudo-quantization during model training to simulate the errors introduced during quantization. It treats the quantization error as a kind of training noise, allowing the model to learn and adapt to this noise during training and continuously optimize the accuracy to obtain the best quantization parameters. This can further reduce the accuracy loss of the model after quantization, and at the same time, quickly and efficiently deploy the model to other low-power edge devices.

[0095] The model deployment process will be explained in detail below.

[0096] As one possible approach, in some embodiments, model deployment is based on a de-identified fixed-point model, including: checking the accuracy of the structure and parameters of the de-identified fixed-point model; converting the checked de-identified fixed-point model into a runnable de-identified fixed-point model using a preset compilation tool; performing performance evaluation on the runnable de-identified fixed-point model, and adjusting the runnable model parameters and / or optimizing the model structure based on the evaluation results; and performing visual testing on the evaluated runnable de-identified fixed-point model using a preset visualization tool, and making corresponding repairs and optimizations based on the visualization results.

[0097] Specifically, such as Figure 4 As shown, the model deployment process mainly involves engineered model checking, compilation, performance testing, and visualization testing. Model checking primarily refers to checking the accuracy of the structure and parameters of the anonymized fixed-point model, verifying the integrity of the model structure, and ensuring correct connections between layers. Parameter checking mainly involves verifying whether parameters such as model weights have been correctly loaded. Compilation mainly refers to converting the checked anonymized fixed-point model into a runnable anonymized fixed-point model using pre-defined compilation tools (such as TensorRT), which helps reduce the computational resources and storage space requirements during model deployment. Performance testing mainly involves evaluating the performance of the runnable anonymized fixed-point model by calculating metrics such as recall and precision, checking whether the anonymized data still retains valid information, and whether sensitive information is effectively protected. If the evaluation results are unsatisfactory, the parameters of the runnable model (such as learning rate, batch size, etc.) can be adjusted to improve the model's performance. If there are problems with the model results, the model can be reconstructed or new components can be introduced to optimize the model structure. Visualization testing mainly refers to conducting visualization tests on the runnable desensitized fixed-point model after evaluation based on preset visualization tools (such as TensorBoard). That is, displaying the model's structure and data flow helps to understand the model's operation process more intuitively and discover potential problems. Through visualization results, the model's performance bottlenecks and optimization space can be further analyzed. Based on the problems found, the model can be targeted for repair and optimization. After optimization, performance evaluation and visualization testing can be carried out again to ensure that the model's performance meets the needs of actual applications.

[0098] After the model is deployed, in practical applications, such as Figure 5 As shown, the original data (i.e., image data and video data to be desensitized) is input into the desensitization fixed-point model. The input original data is preprocessed (e.g., data conversion, data encoding, image frame extraction, etc.). Sensitive areas are detected in the preprocessed original data, and the sensitive areas are desensitized. After data conversion and data decoding operations, the desensitized original data (i.e., image desensitized data and video desensitized data) can be output.

[0099] Among them, the simulation results of model deployment are as follows Figure 6 As shown, Figure 6 (a) is the original input image. Figure 6 (b) shows the simulation results after deployment and desensitization. As can be seen from the figure, sensitive information (faces, license plates) in video and images in the vehicle network data was located, detected, and effectively desensitized.

[0100] Furthermore, in some embodiments, before deploying the model based on the desensitized fixed-point model, the method further includes: determining whether the accuracy of the desensitized fixed-point model meets a second preset standard; if the accuracy of the desensitized fixed-point model meets the second preset standard, then deploying the model based on the desensitized fixed-point model; otherwise, re-executing the quantitative perception training of the desensitized verification model based on the desensitized training data until the accuracy of the desensitized verification model meets the first preset standard.

[0101] In other words, before deploying a model based on a de-identified fixed-point model, it is necessary to ensure that the model's accuracy meets certain standards to guarantee its reliability and accuracy in practical applications. After obtaining the de-identified fixed-point model, it is first necessary to determine whether the model's accuracy meets a second preset standard. This second preset standard can be an accuracy threshold set according to specific application scenarios and requirements. If the accuracy of the de-identified fixed-point model meets the second preset standard, it indicates that the model's performance has reached or exceeded the expected level, and the model can be deployed. If the accuracy of the de-identified fixed-point model does not meet the second preset standard, the model needs to be further optimized. That is, the quantitative perceptual training of the de-identified verification model based on the de-identified training data should be re-executed until the accuracy of the de-identified verification model meets the first preset standard, and so on, until the accuracy of the finally obtained de-identified fixed-point model meets the requirements.

[0102] It is understandable that the design of the desensitization pre-training model is a prerequisite for the quantitative perception training and deployment of the face and license plate desensitization algorithm in the embodiments of this application. The following details how to obtain the desensitization pre-training model.

[0103] In some embodiments, before inputting the desensitization verification data into the desensitization pre-trained model for model verification, the method further includes: acquiring image data to be desensitized and video data to be desensitized; preprocessing the image data to be desensitized and video data to be desensitized, and dividing the preprocessed image data to be desensitized and video data to be desensitized into a training set and a validation set; training a preset neural network using the training set based on a preset YOLOFP algorithm and a preset desensitization algorithm to obtain an initial neural network model, and validating the initial neural network model using the validation set, until the initial neural network model meets preset conditions, ending the iterative training of the preset neural network to obtain the desensitization pre-trained model; otherwise, adjusting the training set of the preset neural network and continuing iterative training.

[0104] The implementation of the face and license plate desensitization algorithm in this application embodiment lies in the process of accurately locating face and license plate information in a specified area and blurring / occluding this sensitive information, namely: accurate target (face, license plate) detection and sensitive area information processing. As traditional target detection algorithms are gradually being replaced by deep learning methods, the YOLO (You Only Look Once) series of detection algorithms has become the mainstream trend in target detection in terms of detection accuracy, running speed, computational cost, model deployment, and low power consumption. The YOLOv5Face algorithm treats face detection as a general target detection task based on the YOLO algorithm; similarly, license plate detection can also be treated as a general target detection task and implemented based on the ideas of the YOLOv5 algorithm. Therefore, this application embodiment designs a new face and license plate detection algorithm—YOLOFP (You Only Look Once Face and Plate)—based on the YOLO algorithm, and then uses a re-desensitization algorithm to achieve the desensitization process of face and license plate.

[0105] Specifically, the process involves acquiring image and video data containing sensitive information to be de-identified, and preprocessing this data (e.g., data conversion, data encoding, image frame extraction) to facilitate model training. The preprocessed image and video data are then randomly divided into training and validation sets. The training set is used for model training, and the validation set is used for model performance evaluation. A suitable neural network architecture, such as a YOLO-based object detection algorithm network, is selected as the pre-defined neural network. The pre-defined YOLOFP algorithm and a pre-defined de-identification algorithm are integrated into this neural network. The pre-defined YOLOFP algorithm modifies the YOLO algorithm's head to achieve license plate detection in different scenarios. The goal of the pre-defined YOLOFP algorithm is to provide a model combination for different applications, achieving the best trade-off between performance and speed on cloud, embedded, or mobile devices. Using the pre-defined YOLOFP algorithm, [the following steps can be taken]. Image and video data to be anonymized are detected to identify face and license plate regions. Pre-defined anonymization algorithms are used to anonymize these regions, resulting in anonymized face and license plate regions. Based on the pre-defined YOLOFP algorithm and the pre-defined anonymization algorithm, a pre-defined neural network is trained using the image and video data to be anonymized in the training set. This yields an initial neural network model that outputs the anonymized face and license plate regions. After each training cycle, the model's performance is evaluated using a validation set. If the initial neural network model meets pre-defined conditions (e.g., the model's performance on the validation set reaches a set threshold), training stops, and the current initial neural network model is saved as a pre-training model for anonymization. If the initial neural network model does not meet the pre-defined conditions, the training set can be adjusted (e.g., data augmentation, re-partitioning the dataset) to iteratively train the pre-defined neural network until the model meets the pre-defined conditions.

[0106] Among them, the core architecture of the desensitization pre-training model is as follows: Figure 7 As shown, it includes a preset YOLOFP algorithm and a preset de-identification algorithm. The network architecture of the preset YOLOFP algorithm is as follows: Figure 8As shown, the model consists of modules such as Backbone, Neck, and Head. The main function of the Backbone module is feature extraction. It consists of multiple stages of Backbone, namely Stage 1 Backbone (CBR*N1), Stage 2 Backbone (CBR*N2), Stage 3 Backbone (CBR*N3), and Stage 4 Backbone (CBR*N4). Each stage of Backbone contains a certain number of CBR (Conv+Bn+ReLU) fusion operator units. The CBR unit is a combination of convolution (Conv) followed by batch normalization (Bn) and ReLU activation functions. This combination helps improve the training stability and feature representation ability of the model. Here, N1, N2, N3, and N4 are integers, and CBR*N1 indicates that the fused CBR operator unit appears N1 times. The main function of the Neck module is to further process and fuse the features extracted by the Backbone module, including CBR, Upsample, and Concat operator units. Upsample is used to improve the feature map (Feature) The resolution of a feature map (Map) can be converted from a low-resolution feature map to a high-resolution feature map through interpolation and other methods. In object detection, high-resolution feature maps are very important for accurate target localization. The Concat operation is a feature fusion algorithm that can connect feature maps from different layers along the channel dimension. Through the Concat operation, a low-resolution feature map with rich semantic information is combined with a high-resolution feature map with rich localization information to form an output feature map that contains both rich semantic information and retains localization information. This cross-layer connection can take into account both details and perception range, thereby improving the accuracy of object detection. The Head module includes CNN (Convolutional Neural Network) units, which are used to extract feature maps and convert them into the output of the model. These outputs can include the detected target category, coordinate region, confidence score, and key point information for faces and license plates.

[0107] Furthermore, after detecting the face and license plate regions, a preset desensitization algorithm is used to desensitize the regions containing faces and license plates in each frame. This is achieved by directly replacing and erasing with color blocks, or by blurring the pixel data on the original image with color blocks, ensuring that the desensitized data is irreversible and unrecoverable. This embodiment uses a Gaussian desensitization algorithm to desensitize the license plate region:

[0108] a) Stokes function:

[0109] GaussianBlur(src,ksize,sigmaX,dst=None,sigmaY=None,borderType=None);

[0110] b) Detailed explanation of Gaussian desensitization function parameters:

[0111] Src: The original input image;

[0112] Ksize: Convolution kernel (Gaussian kernel) size (ksize(w,h), w - pixel width, h - pixel height);

[0113] sigmaX: The filter kernel in the X direction, representing the standard deviation of the convolution kernel in the X direction, which controls the weight ratio;

[0114] Dst: Output image, can be None.

[0115] sigmaY: The filter kernel in the Y direction, representing the standard deviation of the convolution kernel in the Y direction, which controls the weight ratio;

[0116] borderType: Infers a boundary mode for the outer pixels of the image. The default value is BORDER_DEFAULT.

[0117] After face and license plate anonymization, the face anonymization pass evaluation criteria must meet regulatory requirements to demonstrate the effectiveness of the algorithm deployment method in this application embodiment. The specific face anonymization pass evaluation criteria are shown in Table 1 below:

[0118] Table 1

[0119] category Detection rate False positive rate Face The face detection rate should be no less than 90%. The false positive rate for facial recognition is no more than 5%. license plate The license plate detection rate should be no less than 90%. The license plate false detection rate is no higher than 10%.

[0120] The detection rate and false detection rate are calculated as follows:

[0121] Detection rate = the proportion of correctly detected items of a particular category out of the total number of items in that category in the detection results;

[0122] False positive rate = the proportion of non-category results in the total number of results for a given category.

[0123] In other words, after the desensitization is completed, the criteria for judging whether the desensitization is qualified are mainly based on the detection rate and the false detection rate. The qualified criteria for face desensitization is that the face detection rate is greater than or equal to 90% and the face false detection rate is less than or equal to 5%; the qualified criteria for license plate desensitization is that the license plate detection rate is greater than or equal to 90% and the license plate false detection rate is less than or equal to 10%.

[0124] To facilitate those skilled in the art to further understand the deployment method of the image and video desensitization model proposed in the embodiments of this application, the following is combined with... Figure 9Further details are provided.

[0125] like Figure 9 As shown, the deployment method of this image and video desensitization model includes the following steps:

[0126] Step S901: Construct a desensitization pre-training model.

[0127] Step S902: Generate a desensitization verification model based on the desensitization verification data.

[0128] Step S903: Determine whether the accuracy of the desensitization verification model meets the standard. If yes, proceed to step S905; otherwise, proceed to step S904.

[0129] Step S904: Perform quantitative perception training based on the desensitized training data. Step S903 is then executed after the quantitative perception training is complete.

[0130] Step S905: The desensitization verification model is converted into a desensitization fixed-point model.

[0131] Step S906: Determine whether the accuracy of the desensitization and point-based model meets the standard. If yes, proceed to step S907; otherwise, proceed to step S904, and then proceed to step S903.

[0132] Step S907: Deployment of the desensitization and fixed-point model.

[0133] According to the image and video desensitization model deployment method proposed in this application, desensitization verification data is input into the desensitization pre-trained model for model verification to obtain a desensitization verification model. When the accuracy of the desensitization verification model meets a first preset standard, the desensitization model is converted into a desensitization fixed-point model, and the model is deployed based on the desensitization fixed-point model. Otherwise, the desensitization verification model is subjected to quantization-aware training based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard. Therefore, by using quantization-aware training technology during model training, the accuracy loss of the model after quantization can be further reduced, and the model can be deployed to other devices quickly and efficiently. This solves the problem in existing technologies where well-pre-trained models suffer accuracy loss during model quantization and conversion to fixed-point models, resulting in poor model deployment performance, and improves the accuracy of the desensitization algorithm.

[0134] Next, with reference to the accompanying drawings, a deployment apparatus for an image and video desensitization models proposed according to embodiments of this application is described.

[0135] Figure 10 This is a block diagram of a deployment apparatus for an image and video desensitization model according to an embodiment of this application.

[0136] like Figure 10As shown, the deployment device 10 for the image and video desensitization model includes: an acquisition module 100, a verification module 200, a judgment module 300, and a processing module 400.

[0137] Among them, the acquisition module 100 is used to acquire desensitization verification data and desensitization training data;

[0138] The verification module 200 is used to input the desensitization verification data into the desensitization pre-training model for model verification, and obtain the desensitization verification model.

[0139] The judgment module 300 is used to determine whether the accuracy of the desensitization verification model meets the first preset standard;

[0140] The processing module 400 is used to convert the desensitized model into a desensitized fixed-point model when the accuracy of the desensitized verification model meets the first preset standard, and to deploy the model based on the desensitized fixed-point model; otherwise, it performs quantitative perception training on the desensitized verification model based on the desensitized training data until the accuracy of the desensitized verification model meets the first preset standard.

[0141] Furthermore, in some embodiments, before deploying the model based on the desensitized point model, the processing module 400 is also used to:

[0142] Determine whether the accuracy of the desensitization point model meets the second preset standard;

[0143] If the accuracy of the desensitized fixed-point model meets the second preset standard, then the model is deployed based on the desensitized fixed-point model; otherwise, the quantitative perception training of the desensitized verification model based on the desensitized training data is re-executed until the accuracy of the desensitized verification model meets the first preset standard.

[0144] Furthermore, in some embodiments, before inputting the de-identified verification data into the de-identified pre-trained model for model verification, the verification module 200 is also used for:

[0145] Obtain image and video data to be de-identified;

[0146] The image and video data to be de-identified are preprocessed, and the preprocessed image and video data to be de-identified are divided into training and validation sets.

[0147] Based on the preset YOLOFP algorithm and the preset desensitization algorithm, the preset neural network is trained using the training set to obtain an initial neural network model, and the initial neural network model is verified using the validation set until the initial neural network model meets the preset conditions. Then, the iterative training of the preset neural network ends to obtain the desensitized pre-trained model. Otherwise, the training set of the preset neural network is adjusted and iterative training continues.

[0148] Furthermore, in some embodiments, the processing module 400 is specifically used for:

[0149] The basic desensitization model is obtained by training with the desensitized training data at a preset precision.

[0150] By inserting pseudo-quantization nodes into the desensitization base model, a quantization perception model is obtained.

[0151] The quantization perception model was optimized and retrained using desensitized training data, and the quantization parameters during the optimization and retraining process were calculated.

[0152] The quantization model is quantized using quantization parameters to obtain a preset quantization model.

[0153] Furthermore, in some embodiments, the processing module 400 is specifically used for:

[0154] Check the accuracy of the structure and parameters of the desensitization site-specific model;

[0155] Based on the preset compilation tools, the checked de-identified fixed-point model is converted into an executable de-identified fixed-point model;

[0156] The performance of the runnable desensitization site-specific model is evaluated, and the parameters of the runnable model and / or the model structure is adjusted and optimized based on the evaluation results.

[0157] Based on the preset visualization tools, the evaluated and operational desensitization and point-of-care model is visually tested, and corresponding repairs and optimizations are made based on the visualization results.

[0158] It should be noted that the explanation of the above-described method for deploying image and video desensitization models also applies to the deployment apparatus for image and video desensitization models in this embodiment, and will not be repeated here.

[0159] According to the image and video desensitization model deployment apparatus proposed in this application, a desensitization verification model can be obtained by inputting desensitization verification data into a desensitization pre-trained model for model verification. When the accuracy of the desensitization verification model meets a first preset standard, the desensitization model is converted into a desensitization fixed-point model, and the model is deployed based on the desensitization fixed-point model. Otherwise, the desensitization verification model is subjected to quantization-aware training based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard. Therefore, by using quantization-aware training technology during model training, the accuracy loss of the model after quantization can be further reduced, and the model can be deployed to other devices quickly and efficiently. This solves the problem in existing technologies where well-pre-trained models suffer accuracy loss during model quantization and conversion to fixed-point models, resulting in poor model deployment performance, and improves the accuracy of the desensitization algorithm.

[0160] Figure 11A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0161] The memory 1101, the processor 1102, and the computer program stored on the memory 1101 and executable on the processor 1102.

[0162] When the processor 1102 executes the program, it implements the deployment method of the image and video desensitization model provided in the above embodiments.

[0163] Furthermore, electronic devices also include:

[0164] Communication interface 1103 is used for communication between memory 1101 and processor 1102.

[0165] The memory 1101 is used to store computer programs that can run on the processor 1102.

[0166] The memory 1101 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0167] If the memory 1101, processor 1102, and communication interface 1103 are implemented independently, then the communication interface 1103, memory 1101, and processor 1102 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0168] Optionally, in a specific implementation, if the memory 1101, processor 1102, and communication interface 1103 are integrated on a single chip, then the memory 1101, processor 1102, and communication interface 1103 can communicate with each other through an internal interface.

[0169] The processor 1102 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0170] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for deploying image and video desensitization models.

[0171] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0172] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0173] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for deploying a desensitization model of images and videos, characterized in that, The method comprises the following steps: obtaining desensitization verification data and desensitization training data; inputting the desensitization verification data into a desensitization pre-training model for model verification to obtain a desensitization verification model; judging whether the accuracy of the desensitization verification model meets a first preset standard; if the accuracy of the desensitization verification model meets the first preset standard, converting the desensitization model into a desensitization fixed-point model and performing model deployment based on the desensitization fixed-point model, otherwise performing quantization-aware training on the desensitization verification model based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard; wherein the desensitization pre-training model comprises a preset YOLOFP algorithm and a preset desensitization algorithm, the preset YOLOFP algorithm comprises a Backbone module, a Neck module and a Head module, wherein the Backbone module is composed of Backbone units of multiple stages, each stage of Backbone unit comprises a preset number of CBR fusion operator units, the Neck module comprises the CBR fusion operator unit, an Upsample unit and a Concat operator unit, and the Head module comprises a CNN unit.

2. The method of claim 1, wherein, Before performing model deployment based on the desensitization fixed-point model, further comprising: judging whether the accuracy of the desensitization fixed-point model meets a second preset standard; if the accuracy of the desensitization fixed-point model meets the second preset standard, performing model deployment based on the desensitization fixed-point model, otherwise re-executing the quantization-aware training on the desensitization verification model based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard.

3. The method of claim 1, wherein, Before inputting the desensitization verification data into the desensitization pre-training model for model verification, further comprising: obtaining image desensitization data and video desensitization data; preprocessing the image desensitization data and the video desensitization data, and dividing the preprocessed image desensitization data and the video desensitization data into a training set and a verification set; based on the preset YOLOFP algorithm and the preset desensitization algorithm, training a preset neural network using the training set to obtain an initial neural network model, and verifying the initial neural network model using the verification set until the initial neural network model meets a preset condition, ending the iterative training of the preset neural network to obtain the desensitization pre-training model, otherwise adjusting the training set of the preset neural network and continuing the iterative training.

4. The method of claim 1, wherein, The quantization-aware training on the desensitization verification model based on the desensitization training data comprises: training the desensitization training data at a preset accuracy to obtain a desensitization base model; inserting a pseudo-quantization node into the desensitization base model to obtain a quantization-aware model; optimizing and retraining the quantization-aware model using the desensitization training data, and calculating the quantization parameters in the optimization and retraining process; performing quantization operation on the quantization-aware model using the quantization parameters to obtain a preset quantization model.

5. The method of claim 1, wherein, The model deployment based on the desensitization fixed-point model comprises: checking the accuracy of the structure and parameters of the desensitization fixed-point model; converting the checked desensitization fixed-point model into a runnable desensitization fixed-point model based on a preset compiling tool; performing performance evaluation on the runnable desensitization fixed-point model, and adjusting the runnable model parameters and / or optimizing the model structure according to the evaluation result; based on a preset visualization tool, visualizing the evaluated runnable desensitization fixed-point model, and performing corresponding repair and optimization based on the visualization result.

6. An apparatus for deploying a desensitization model of images and videos, characterized by, comprise: an acquisition module configured to acquire desensitization verification data and desensitization training data; a verification module configured to input the desensitization verification data into a desensitization pre-training model for model verification to obtain a desensitization verification model; a judgment module configured to judge whether the accuracy of the desensitization verification model meets a first preset standard; a processing module configured to, when the accuracy of the desensitization verification model meets the first preset standard, convert the desensitization model into a desensitization fixed-point model and perform model deployment based on the desensitization fixed-point model, or otherwise perform quantitative perception training on the desensitization verification model based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard; wherein the desensitization pre-training model comprises a preset YOLOFP algorithm and a preset desensitization algorithm, the preset YOLOFP algorithm comprises a Backbone module, a Neck module and a Head module, the Backbone module is composed of Backbone units of multiple stages, each stage of Backbone unit contains a preset number of CBR fusion operator units, the Neck module comprises the CBR fusion operator unit, an Upsample unit and a Concat operator unit, and the Head module comprises a CNN unit.

7. The apparatus of claim 6, wherein, Before performing model deployment based on the desensitization fixed-point model, the processing module is further configured to: judge whether the accuracy of the desensitization fixed-point model meets a second preset standard; if the accuracy of the desensitization fixed-point model meets the second preset standard, perform model deployment based on the desensitization fixed-point model, otherwise re-execute the quantitative perception training on the desensitization verification model based on the desensitization training data until the accuracy of the desensitization verification model meets the first preset standard.

8. The apparatus of claim 6, wherein, Before inputting the desensitization verification data into the desensitization pre-training model for model verification, the verification module is further configured to: acquire image desensitization data and video desensitization data; preprocess the image desensitization data and the video desensitization data, and divide the preprocessed image desensitization data and the video desensitization data into a training set and a validation set; Based on the preset YOLOFP algorithm and the preset desensitization algorithm, an initial neural network model is obtained by training a preset neural network using the training set, and the initial neural network model is verified using the verification set until the initial neural network model meets a preset condition, and the iteration training of the preset neural network is ended to obtain the desensitization pre-training model, otherwise the training set of the preset neural network is adjusted and the iteration training is continued.

9. An electronic device, comprising: Comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the deployment method of the image and video desensitization model according to any one of claims 1-5.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the deployment method of the image and video desensitization model according to any one of claims 1-5.

Images