Image data desensitization method and apparatus

Abstract

The application discloses an image data desensitization method and device, which comprises the following steps: determining a face detection model for inference according to application scene requirements, a device performance feature library and computing device data of a computing device; monitoring computing device hardware parameters of the computing device in real time to determine a processor of the computing device for current inference; performing an inference task on an input face image based on the determined face detection model on the processor for current inference to obtain at least one face region, wherein the inference task is migrated according to the update of the processor for current inference; when it is determined that the face region needs desensitization processing, determining a corrected mosaic intensity according to an application scene adjustment factor and a desensitization risk index corresponding to each face region, performing desensitization processing on each face region according to the corrected mosaic intensity, and writing the face region after the desensitization processing back to the face image. The application can automatically identify face regions and perform desensitization processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image data desensitization method and device. BACKGROUND

[0002] This section is intended to provide background information to facilitate an understanding of embodiments of the application as set forth in the claims. The description herein does not constitute admission of prior art.

[0003] With the rapid development of artificial intelligence and big data technology, image data (including image data in video data) plays a crucial role in many fields. These data often contain sensitive information, such as facial features, home scenes, etc., which may lead to privacy leakage if not properly handled. At the same time, the phenomenon of data silos is widespread, and the lack of efficient privacy protection mechanisms hinders the sharing and utilization of data. Therefore, there is currently a need for an image data processing scheme that can automatically identify face regions and perform desensitization processing to protect personal privacy while promoting the legal and compliant sharing of data. SUMMARY

[0004] Embodiments of the present application provide an image data desensitization method that can automatically identify face regions and perform desensitization processing, comprising:

[0005] According to the application scenario requirements, the device performance feature library and the computing device data of the computing device, determine the face detection model for inference, the device performance feature library is used to record the inference effect data of different types of face detection models for inference by different computing devices under different computing device hardware parameters;

[0006] Real-time monitoring of the computing device hardware parameters of the computing device, determining the processor of the computing device for current inference;

[0007] Performing inference tasks on the input face image based on the determined face detection model on the processor for current inference, obtaining at least one face region, wherein the inference task is migrated according to the update of the processor for current inference;

[0008] When it is determined that the face region needs to be desensitized, the modified mosaic intensity is determined according to the application scenario adjustment factor and the desensitization risk index corresponding to each face region, and each face region is desensitized according to the modified mosaic intensity, and the face region after desensitization is written back to the face image.

[0009] Embodiments of the present application provide an image data desensitization device that can automatically identify face regions and perform desensitization processing, the device comprising:

[0010] The face detection model selection module is configured to determine a face detection model for inference according to application scenario requirements, a device performance feature library, and computing device data of the computing device, the device performance feature library being configured to record inference effect data of different types of face detection models for inference by different computing devices under different computing device hardware parameters;

[0011] The processor selection module is configured to monitor computing device hardware parameters of the computing device in real time and determine a processor of the computing device for current inference;

[0012] The face detection module is configured to perform an inference task on an input face image based on the determined face detection model on the processor for current inference to obtain at least one face region, wherein the inference task is migrated according to an update of the processor for current inference;

[0013] The desensitization processing module is configured to, when it is determined that the face region needs to be desensitized, determine a corrected mosaic intensity according to an application scenario adjustment factor and a desensitization risk index corresponding to each face region, desensitize each face region according to the corrected mosaic intensity, and write the desensitized face region back to the face image.

[0014] The embodiment of the application further provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the image data desensitization method when executing the computer program.

[0015] The embodiment of the application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image data desensitization method.

[0016] The embodiment of the application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the image data desensitization method.

[0017] In the embodiment of the present application, according to the application scenario requirement, the device performance feature library and the computing device data of the computing device, a face detection model for inference is determined, and the device performance feature library is used to record the inference effect data of different types of face detection models for inference under different computing device hardware parameters of different computing devices; the computing device hardware parameters of the computing device are monitored in real time, and the processor of the current computing device for inference is determined; the inference task of the input face image is performed on the processor for current inference based on the determined face detection model, and at least one face region is obtained, wherein the inference task is migrated according to the update of the processor for current inference; when it is determined that the face region needs to be desensitized, the correction mosaic intensity is determined according to the application scenario adjustment factor and the desensitization risk index corresponding to each face region, and each face region is desensitized according to the correction mosaic intensity, and the face region after desensitization is written back to the face image. Through the decision tree combined with the application scenario, the device performance feature library and the real-time hardware parameter selection model, the model is accurately matched with the hardware capability of the computing device, the resource waste is reduced while the inference effect is ensured, and the inference speed and stability are improved. The processor is monitored in real time and the inference task is migrated, so that the inference process is not interrupted when the processor dynamically changes (such as switching from CPU to GPU), the system robustness is improved, and the device load fluctuation and other scenes are adapted. The correction mosaic intensity is determined by combining the application scenario adjustment factor and the desensitization risk index, so that the desensitization processing can meet the privacy protection requirements of different scenes (such as enhancing desensitization in high-risk scenes), and the necessary image information can be preserved, balancing privacy and usability. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief descriptions will be given below for the drawings needed to be used in the embodiments or prior art descriptions. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0019] Figure 1 The flowchart of the image data desensitization method in the embodiment of the present application;

[0020] Figure 2 The flowchart of determining the face detection model for inference in the embodiment of the present application;

[0021] Figure 3 The flowchart of determining the processor of the current computing device for inference in the embodiment of the present application;

[0022] Figure 4 The flowchart of desensitizing each face region in the embodiment of the present application;

[0023] Figure 5 FIG. 1 is a schematic diagram of an image data desensitization device according to an embodiment of the present application;

[0024] Figure 6 FIG. 2 is a schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, further detailed descriptions will be given below with reference to the drawings. Here, the illustrative embodiments of the present application and their descriptions are used to explain the present application but are not intended to limit the present application.

[0026] In the technical solutions of the present application, the acquisition, storage, use, processing, etc. of data all comply with relevant provisions of national laws and regulations.

[0027] It should be noted that in the embodiments of the present application, some industry existing solutions, components, models, etc. may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solutions of the present application, but it does not mean that the applicant has or will necessarily use the solutions.

[0028] In the embodiments of the present application, when performing automatic desensitization of image data, the sensitive area (such as a face) in the image is automatically detected through an advanced computer vision algorithm, and is processed through a mosaic or other covering technology, so as to protect privacy while retaining data usability, to realize face detection and desensitization in an efficient and flexible manner, to meet the needs of diversified application scenarios, and to have a wide application prospect in the fields of smart home, security monitoring, medical image analysis, etc.

[0029] Figure 1 FIG. 3 is a flowchart of an image data desensitization method according to an embodiment of the present application, and the method comprises the following steps:

[0030] Step 101, determining a face detection model for inference according to application scenario requirements, a device performance feature library, and computer device data of a computer device, the device performance feature library being used to record inference effect data of different types of face detection models for inference by different computer devices under different computer device hardware parameters;

[0031] Step 102, monitoring computer device hardware parameters of the computer device in real time to determine a processor of the computer device for current inference;

[0032] Step 103, performing an inference task on an input face image based on the determined face detection model on the processor for current inference to obtain at least one face region, wherein the inference task is migrated according to an update of the processor for current inference;

[0033] Step 104, when it is determined that the face region needs to be desensitized, the modified mosaic intensity is determined according to the application scene adjustment factor and the desensitization risk index corresponding to each face region, and each face region is desensitized according to the modified mosaic intensity, and the face region after desensitization is written back to the face image.

[0034] In the embodiment of the application, the input face image supports various formats such as JPEG format. The face region is represented by a real box, and the format: [x1, y1, x2, y2] represents the coordinates of the real box. When the face region is output, the confidence and the IoU threshold can be specified, and there is at least one face region, that is, one face image can identify multiple face regions. The input face image can be preprocessed, including size checking and format conversion, and the resolution (maximum side length 2000 pixels) is dynamically adjusted for large size images, to optimize the inference speed.

[0035] In the embodiment of the application, the device performance feature library is generated by performing benchmark tests on different computing devices (such as different models of CPUs and GPUs) in advance, and the inference effect data includes time consumption, power consumption and energy efficiency ratio data, which provides a decision basis for subsequent intelligent scheduling.

[0036] In an embodiment, the application scene requirements include real-time requirements, accuracy requirements, face key point error requirements, multi-task support requirements and extreme robustness requirements.

[0037] The computing device data includes computing device type and computing device hardware parameters.

[0038] The real-time requirement values can be divided as follows according to the application scene:

[0039] High (FPS≥30): autonomous driving, video conference, live broadcast barrage;

[0040] Medium (FPS≥15): security monitoring, access control system, AR filter;

[0041] Low (offline processing): face library retrieval, data analysis, medical image;

[0042] The accuracy requirement values can be divided as follows:

[0043] Average accuracy mean: ≥85% for general scenes such as security scenes, and ≥95% for high-precision scenes such as payment verification;

[0044] The face key point error requirement values can be divided as follows: <5% for key point detection scenes (such as expression analysis scenes);

[0045] The multi-task support requirement can be divided as follows:

[0046] If there are at least two or more tasks (face detection task, expression detection task, posture detection task, age detection task, etc.), it is determined that there is a multi-task support requirement, otherwise it is determined that there is no multi-task support requirement;

[0047] The value of the extreme scene robustness can be divided as follows:

[0048] Large angle (±60°), occlusion (>30%), low light (<10 lux).

[0049] The computing device type can be divided into high-performance servers (such as GPU clusters, TPUs), edge servers, embedded devices, and mobile terminals (such as mobile phones, etc.).

[0050] The computing device hardware parameters can include CPU parameters (such as CPU utilization, memory occupancy, real-time temperature, etc.), GPU parameters (GPU utilization, video memory occupancy, real-time temperature), and NPU (such as dedicated AI computing power, etc.).

[0051] Figure 2 The flowchart for determining the face detection model for inference in the embodiments of the present application, in an embodiment, according to the application scenario requirement, the device performance feature library and the computing device data of the computing device, the face detection model for inference is determined, comprising:

[0052] Step 201, according to the computing device type and the real-time requirement, combining the decision tree to preliminarily screen the face detection model;

[0053] If the computing device type is a high-performance server:

[0054] High real-time requirement: preliminarily screen a single-stage model.

[0055] Medium / low real-time requirement: preliminarily screen a double-stage model or a high-precision single-stage model.

[0056] If the computing device type is an edge server:

[0057] High real-time requirement: preliminarily screen a lightweight single-stage model.

[0058] Medium / low real-time requirement: preliminarily screen a medium single-stage model.

[0059] If the computing device type is an embedded device:

[0060] Combined with the computing capability in the hardware parameters, the computing capability is greater than or equal to 5TOPS and the real-time requirement is high: preliminarily screen a lightweight single-stage model.

[0061] The computing capability is less than 5TOPS or the real-time requirement is low: preliminarily screen an ultralight model.

[0062] If the type of computing device is a mobile terminal:

[0063] If there is a dedicated NPU and real-time requirements: preliminary screening of lightweight models optimized by NPU.

[0064] If there is no dedicated NPU or the real-time requirement is low: preliminary screening of minimalist models.

[0065] Step 202, according to the accuracy requirement, the set of face detection models screened in the preliminary screening is subjected to secondary screening;

[0066] If the accuracy requirement is extremely high (for example, accuracy requirement ≥ 95%):

[0067] High-performance server and high real-time: select high-precision models (such as RetinaFace + ResNet-50) from the single-stage models screened in the preliminary screening.

[0068] High-performance server and medium / low real-time: select high-precision models (such as Faster R-CNN + ResNet-101) from the two-stage models screened in the preliminary screening.

[0069] Other devices such as edge servers: select models with better accuracy and device performance from the preliminary screening models.

[0070] If the accuracy requirement is general (for example, accuracy requirement ≥ 85%):

[0071] High-performance server and high real-time: select large-bone single-stage models (such as YOLOv8x).

[0072] Edge server and high real-time: select models such as YOLOv8m + TensorRT optimization.

[0073] Other device types and real-time conditions: select models that meet the accuracy requirement from the preliminary screening models.

[0074] Step 203: According to the face key point error requirement, the set of face detection models screened in the secondary screening is subjected to tertiary screening;

[0075] If it is a key point detection scene (error needs to be <5%): in the set of face detection models screened in the secondary screening, select models with key point detection branches or models that can add key point detection heads, and refer to the device performance feature library to ensure that the model can meet the error requirement when it is in Inference on the current device.

[0076] If there is no strict key point error requirement: keep the set of face detection models screened in the secondary screening.

[0077] Step 204: According to the multi-task support requirement, the third screened face detection model set is screened four times; if there is a multi-task support requirement: select a face detection model supporting at least two and more tasks from the existing face detection model set, and for a high-performance server, a multi-task version model (such as YOLOv8x multi-task version) can be selected; for an edge server, the computing device hardware parameters are combined, and if the computing capability is greater than or equal to 20 TOPS, a face detection model supporting multi-task can be selected, otherwise only a face detection model supporting detection + key point task is reserved; according to the device performance feature library, an embedded device and a mobile terminal select a lightweight model that can efficiently run multi-task on the current device.

[0078] If there is no multi-task support requirement: select a model supporting only face detection single task from the existing face detection model set.

[0079] Step 205: Considering the extreme robustness requirement, the fourth screened face detection model set is screened five times.

[0080] If the extreme scene (large angle, occlusion, low light) needs to be coped with: the model with strong adaptability to the extreme scene (such as RetinaFace) is preferentially selected, and the device performance feature library is referred to to ensure that the time consumption, power consumption and the like of the model meet the requirements when the model is in inference on the current device, and the model is optimized (such as adding an attention mechanism) if necessary. If the extreme scene does not need to be coped with: the existing model range is maintained.

[0081] Step 206: The final face detection model is determined from the fifth screened face detection model set in combination with the computing device hardware parameters and the device performance feature library.

[0082] The computing device hardware parameters (CPU core number, frequency, GPU video memory, CUDA core number and the like) are viewed, and the time consumption, power consumption and energy efficiency ratio data of each candidate model on the device in the device performance feature library are combined.

[0083] The models that cannot be efficiently run due to hardware parameter limitations are excluded, such as models with a parameter quantity greater than 10 MB when the device memory is less than 2 GB, and models with a power consumption limit less than 10 W. The final selection is a face detection model with less time consumption, low power consumption and high energy efficiency ratio under the premise of meeting all requirements of the application scene.

[0084] Figure 3 A flowchart for determining the processor of the computing device for current inference in the embodiments of the application, in an embodiment, the computing device hardware parameters of the computing device are monitored in real time to determine the processor of the computing device for current inference, comprising:

[0085] Step 301, real-time monitoring of the computing device hardware parameters of the computing device;

[0086] When the system is initialized, the built-in monitoring agent is started to monitor the key hardware parameters of CPU and GPU in the computing device in real time, including:

[0087] CPU parameters: CPU utilization, memory occupancy, real-time temperature;

[0088] GPU parameters: GPU utilization, video memory occupancy, real-time temperature;

[0089] The monitoring frequency is set to once every 100ms to ensure timely capture of hardware state changes.

[0090] Step 302: In the initial state, determine the processor for inference as GPU;

[0091] Make full use of the parallel computing capability of GPU to improve inference efficiency.

[0092] Step 303: Based on the GPU parameters, determine whether to switch the processor to CPU;

[0093] When the GPU utilization is continuously (3 consecutive samples) higher than the high water line (such as > 80%), or the GPU video memory occupancy is close to the upper limit (such as > 90%) (at this time regardless of GPU utilization), the processor is switched to CPU;

[0094] Enable INT8 quantization acceleration for the task, compensate for the performance gap between CPU and GPU by reducing data precision, and ensure that the inference speed is basically stable.

[0095] Step 304, monitor the CPU parameters during CPU execution to determine whether to trigger the CPU delay preemption strategy;

[0096] After the inference task is switched to CPU, the monitoring agent synchronously records the time consumption data of each frame of inference task, the scheduling algorithm calculates the average inference time consumption of multiple frames (such as 3 frames), and compares it with the preset threshold (such as 200ms):

[0097] If the average time consumption is less than or equal to the preset threshold, it indicates that the CPU can stably process the current inference task, and the CPU execution mode is maintained

[0098] If the average time consumption is greater than the preset threshold, it indicates that the CPU processing capacity is insufficient and cannot meet the real-time requirement, triggering the CPU delay preemption strategy;

[0099] Step 305, if the CPU delay preemption strategy is triggered, determine to switch the processor back to GPU;

[0100] When the CPU delay preemption strategy is triggered, the scheduler performs the following operations:

[0101] Immediately interrupt the current inference task of the CPU by the inference module (save the intermediate state to avoid data loss);

[0102] Trigger GPU priority preemption to switch the inference task back to GPU execution;

[0103] If the GPU is still in a high load state at this time, the scheduler temporarily enables the task queue mechanism, caches new tasks to the queue, and executes them in order when the GPU is idle, while reducing the GPU resource occupation of single batch tasks by dynamically adjusting the batch size (such as reducing the batch size from 8 to 4).

[0104] Step 306: Real-time monitor the real-time temperature of the processor, and adjust the processor according to the real-time temperature.

[0105] During the execution of the inference task, the monitoring agent continuously tracks the real-time temperature of the CPU and GPU:

[0106] If the real-time temperature of the GPU continues to be higher than the safety threshold (such as 85℃), even if the load switching condition is not reached, part of the low-priority inference tasks (such as non-real-time log analysis and correlation inference) are migrated to the CPU to reduce the GPU heat dissipation pressure;

[0107] If the real-time temperature of the CPU continues to be higher than the safety threshold (such as 75℃), the amount of inference tasks allocated to the CPU is reduced, and part of the non-core inference tasks are suspended if necessary to avoid performance degradation caused by hardware overheating.

[0108] In the embodiment of the present application, the system is built-in with an adaptive learning module, which dynamically adjusts various threshold parameters according to historical task execution data and hardware state changes: if the GPU frequently fluctuates between high load and low load (switching ≥5 times within 1 minute) in a certain scenario, the high water line is automatically raised from 80% to 85%, reducing unnecessary task switching; if the CPU still frequently triggers delay preemption under INT8 quantization acceleration (≥3 times within 10 minutes), the CPU inference time threshold is automatically raised from 200ms to 250ms, balancing delay and processor stability.

[0109] In the embodiment of the present application, after each processor switching (GPU→CPU or CPU→GPU), the system automatically records the switching time, the triggering reason (such as GPU load being too high, CPU time being out of limit), the inference performance data (FPS, delay) before and after switching to form a scheduling log.

[0110] The scheduling algorithm analyzes the log data periodically (e.g., every hour) to optimize the task allocation strategy, for example: for the scene of frequently triggering GPU high load, 20% of GPU resources are reserved in advance for emergency switching; for the scene of insufficient CPU performance, the high-frequency core is preferentially bound to the CPU to improve single-thread processing capability.

[0111] In an embodiment, the method further comprises:

[0112] After obtaining at least one face region, non-maximum suppression is applied, and overlapping face regions are merged based on an IoU threshold of the face regions.

[0113] Low-confidence face regions and face regions exceeding the boundary are filtered to generate final face regions.

[0114] For example, low-confidence face regions (e.g., confidence < 0.5) are filtered, and a face region can be represented as [x1, y1, x2, y2, confidence], where x1, y1, x2, and y2 are coordinate values of the face region, and confidence is the confidence.

[0115] To improve robustness, when performing boundary checking, it is ensured that the coordinate values of the face region do not exceed the range of the face image (x1, y1≥0, x2≤image width, y2≤image height).

[0116] In an embodiment, determining that the face region needs to be desensitized includes:

[0117] When the size of the face region is greater than a first preset size, it is determined that desensitization is needed.

[0118] In an embodiment, the method further comprises:

[0119] According to the scene weight of the application scenario and the area weight of each face region, an application scenario adjustment factor corresponding to each face region is determined.

[0120] The application scenario adjustment factor = scene weight × 0.3 + area weight × 0.7.

[0121] Scenarios can be divided into public scenarios (such as streets and shopping malls), private scenarios (such as bedrooms and offices), and sensitive scenarios (such as financial counters and government windows), and the corresponding scene weights increase from small to large. The specific scene weights are determined according to actual conditions, which are not limited here; the area weight can be divided according to actual conditions, for example, large-size faces (the size of the face region is greater than a second size, e.g., area > 5000 pixels²) and small-size faces (the size of the face region is less than a third size, area < 1500 pixels²) correspond to different area weights, and the area weight of large-size faces (area > 5000 pixels²) is greater.

[0122] In an embodiment, the desensitization risk index of each face region is calculated by the following formula: PRI=a×qxd+b×gz+c×zt, wherein qxd is the face sharpness, gz is the light stability, and zt is the pose angle, and a, b and c are the weights corresponding to the face sharpness, the light stability and the pose angle respectively.

[0123] Figure 4 For the flowchart of the desensitization processing of each face region in the embodiment of the application, in an embodiment, the desensitization processing of each face region is performed according to the application scene adjustment factor and the desensitization risk index corresponding to each face region, including:

[0124] Step 401, correcting the base mosaic intensity corresponding to the size range of each face region according to the application scene adjustment factor and the first promotion ratio corresponding to the risk threshold range where the desensitization risk index is located, to obtain the corrected mosaic intensity of each face region;

[0125] The correction formula includes: corrected mosaic intensity=base mosaic intensity corresponding to the size range of the face region×(1+first promotion ratio)×(1+application scene adjustment factor).

[0126] In the embodiment of the application, the size range of the face region can be divided according to actual conditions, for example, into large-size faces (area>5000 pixels²) and small-size faces (area<1500 pixels²), and the corresponding base mosaic intensities are 10×10 pixel grid and 50×50 pixel grid respectively.

[0127] For example, for the desensitization risk index in the risk threshold range greater than the first risk threshold 0.7, the corresponding promotion ratio is 0.2, and then the corrected mosaic intensity=base mosaic intensity corresponding to the size range of the face region×(1+0.2)×(1+application scene adjustment factor).

[0128] The key regions include high-sensitivity regions such as eyes, nose and mouth, for example, the highest intensity mosaic (1.3 times the corrected mosaic intensity) is applied to the key point region, 0.7 times the corrected mosaic intensity is applied to the non-key region (cheek, forehead), and 0 times the corrected mosaic intensity is applied to the exempt region (for example, the mask covering part), that is, no mosaic processing is performed.

[0129] Step 402, performing mosaic processing on each face region according to the corrected mosaic intensity of the face region;

[0130] In an embodiment, before performing the mosaic processing on each face region according to the corrected mosaic intensity of the face region, it further includes:

[0131] For each face region, when the size of the face region is greater than the second size, divide the face region into a key region, a non-key region and an exempt region;

[0132] Adjust the modified mosaic intensity of the key region, the non-key region and the exempt region;

[0133] If the change amount of the modified mosaic intensity of the adjacent frame of the adjusted face region exceeds the change amount range, perform smooth transition on the modified mosaic intensity of the adjacent frame by an interpolation algorithm;

[0134] Perform mosaic processing on each face region according to the current modified mosaic intensity of the face region.

[0135] In an embodiment, the method further comprises:

[0136] Obtaining a real box of a pre-labeled face region of a face image;

[0137] According to the real box and the detection box of the face region obtained according to the inference task, calculating a performance index of face image detection; the performance index includes:

[0138] True positive (TP): the IoU of the detection box and the real box is greater than or equal to 0.45, and there is no repeated matching.

[0139] False positive (FP): the detection box that is not matched to the real box.

[0140] False negative (FN): the real box that is not detected.

[0141] Using a greedy matching algorithm, matching the detection box of the face region and the real box in descending order of confidence, and optimizing the TP count.

[0142] Recording the confidence distribution and the TP / FP label for subsequent drawing.

[0143] Calculating the overall precision, recall rate and F1 score.

[0144] Saving the face image after desensitization to the prediction directory, and the file name is consistent with the original face image.

[0145] Generating a contrast visualization image, including:

[0146] Real box (green, the matched one is a solid line, and the unmatched one is a yellow dashed line).

[0147] Detection box (red, TP is a solid line, and FP is a blue dashed line), labeled with confidence.

[0148] Generating a detailed visualization for the first face image, superimposing the mosaic effect, the real box and the detection box.

[0149] Finally, three types of analysis charts can be generated:

[0150] PR curve: based on the confidence of all bounding boxes and TP / FP labels, draw the PR curve, mark the average precision (AP) and operating point (confidence 0.5).

[0151] Confidence histogram: draw the confidence histogram of TP and FP, and mark the confidence threshold.

[0152] Normalized center coordinate chart of real box: analyze the normalized center coordinates, width and height of the real box, and display the distribution in the form of scatter plot and histogram.

[0153] All charts are generated using existing tools and can be saved in a preset format (such as PNG format) for easy analysis and reporting.

[0154] The embodiment of the application also provides an image data desensitization device, which has similar principles to the image data desensitization method, which will not be repeated here.

[0155] Figure 5 The schematic diagram of the image data desensitization device in the embodiment of the application comprises:

[0156] The face detection model selection module 501 is configured to determine a face detection model for inference according to application scenario requirements, a device performance feature library, and computing device data of a computing device, wherein the device performance feature library is configured to record inference effect data of different types of face detection models for inference by different computing devices under different computing device hardware parameters.

[0157] The processor selection module 502 is configured to monitor computing device hardware parameters of the computing device in real time and determine a processor of the computing device for current inference.

[0158] The face detection module 503 is configured to perform an inference task on an input face image based on the determined face detection model on the processor for current inference, and obtain at least one face region, wherein the inference task is migrated according to an update of the processor for current inference.

[0159] The desensitization processing module 504 is configured to, when it is determined that the face region needs desensitization processing, determine a corrected mosaic intensity according to an application scenario adjustment factor and a desensitization risk index corresponding to each face region, perform desensitization processing on each face region according to the corrected mosaic intensity, and write the face region after desensitization processing back to the face image.

[0160] In an embodiment, the face detection model selection module is configured to:

[0161] According to the type of the computing device and the real-time requirement, the face detection model is preliminarily screened in combination with a decision tree.

[0162] According to the accuracy requirement, the primary screening face detection model set is secondarily screened;

[0163] According to the face key point error requirement, the secondarily screened face detection model set is thirdly screened;

[0164] According to the multi-task support requirement, the thirdly screened face detection model set is fourthly screened;

[0165] Considering the extreme robustness requirement, the fourthly screened face detection model set is fifthly screened;

[0166] The final face detection model is determined from the fifthly screened face detection model set in combination with the computing device hardware parameters and the device performance feature library.

[0167] In an embodiment, the processor selection module is configured to:

[0168] Monitor the computing device hardware parameters of the computing device in real time;

[0169] In the initial state, determine that the inference processor is a GPU;

[0170] Based on the GPU parameters, determine whether to switch the processor to a CPU;

[0171] During the CPU execution, monitor the CPU parameters and determine whether to trigger a CPU delay preemption strategy;

[0172] If the CPU delay preemption strategy is triggered, determine to switch the processor back to the GPU;

[0173] Monitor the real-time temperature of the processor in real time, and adjust the processor according to the real-time temperature.

[0174] In an embodiment, the face detection module is further configured to:

[0175] After obtaining at least one face region, apply non-maximum suppression to merge overlapping face regions based on an IoU threshold of the face regions;

[0176] Filter low-confidence face regions and face regions beyond the boundary to generate final face regions.

[0177] In an embodiment, the desensitization processing module is configured to:

[0178] When the size of the face region is greater than a first preset size, determine that desensitization processing is needed.

[0179] In an embodiment, the desensitization processing module is configured to:

[0180] According to the scene weight of the application scene and the area weight of each face region, a corresponding application scene adjustment factor of each face region is determined.

[0181] In an embodiment, the desensitization processing module is configured to:

[0182] The desensitization risk index of each face region is calculated by using the following formula:

[0183] PRI=a× qxd + b× gz + c× zt, wherein qxd is the face sharpness, gz is the light stability, zt is the pose angle, and a, b and c are weights corresponding to the face sharpness, the light stability and the pose angle, respectively.

[0184] In an embodiment, the desensitization processing module is configured to:

[0185] According to the application scene adjustment factor and the promotion ratio corresponding to the risk threshold range in which the desensitization risk index is located, the base mosaic intensity corresponding to the size range of each face region is corrected to obtain a corrected mosaic intensity of each face region.

[0186] The face region is processed by using the corrected mosaic intensity of the face region.

[0187] In an embodiment, the desensitization processing module is configured to:

[0188] Before the face region is processed by using the corrected mosaic intensity of the face region, for each face region, when the size of the face region is greater than the second size, the face region is divided into a key region, a non-key region and an exempt region.

[0189] The corrected mosaic intensities of the key region, the non-key region and the exempt region are adjusted.

[0190] If the change amount of the corrected mosaic intensity of the adjacent frame of the adjusted face region exceeds the change amount range, the corrected mosaic intensity of the adjacent frame is smoothly transitioned by using an interpolation algorithm.

[0191] The face region is processed by using the current corrected mosaic intensity of the face region.

[0192] In summary, in the method and device provided in the embodiment of the present application, a face detection model for inference is determined according to the application scenario requirement, the device performance feature library and the computing device data of the computing device, the device performance feature library is used to record the inference effect data of different types of face detection models for inference under different computing device hardware parameters of different computing devices; the computing device hardware parameters of the computing device are monitored in real time, and the processor of the current inference is determined; the inference task of the input face image is performed on the processor determined for current inference based on the determined face detection model, and at least one face region is obtained, wherein the inference task is migrated according to the update of the processor for current inference; when it is determined that the face region needs to be desensitized, the correction mosaic intensity is determined according to the application scenario adjustment factor and the desensitization risk index corresponding to each face region, and each face region is desensitized according to the correction mosaic intensity, and the face region after desensitization is written back to the face image. Through the decision tree combined with the application scenario, the device performance feature library and the real-time hardware parameter selection model, the model is accurately matched with the hardware capability of the computing device, the resource waste is reduced while the inference effect is ensured, and the inference speed and stability are improved. The processor is monitored in real time and the inference task is migrated, so that the inference process is not interrupted when the processor dynamically changes (such as switching from CPU to GPU), the system robustness is improved, and the device load fluctuation and the like are adapted. The correction mosaic intensity is determined by combining the application scenario adjustment factor and the desensitization risk index, so that the desensitization processing can meet the privacy protection requirement of different scenes (such as enhancing desensitization in a high-risk scene), and the necessary image information can be preserved, and the privacy and usability are balanced.

[0193] The embodiment of the present application further provides a computer device, Figure 6 For the schematic diagram of the computer device in the embodiment of the present application, the computer device 600 comprises a memory 610, a processor 620 and a computer program 630 stored in the memory 610 and capable of running on the processor 620, and the processor 620 implements the image data desensitization method described above when executing the computer program 630.

[0194] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image data desensitization method described above.

[0195] The embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the image data desensitization method described above.

[0196] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0197] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0198] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0199] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0200] The specific embodiments described above are illustrative for purposes of the present application. The particular implementations are not intended to limit the scope of the present application, which is defined by the appended claims. Numerous variations, changes, and substitutions can be made without departing from the application. It is the intention of the present application to embrace all such variations, changes, and substitutions that fall within the scope of the present application.

Claims

1. An image data de-sensitization method, characterized by, Comprise: According to the application scene demand, the device performance characteristic library and the computing device data of the computing device, determine the face detection model for inference, the device performance characteristic library is used for recording the inference effect data of different computing devices on different types of face detection model under different computing device hardware parameters; Real-time monitoring of the computing device hardware parameters of the computing device, determine the processor of the current inference computing device; On the current inference processor, based on the determined face detection model, the inference task is executed on the input face image, and at least one face region is obtained, wherein the inference task is migrated according to the update of the current inference processor; When it is determined that the face region needs to be desensitized, according to the scene weight of the application scene and the area weight of each face region, the application scene adjustment factor corresponding to each face region is determined; according to the promotion ratio corresponding to the risk threshold range of the desensitization risk index and the application scene adjustment factor, the basic mosaic intensity corresponding to the size range of each face region is corrected, and the corrected mosaic intensity of each face region is obtained; according to the corrected mosaic intensity of each face region, the face region is processed; the face region after desensitization is written back to the face image; The corrected mosaic intensity = the basic mosaic intensity corresponding to the size range of the face region × (1 + the promotion ratio) × (1 + the application scene adjustment factor).

2. The method of claim 1, wherein, The application scene demand includes one or any combination of real-time demand, precision demand, face key point error demand, multi-task support demand and extreme robustness demand; The computing device data includes computing device type and / or computing device hardware parameters.

3. The method of claim 2, wherein, According to the application scene demand, the device performance characteristic library and the computing device data of the computing device, determine the face detection model for inference, comprising: According to the computing device type and real-time demand, combine decision tree to preliminarily screen face detection model; According to the precision demand, the face detection model set after preliminary screening is screened twice; According to the face key point error demand, the face detection model set after secondary screening is screened three times; According to the multi-task support demand, the face detection model set after three times screening is screened four times; Considering the extreme robustness demand, the face detection model set after four times screening is screened five times; Combine the computing device hardware parameters and the device performance characteristic library, determine the final face detection model from the face detection model set after five times screening.

4. The method of claim 2, wherein, Real-time monitoring of the computing device hardware parameters of the computing device, determine the processor of the current inference computing device, comprising: Real-time monitoring of the computing device hardware parameters of the computing device; In the initial state, the processor for inference is determined as GPU; Based on GPU parameters, determine whether to switch the processor to CPU; During CPU execution, monitor the CPU parameters, and determine whether to trigger the CPU delay preemption strategy; If the CPU delay preemption strategy is triggered, determine that the processor is switched back to GPU; Real-time monitoring of the real-time temperature of the processor, and adjusting the processor according to the real-time temperature.

5. The method of claim 1, wherein, Also include: After obtaining at least one face region, non-maximum suppression is applied, and overlapping face regions are merged based on an IoU threshold of the face regions; Low-confidence face regions and out-of-bound face regions are filtered to generate final face regions.

6. The method of claim 1, wherein, Determining that a face region needs to be desensitized includes: When the size of the face region is greater than a first preset size, it is determined that desensitization is needed.

7. The method of claim 1, wherein, Also includes: The desensitization risk index of each face region is calculated using the following formula: PRI=a× qxd + b× gz + c× zt, wherein qxd is face sharpness, gz is light stability, zt is pose angle, a, b and c are weights corresponding to face sharpness, light stability and pose angle.

8. The method of claim 1, wherein, Before performing mosaic processing on each face region according to the modified mosaic intensity of the face region, it further includes: For each face region, when the size of the face region is greater than a second size, the face region is divided into a key region, a non-key region and an exempt region; Adjust the modified mosaic intensity of the key region, the non-key region and the exempt region; If the change in the modified mosaic intensity of the adjacent frame of the adjusted face region exceeds the change range, the modified mosaic intensity of the adjacent frame is smoothly transitioned through an interpolation algorithm; Performing mosaic processing on each face region according to the current modified mosaic intensity of the face region.

9. An image data de-sensitizing apparatus characterized by comprising: It includes: A face detection model selection module is configured to determine a face detection model for inference based on application scenario requirements, a device performance feature library, and computing device data of a computing device, the device performance feature library being configured to record inference effect data of different types of face detection models inferred by different computing devices under different computing device hardware parameters; A processor selection module is configured to monitor computing device hardware parameters of the computing device in real time and determine a processor of the computing device for current inference; A face detection module is configured to perform an inference task on an input face image based on the determined face detection model on the processor for current inference, and obtain at least one face region, wherein the inference task is migrated according to updates of the processor for current inference; A desensitization processing module is configured to, when it is determined that a face region needs to be desensitized, determine an application scenario adjustment factor corresponding to each face region based on a scene weight of an application scenario and an area weight of each face region, correct a basic mosaic intensity corresponding to a size range of each face region based on a promotion ratio corresponding to a risk threshold range of the desensitization risk index and the application scenario adjustment factor, obtain a modified mosaic intensity of each face region according to the modified mosaic intensity of each face region, and perform mosaic processing on the face region; and write the face region after desensitization back to the face image. The modified mosaic intensity=the basic mosaic intensity corresponding to the size range of the face region×(1+the promotion ratio)×(1+the application scenario adjustment factor).

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 8. The processor executes the computer program to implement the method of any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1 to 8.

12. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1 to 8.

Images