Privacy protection method, device and computer equipment based on multi-algorithm resource scheduling

Abstract

The present application relates to a privacy protection method, apparatus and computer equipment based on multi-algorithm resource scheduling, wherein the method obtains a target quantization result by quantifying the remaining performance of the back-end server and the front-end image acquisition device; determines the target algorithm resource based on the credibility of the algorithm resource matching the target quantization result; the credibility of the algorithm resource is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server and the front-end image acquisition device; performs privacy protection processing on the collected video data based on the detection result of the target algorithm resource, solves the problem of high algorithm startup consumption and limited privacy protection specifications, realizes full utilization of the remaining performance of the front-end and back-end devices, flexibly schedules computing resources, improves the supported privacy protection specifications, and improves the privacy protection effect.

Description

Technical Field

[0001] The present application relates to the technical field of surveillance video privacy protection data processing, and in particular to a privacy protection method, device and computer equipment based on multi-algorithm resource scheduling. Background Art

[0002] In recent years, as privacy protection regulations have become increasingly stringent, the implementation of privacy protection has gradually shifted from manually superimposing designated areas to using intelligent image analysis such as face recognition and license plate recognition to be more precise and automated. For back-end storage / servers (hereinafter referred to as NVRs) that are connected to a large number of front-end cameras (hereinafter referred to as IPCs), the protection of privacy information is particularly critical during real-time preview / recording playback. If each frame of the picture is directly analyzed through the NVR for privacy protection, then the NVR does not have the analysis performance after a large number of IPCs are connected. So for typical privacy solutions for security, how to achieve effective privacy protection and shielding capabilities is a pain point at this stage.

[0003] In existing privacy protection technologies, the host usually schedules multiple resources to collaboratively complete privacy protection, such as Figure 1 As shown in the figure, the host receives the video code stream data of the corresponding video channel through the network, pre-processes the code stream, distributes it to the decoding card through thread driver to complete video decoding, and then sends it to the intelligent analysis card to perform analysis tasks; after the analysis is completed, the results are returned to the host, and the host processes the results. The privacy overlay processing performance of the privacy protection host is determined by the computing performance and decoding capability of the computing power card and the CPU overlay capability. An intelligent privacy protection host requires that it can analyze effective targets (diverse objects) to the greatest extent through different algorithms for privacy overlay protection. The computing power card can generally process multiple video analysis tasks at the same time, but the resources consumed are different depending on the type of video analysis task. The maximum number of channels that a single card can process will also vary depending on the task type. The decoding card capacity is fixed, but the total number of decoding channels will also vary with the size and frame number, and the same is true for the CPU overlay scaling capability.

[0004] In summary, it is difficult for existing technologies to fully utilize host resources. There are problems such as high algorithm activation consumption and limited specifications for privacy protection paths, resulting in unsatisfactory privacy protection effects. Summary of the invention

[0005] In this embodiment, a privacy protection method, apparatus and computer device based on multi-algorithm resource scheduling are provided to solve the problems in related technologies of high algorithm activation consumption and limited number of privacy protection paths.

[0006] In a first aspect, a privacy protection method based on multi-algorithm resource scheduling is provided in this embodiment, and the method is applied to a video surveillance system, wherein the video surveillance system includes a front-end image acquisition device and a back-end server; the method is applicable to the back-end server; the method includes:

[0007] Based on the received privacy protection task request, quantify the remaining performance of the back-end server and the front-end image acquisition device to obtain a target quantification result;

[0008] Determine the target algorithm resource based on the credibility of the algorithm resource matching the target quantization result; the credibility of the algorithm resource is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server and the front-end image acquisition device;

[0009] The collected video data is subjected to privacy protection processing based on the detection result of the target algorithm resource.

[0010] In some of the embodiments, based on the received privacy protection task request, quantifying the remaining performance of the backend server and the frontend image acquisition device to obtain a target quantification result includes:

[0011] Based on the received privacy protection task request, quantify the remaining performance of the backend server to obtain a backend quantification result; the backend quantification result includes any one of sufficient resources, ordinary resources and tight resources;

[0012] When the backend quantization result indicates that the resources are sufficient, a first target quantization result is obtained;

[0013] When the back-end quantization result indicates that the resources are normal or the resources are tight, the front-end quantization index of the front-end image acquisition device is obtained, and based on the front-end quantization index, the remaining performance of the front-end image acquisition device is quantified to obtain the front-end quantization result, and the second target quantization result is obtained based on the front-end quantization result and the back-end quantization result.

[0014] In some embodiments, based on the received privacy protection task request, the remaining performance of the backend server is quantified to obtain a backend quantification result; the backend quantification result includes any one of sufficient resources, ordinary resources and tight resources, including:

[0015] Based on the received privacy protection task request, obtaining backend quantitative indicators of the backend server;

[0016] Performing weighted sum calculation on each of the quantitative indicators based on a preset load weight ratio to obtain a backend quantitative result;

[0017] Based on a preset quantification degree standard, the backend quantification result is determined as any one of sufficient resources, ordinary resources, and scarce resources.

[0018] In some of the embodiments, the backend quantitative indicators include remaining neural network computing power, remaining memory, and remaining CPU resources;

[0019] Wherein, when one type of algorithm task is started, the remaining neural network computing power is calculated based on the number of the algorithm tasks and the single-type environment computing power consumption of the algorithm tasks;

[0020] When multiple types of algorithm tasks are enabled, the remaining neural network computing power is calculated based on the number of the algorithm tasks and the mixed environment conservative consumption value of the currently enabled algorithm tasks;

[0021] The remaining memory and the remaining CPU resources are obtained based on a query command in the operating system.

[0022] In some of the embodiments, determining a target algorithm resource based on the credibility of the algorithm resource matching the target quantization result includes:

[0023] When the first target quantification result is obtained, the algorithm resource with the highest credibility is used as the target algorithm resource;

[0024] When the second target quantization result is obtained, a target algorithm resource is selected from the algorithm resources that are not of the highest credibility.

[0025] In some embodiments, when the second target quantization result is obtained, selecting a target algorithm resource from algorithm resources that are not of the highest credibility includes:

[0026] When the back-end quantization result indicates that the resources are ordinary or the resources are tight, and the front-end quantization result indicates that the front-end resources are sufficient, the high-precision algorithm resources deployed on the front-end image acquisition device are used as the target algorithm resources;

[0027] When the back-end quantization result is that the resources are ordinary, and the front-end quantization result is that the non-front-end resources are sufficient, the medium-precision algorithm resources deployed on the front-end image acquisition device are used as the target algorithm resources;

[0028] When the back-end quantization result is that resources are tight and the front-end quantization result is that the non-front-end resources are sufficient, the medium-precision algorithm resources deployed on the front-end image acquisition device, the low-precision algorithm resources deployed on the front-end image acquisition device, the medium-precision algorithm resources deployed on the back-end server, and the low-precision algorithm resources deployed on the back-end server will be used as the target algorithm resources.

[0029] In some of the embodiments, privacy protection processing is performed on the collected video data based on the detection result of the target algorithm resource, including:

[0030] When the back-end quantification result is that resources are tight, and the front-end quantification result is that the non-front-end resources are sufficient, the video data collected by the front-end image acquisition device is obtained, and the video data is detected based on the target algorithm resources deployed on the back-end server to obtain the back-end detection result; and the front-end detection result is obtained after the target algorithm resources deployed on the front-end image acquisition device detect the video data;

[0031] In the front-end detection results and the back-end detection results, based on the detection result with the largest number of detected targets, the privacy target to be shielded is obtained;

[0032] Based on the intersection of the target areas in each of the front-end detection results and the back-end detection results, the superposition range of the privacy target to be shielded is obtained.

[0033] In a second aspect, a privacy protection device based on multi-algorithm resource scheduling is provided in this embodiment, and the device is applied to a video surveillance system, and the video surveillance system includes a front-end image acquisition device and a back-end server; the device is applicable to the back-end server; the device includes:

[0034] A residual performance quantification module, used to quantify the residual performance of the back-end server and the front-end image acquisition device based on the received privacy protection task request to obtain a target quantification result;

[0035] A matching algorithm resource module, used to determine a target algorithm resource based on the credibility of the algorithm resource matching the target quantization result; the credibility of the algorithm resource is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server and the front-end image acquisition device;

[0036] The privacy protection processing module is used to perform privacy protection processing on the collected video data based on the detection result of the target algorithm resource.

[0037] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the privacy protection method based on multi-algorithm resource scheduling described in the first aspect is implemented.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the privacy protection method based on multi-algorithm resource scheduling described in the first aspect is implemented.

[0039] Compared with the related art, the privacy protection method, apparatus and computer equipment based on multi-algorithm resource scheduling provided in this embodiment quantify the remaining performance of the back-end server and the front-end image acquisition device based on the received privacy protection task request to obtain the target quantification result; determine the target algorithm resource based on the algorithm resource credibility matching the target quantification result; the algorithm resource credibility is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server and the front-end image acquisition device; and perform privacy protection processing on the collected video data based on the detection result of the target algorithm resource, which solves the problem of high algorithm startup consumption and limited privacy protection specifications, realizes full utilization of the remaining performance of the front-end and back-end devices, flexibly schedules computing resources, improves the supported privacy protection specifications, and improves the privacy protection effect.

[0040] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 It is a structural diagram of a privacy protection processing system in the prior art;

[0043] Figure 2 A hardware structure block diagram of a terminal of a privacy protection method based on multi-algorithm resource scheduling in one embodiment;

[0044] Figure 3 A schematic diagram of the structure of a video surveillance system in one embodiment;

[0045] Figure 4 is a flowchart of a privacy protection method based on multi-algorithm resource scheduling in one embodiment;

[0046] Figure 5 A schematic diagram of a flow chart of calculating the residual performance in one embodiment;

[0047] Figure 6 It is a flowchart of a privacy protection method based on multi-algorithm resource scheduling in a preferred embodiment;

[0048] Figure 7 It is a flowchart of the first resource scheduling strategy in a preferred embodiment;

[0049] Figure 8It is a flow chart of the second resource scheduling strategy in a preferred embodiment;

[0050] Fig. 9 It is a flowchart of the third resource scheduling strategy in a preferred embodiment;

[0051] Fig.10 It is a structural block diagram of a privacy protection device based on multi-algorithm resource scheduling in one embodiment.

[0052] Figure numerals: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 31, front-end image acquisition device; 32, back-end server; 10, residual performance quantification module; 20, matching algorithm resource module; 30, privacy protection processing module. DETAILED DESCRIPTION

[0053] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0054] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0055] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 2is a hardware structure block diagram of a terminal of the privacy protection method based on multi-algorithm resource scheduling of this embodiment. Figure 2 As shown, the terminal may include one or more ( Figure 2 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 2 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 2 More or fewer components as shown, or with Figure 2 Different configurations shown.

[0056] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the privacy protection method based on multi-algorithm resource scheduling in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0057] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0058] In this embodiment, a video monitoring system is provided. Figure 3 As shown, the video monitoring system includes a front-end image acquisition device 31 and a back-end server 32.

[0059] Specifically, the front-end image acquisition device 31 is an Internet Protocol Camera (IPC), and the back-end server 32 is a network video recorder (NVR). The front-end image acquisition device 31 includes a video acquisition module, a video transmission module, an intelligent analysis module, and a video detection and analysis module. The back-end server 32 includes a video receiving module, a video transmission module, a backup storage module, an intelligent analysis module, a display module, a video detection and analysis module, an auxiliary data processing module, and a resource statistics and weight balancing module. It should be noted that the division method adopted by the above modules is the conventional division method of the software functions of security products. Among them, the video intelligent analysis module in the front-end image acquisition device 31 or the back-end server 32 focuses on the intelligent target analysis of the AI ​​deep learning model, and the video detection and analysis module is the detection and analysis of image changes such as motion detection, occlusion, and image quality. The most significant difference between the two is the difference in computing power, algorithm memory, and CPU resource consumption.

[0060] In this embodiment, a privacy protection method based on multi-algorithm resource scheduling is also provided, which is applied to Figure 3 The video monitoring system shown in the figure is applicable to the backend server 32. Figure 4 is a flowchart of the privacy protection method based on multi-algorithm resource scheduling of this embodiment, such as Figure 4 As shown, the process includes the following steps:

[0061] Step S110, based on the received privacy protection task request, quantify the remaining performance of the back-end server 32 and the front-end image acquisition device 31 to obtain a target quantification result.

[0062] Specifically, since the ability of the back-end server 32 to perform intelligent analysis (hereinafter referred to as post-intelligence) takes precedence over the ability of the front-end image acquisition device 31 to perform intelligent analysis (hereinafter referred to as front-end intelligence), the resource statistics and weight balancing module in the back-end server 32 first quantifies the remaining performance of the back-end server 32 according to the load weight, and decides whether to quantify the remaining performance of the front-end image acquisition device 31 according to the quantification result. When it is necessary to quantify the remaining performance of the front-end image acquisition device 31, the resource statistics and weight balancing module in the back-end server 32 obtains the current front-end quantitative index of the front-end image acquisition device 31, and performs secondary quantization based on the front-end quantitative index, wherein the front-end quantitative index is used to describe the current operating status of the device. The quantification method used in the front-end and back-end can be the same or different, and the specific quantization method is not limited in this embodiment.

[0063] Step S120, determining the target algorithm resource based on the credibility of the algorithm resource matching the target quantization result; the credibility of the algorithm resource is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server 32 and the front-end image acquisition device 31.

[0064] Specifically, the credibility of algorithm resources is pre-set, and the accuracy of different types of algorithms is different. For example, there is the following relationship: face / body detection>non-deep learning version of face detection>object detection>SMD detection>motion detection; the order of algorithm resource credibility can be set according to the order of algorithm accuracy. The above algorithm resources can be deployed on the front-end image acquisition device 31 and the back-end server 32 at the same time, that is, the video surveillance system has both front intelligence and back intelligence. Therefore, the algorithm resources of the front intelligence can be set to multiple credibility levels, such as high, medium, and low levels; the algorithm resources of the back intelligence can be set to multiple levels, such as high, medium, and low levels. According to different target quantification results, choose to use different levels of front intelligence or back intelligence for privacy target detection.

[0065] Because the actual privacy protection specifications and resource consumption are different in different operating environments during the privacy target determination process. For example, if the detection and recognition of privacy targets only runs the face algorithm, assuming that the memory and CPU performance are sufficient, only from the perspective of the neural network CNN computing power limitation, it can process up to 4 channels. If only the perimeter is run, it can process up to 16 channels. However, if the two algorithm services are mixed, in theory, it can support a combination mode of 3 faces + 4 perimeters. In reality, due to the different number of targets in the picture, this combination cannot be achieved, showing that the privacy protection effect is poor due to the insufficient number of detection targets and the number of channels that can be opened for privacy protection is insufficient.

[0066] In addition, from the perspective of the back-end server 32 resources, for this kind of intelligent privacy protection business scenario, each time a function is created and enabled, the more accurate the algorithm used, the greater the memory and decoding resource consumption. If the same algorithm is used in the front-end detection, the memory and consumption will be much smaller. The reason is that the detection algorithm is enabled on the back-end server 32, which requires the server's own decoding, CPU, computing power, and other necessary public resources. After the front-end detection algorithm is enabled, these server resources are no longer needed. The front-end image acquisition device 31 can directly provide the detection results for the back-end server 32 to classify, and the front-end device itself has rich auxiliary data, such as dynamic inspection data, SMD, IVS, face and other data frames generated by front-end intelligent events, which are not affected by whether the back-end server 32 is used or not, but are continuously transmitted and stored as auxiliary data after the connection is established. Therefore, in the privacy target detection scenario, a variety of auxiliary data sources should be used in combination to efficiently utilize resources.

[0067] Step S130: performing privacy protection processing on the collected video data based on the detection result of the target algorithm resource.

[0068] Specifically, privacy protection is superimposed on the detection result of the previous intelligence or the privacy protection target in the detection result of the previous intelligence, thereby shielding the privacy protection target.

[0069] The privacy protection method, apparatus and computer equipment based on multi-algorithm resource scheduling provided in this embodiment quantify the remaining performance of the back-end server 32 and the front-end image acquisition device 31 based on the received privacy protection task request to obtain the target quantification result; determine the target algorithm resource based on the algorithm resource credibility matching the target quantification result; the algorithm resource credibility is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server 32 and the front-end image acquisition device 31; perform privacy protection processing on the collected video data based on the detection result of the target algorithm resource, solve the problem of high algorithm startup consumption and limited privacy protection path specifications, make full use of the remaining performance of the front-end and back-end devices, flexibly schedule computing resources, improve the supported privacy protection specifications, and improve the privacy protection effect.

[0070] In some of the embodiments, based on the received privacy protection task request, the remaining performance of the backend server 32 and the front-end image acquisition device 31 is quantified to obtain a target quantification result, including:

[0071] Step S210, based on the received privacy protection task request, quantify the remaining performance of the backend server 32 to obtain a backend quantification result; the backend quantification result includes any one of sufficient resources, normal resources and tight resources.

[0072] Specifically, Figure 5 As shown, the three quantitative indicators of the remaining neural network CNN computing power, remaining memory, and remaining CPU resources of the backend server 32 are queried. The remaining memory is recorded as M free , the remaining CNN computing power is recorded as N free , the remaining CPU is M free The current remaining resources of the backend server 32 are quantified according to the weight of the resource dependency load. The quantification standard is shown in Table 1 below.

[0073] Table 1:

[0074] ;

[0075] Where N represents the total computing power of the current device, M represents the total physical memory of the current device, and the resource quantization reference value of the current device is calculated according to the set coefficients in the table. The resource sufficiency reference value is recorded as P much , the general reference value of resources is recorded as P normal, the resource tension reference value is denoted as P less .

[0076] Preferably, the resource dependency load weight is set to N free :M free :C free =5:3:2, the calculation method of the reference value of resource quantification degree is as follows:

[0077] P much =50%×(0.9×N)+30%×(0.8×M)+20%×0.7;

[0078] P normal =50%×(0.6×N)+30%×(0.5×M)+20%×0.4;

[0079] P less =50%×(0.3×N)+30%×(0.2×M)+20%×0.2.

[0080] The weighted sum of quantitative indicators P free =50%×N free +30%×M free +20%×M free ; P free Compare with the reference values ​​of the three quantization levels in the table to determine the final quantization result. The corresponding relationship is as follows:

[0081] Sufficient resources: P much ≤P free <P normal ;

[0082] Resources: P normal ≤P free <P less ;

[0083] Resources are tight: P free ≤P less .

[0084] In other embodiments, the weight of the resource-dependent load may be set to other ratio values ​​as needed, and is not limited to 5:3:2.

[0085] Step S220: When the backend quantization result indicates that the resources are sufficient, a first target quantization result is obtained.

[0086] Step S230, when the back-end quantization result is normal resources or tight resources, obtain the front-end quantization index of the front-end image acquisition device 31, quantize the remaining performance of the front-end image acquisition device based on the front-end quantization index, and obtain the front-end quantization result, and obtain the second target quantization result based on the front-end quantization result and the back-end quantization result.

[0087] Specifically, the front-end quantitative index includes the remaining neural network CNN computing power, remaining memory, and remaining CPU resources of the front-end image acquisition device 31. The front-end image acquisition device 31 collects and calculates its own parameters according to the instructions of the back-end server 32, and sends the final front-end quantitative index to the back-end server 32. The front-end quantitative index is used to further analyze the remaining performance of the front end. The quantitative standard can adopt the same quantitative standard as the back-end server 32, so as to decide the optimal algorithm scheduling scheme suitable for the remaining load of the current device.

[0088] In some of the embodiments, based on the received privacy protection task request, the remaining performance of the backend server 32 is quantified to obtain a backend quantification result; the backend quantification result includes any one of sufficient resources, ordinary resources and tight resources, including:

[0089] Step S211, based on the received privacy protection task request, obtaining the backend quantitative index of the backend server 32.

[0090] Specifically, the quantitative indicators include remaining neural network computing power, remaining memory, and remaining CPU resources; the remaining memory and remaining CPU resources can be obtained using the query commands provided by the operating system, and the remaining neural network computing power can be calculated based on the number of tasks that are enabled.

[0091] Step S212, performing weighted sum calculation on each quantization index based on a preset load weight ratio to obtain a backend quantization result.

[0092] Step S213: Based on a preset quantization degree standard, the backend quantization result is determined as any one of sufficient resources, ordinary resources, and scarce resources.

[0093] In this embodiment, the remaining performance of the back-end server 32 is deeply analyzed and subdivided based on quantitative indicators, which can improve resource utilization efficiency and provide more reasonable support for subsequent decision-making.

[0094] In some embodiments, the backend quantitative indicators include the remaining neural network computing power, the remaining memory, and the remaining CPU resources. When one type of algorithm task is enabled, the remaining neural network computing power is calculated based on the number of algorithm tasks and the single-type environment computing power consumption of the algorithm tasks; when multiple types of algorithm tasks are enabled, the remaining neural network computing power is calculated based on the number of algorithm tasks and the conservative consumption value of the mixed environment of the currently enabled algorithm tasks.

[0095] Specifically, if only tasks of the same type are currently opened, the calculation formula for the remaining neural network computing power is:

[0096] N free =1-NT ×n;

[0097] Among them, N T is the computing power consumption of a single type of environment corresponding to the task type; n is the number of tasks currently opened.

[0098] The current state is mixed, so the calculation formula for the remaining neural network computing power is:

[0099] ;

[0100] Among them, n is the number of currently opened tasks, N' i It is the conservative consumption value of the mixed environment of the currently opened task. i is the subscript to distinguish different tasks.

[0101] In this embodiment, the remaining neural network computing power is calculated specifically for different task startup scenarios to improve the calculation accuracy of the remaining neural network computing power.

[0102] In some of the embodiments, determining the target algorithm resource based on the credibility of the algorithm resource matching the target quantization result includes:

[0103] Step S310, when the first target quantization result is obtained, the algorithm resource with the highest credibility is used as the target algorithm resource.

[0104] Step S320: When the second target quantization result is obtained, a target algorithm resource is selected from the algorithm resources that are not of the highest credibility.

[0105] In this embodiment, the backend reduces the number of original detection algorithms enabled during privacy protection, thereby improving data utilization and saving the additional resource consumption and computing power performance limitations required to create the original detection algorithm. This alleviates the limitation of the number of backend privacy protection paths due to resource dependence.

[0106] In some of the embodiments, when the second target quantization result is obtained, selecting a target algorithm resource from the algorithm resources that are not of the highest credibility includes:

[0107] Step S321, when the back-end quantization result is that the resources are ordinary or the resources are tight, and the front-end quantization result is that the front-end resources are sufficient, the high-precision algorithm resources deployed on the front-end image acquisition device 31 are used as the target algorithm resources.

[0108] Step S322, when the back-end quantization result is that the resources are ordinary and the front-end quantization result is that the non-front-end resources are sufficient, the medium-precision algorithm resources deployed on the front-end image acquisition device 31 are used as the target algorithm resources.

[0109] Step S323, when the back-end quantization result is that resources are tight and the front-end quantization result is that non-front-end resources are sufficient, the medium-precision algorithm resources deployed on the front-end image acquisition device 31, the low-precision algorithm resources deployed on the front-end image acquisition device 31, the medium-precision algorithm resources deployed on the back-end server 32, and the low-precision algorithm resources deployed on the back-end server 32 are used as target algorithm resources.

[0110] Specifically, the algorithm resources of different precisions deployed on the front-end image acquisition device 31 or the back-end server 32 are not limited to one type, and a plurality of detection and recognition algorithms are provided for each type of credibility as required.

[0111] In this embodiment, target result data generated by various existing front-end and back-end detection and recognition algorithms are used as the target set for privacy protection as much as possible, so that the functional effect of privacy protection covers more targets and expands the usage scenarios of the privacy function.

[0112] In some of the embodiments, privacy protection processing is performed on the collected video data based on the detection result of the target algorithm resource, including:

[0113] Step S410, when the back-end quantization result is that resources are tight and the front-end quantization result is that non-front-end resources are sufficient, obtain the video data collected by the front-end image acquisition device, and detect the video data based on the target algorithm resources deployed on the back-end server 32 to obtain the back-end detection result; and obtain the front-end detection result after the target algorithm resources deployed on the front-end image acquisition device 31 detect the video data.

[0114] Specifically, when the back-end quantization result is that resources are tight and the front-end quantization result is that non-front-end resources are sufficient, the back-end server 32 controls the front-end image acquisition device 31 to start the medium-precision algorithm resources deployed on the front-end image acquisition device 31 and the low-precision algorithm resources deployed on the front-end image acquisition device 31; the back-end server 32 also controls itself to start the medium-precision algorithm resources deployed on the back-end server 32 and the low-precision algorithm resources deployed on the back-end server 32.

[0115] The front-end image acquisition device 31 and the back-end server 32 respectively use their own enabled target algorithm resources to detect the same video data. Since the enabled target algorithm resources on each device include multiple algorithms, the obtained front-end detection results and back-end detection results also each include multiple detection results.

[0116] Step S420, obtaining the privacy target to be shielded based on the detection result with the largest number of detected targets in the front-end detection results and the back-end detection results.

[0117] Step S430, based on the intersection of the target areas in each front-end detection result and the back-end detection result, the superposition range of the privacy target to be shielded is obtained.

[0118] Specifically, in response to resource constraints, this embodiment proposes a scheduling method that combines the analysis of medium and low credibility front-end and back-end analysis algorithms. In order to make target identification more accurate, a privacy protection target method is adopted that prioritizes the maximum number of detections and takes the intersection of the superimposed area. Exemplarily, the number of targets detected by the medium credibility algorithm is recorded as m, and the number of targets detected by the low credibility intelligent algorithm is recorded as n. If m < n, the number of valid targets is counted as n to avoid the omission of privacy shielding due to too few targets. The area of ​​the target detected by the medium credibility algorithm is recorded as A. i (where i represents the i-th target), the target detected by the low-confidence algorithm is denoted as B i (where i represents the i-th target), the area that needs to be superimposed for privacy protection is recorded as C i (where i represents the i-th target) Where: C i =A i ∩B i .

[0119] In this embodiment, the method of taking the intersection of the superimposed areas is used to ensure the optimal effect of the target superimposed range.

[0120] The present embodiment is described and illustrated below through preferred embodiments.

[0121] Figure 6 FIG. 1 is a flowchart of a privacy protection method based on multi-algorithm resource scheduling in this preferred embodiment. Figure 6 As shown, the privacy protection method based on multi-algorithm resource scheduling includes the following steps:

[0122] S1: The backend server 32 receives a new privacy protection task request;

[0123] S2: The post-intelligent analysis capability takes precedence over the pre-intelligent analysis capability. The remaining resources of the back-end device are quantified first according to the load weight, and then a decision is made whether to quantify the remaining resources of the front-end based on the quantification result. The quantification method is the same as that of the back-end.

[0124] S3: Based on the quantification results of S2, make decisions based on the current remaining performance and select the optimal algorithm resources to start. Analyze the algorithm resource weight credibility from high to low: rear intelligent face / body detection, front intelligent face / body detection, back-end CPU non-deep learning version face detection, rear intelligent object detection, SMD2.0 detection, SMD1.0 detection, motion detection.

[0125] S4: Privacy protection is superimposed based on the detection results of the enabled algorithm resources.

[0126] Further, such as Figure 7 As shown, if the quantification result of the backend server 32 is that the resources are sufficient, the post-intelligent high-confidence algorithm is selected to complete the target analysis.

[0127] Further, such as Figure 8 As shown, if the quantification result of the back-end server 32 is that the resources are normal, the remaining load on the front end will continue to be quantified. If the front-end quantification result is that the resources are sufficient, the high-credibility front intelligence will be turned on. If the front-end quantification result is that the resources are normal or the resources are tight, the medium-credibility rear intelligence will be turned on for analysis.

[0128] Further, such as Fig. 9 As shown in the figure, if the back-end quantification result is that resources are tight, the remaining load of the quantified front-end will continue to be quantified. If the front-end resources are sufficient, the high-confidence front-end intelligent analysis algorithm will be selected to start. If the quantification result is that resources are ordinary or tight, the medium- and low-confidence front-end and back-end intelligent combination method will be selected to start the target analysis algorithm. The purpose of the medium- and low-confidence front-end and back-end intelligent combination scheduling method is to obtain the optimal effect of the target superposition range by giving priority to the maximum number of detections and taking the intersection of the superposition area.

[0129] In this preferred embodiment, the focus is on the extreme mining of equipment intelligent analysis under limited performance, using multiple types of auxiliary data to enhance target judgment, reduce algorithm startup consumption and improve the number of paths of the server's overall privacy protection. It has the following advantages: (1) The principle of adopting this scheme is to use the target result data generated by various existing front-end and back-end detection and recognition algorithms as the target set for privacy protection as much as possible. The functional effect of privacy protection can cover more targets and expand the use scenarios of privacy functions; (2) The back-end reduces the activation of the original detection algorithm during privacy protection, so that data utilization is improved, saving the additional resource consumption and computing power performance limitations required to create the original detection algorithm. Relieve the limitation of the number of back-end privacy protection paths by resource dependence. (3) The method of taking quantity first and area intersection allows the device to achieve the maximum number of privacy protection superposition targets and the optimal superposition area under low performance conditions, avoiding the problem of target omission and inaccurate superposition range.

[0130] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0131] In this embodiment, a privacy protection device based on multi-algorithm resource scheduling is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "sub-unit", etc. used below can implement a combination of software and / or hardware with predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0132] Fig.10 is a structural block diagram of the privacy protection device based on multi-algorithm resource scheduling of this embodiment, such as Fig.10 As shown, the device includes: the device is applied to a video monitoring system, the video monitoring system includes a front-end image acquisition device 31 and a back-end server 32; the device is applicable to the back-end server 32; the device includes:

[0133] The remaining performance quantification module 10 is used to quantify the remaining performance of the back-end server 32 and the front-end image acquisition device 31 based on the received privacy protection task request to obtain a target quantification result.

[0134] The matching algorithm resource module 20 is used to determine the target algorithm resource based on the credibility of the algorithm resource matching the target quantization result; the algorithm resource credibility is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server 32 and the front-end image acquisition device 31.

[0135] The privacy protection processing module 30 is used to perform privacy protection processing on the collected video data based on the detection result of the target algorithm resource.

[0136] In some of the embodiments, the remaining performance quantification module 10 is also used to quantify the remaining performance of the back-end server 32 based on the received privacy protection task request to obtain a back-end quantification result; the back-end quantification result includes any one of sufficient resources, normal resources and tight resources; when the back-end quantification result is sufficient resources, a first target quantification result is obtained; when the back-end quantification result is normal resources or tight resources, the front-end quantification index of the front-end image acquisition device 31 is obtained, and based on the front-end quantification index, the remaining performance of the front-end image acquisition device is quantified to obtain the front-end quantification result, and the second target quantification result is obtained based on the front-end quantification result and the back-end quantification result.

[0137] In some of the embodiments, the remaining performance quantification module 10 is also used to obtain the backend quantification indicators of the backend server 32 based on the received privacy protection task request; perform weighted sum calculation on each quantification indicator based on a preset load weight ratio to obtain a backend quantification result; based on a preset quantification degree standard, determine the backend quantification result as any one of sufficient resources, ordinary resources and tight resources.

[0138] In some of the embodiments, the back-end quantitative indicators include remaining neural network computing power, remaining memory and remaining CPU resources; wherein, when one type of algorithm task is started, the remaining neural network computing power is calculated based on the number of algorithm tasks and the single-type environment computing power consumption of the algorithm tasks; when multiple types of algorithm tasks are started, the remaining neural network computing power is calculated based on the number of algorithm tasks and the conservative consumption value of the mixed environment of the currently started algorithm tasks; the remaining memory and the remaining CPU resources are obtained based on the query command in the operating system.

[0139] In some of the embodiments, the matching algorithm resource module 20 is further used to use the algorithm resource with the highest credibility as the target algorithm resource when the first target quantization result is obtained; and to select the target algorithm resource from the algorithm resources that are not the highest credibility when the second target quantization result is obtained.

[0140] In some of the embodiments, the matching algorithm resource module 20 is also used to use the high-precision algorithm resources deployed on the front-end image acquisition device 31 as the target algorithm resources when the back-end quantization result is ordinary resources or tight resources and the front-end quantization result is sufficient front-end resources; when the back-end quantization result is ordinary resources and the front-end quantization result is not sufficient front-end resources, use the medium-precision algorithm resources deployed on the front-end image acquisition device 31 as the target algorithm resources; when the back-end quantization result is tight resources and the front-end quantization result is not sufficient front-end resources, use the medium-precision algorithm resources deployed on the front-end image acquisition device 31, the low-precision algorithm resources deployed on the front-end image acquisition device 31, the medium-precision algorithm resources deployed on the back-end server 32, and the low-precision algorithm resources deployed on the back-end server 32 as the target algorithm resources.

[0141] In some of the embodiments, the privacy protection processing module 30 is also used to obtain video data collected by the front-end image acquisition device 31 when the back-end quantization result is that resources are tight and the front-end quantification result is that non-front-end resources are sufficient, and detect the video data based on the target algorithm resources deployed on the back-end server 32 to obtain the back-end detection result; and obtain the front-end detection result obtained after the target algorithm resources deployed on the front-end image acquisition device 31 detect the video data; obtain the privacy target to be blocked based on the detection result with the largest number of detected targets among the front-end detection results and the back-end detection results; and obtain the superposition range of the privacy target to be blocked based on the intersection of the target areas in each front-end detection result and the back-end detection result.

[0142] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0143] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0144] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0145] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0146] In addition, in combination with the privacy protection method based on multi-algorithm resource scheduling provided in the above embodiment, a storage medium can also be provided in this embodiment to implement. The storage medium stores a computer program; when the computer program is executed by the processor, any one of the privacy protection methods based on multi-algorithm resource scheduling in the above embodiment is implemented.

[0147] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0148] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0149] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0150] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A privacy protection method based on multi-algorithm resource scheduling, characterized in that: The method is applied to a video surveillance system, which includes a front-end image acquisition device and a back-end server; The method is applicable to the back-end server; the method comprises: Based on the received privacy protection task request, quantify the remaining performance of the back-end server and the front-end image acquisition device to obtain a target quantification result; Determine the target algorithm resource based on the credibility of the algorithm resource matching the target quantization result; the credibility of the algorithm resource is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server and the front-end image acquisition device; Wherein, determining the target algorithm resource based on the credibility of the algorithm resource matching the target quantization result includes: The target quantization result includes a back-end quantization result and a front-end quantization result; when the back-end quantization result indicates that the resource is ordinary or the resource is tight, and the front-end quantization result indicates that the front-end resource is sufficient, the high-precision algorithm resource deployed on the front-end image acquisition device is used as the target algorithm resource; When the back-end quantization result is that the resources are ordinary, and the front-end quantization result is that the non-front-end resources are sufficient, the medium-precision algorithm resources deployed on the front-end image acquisition device are used as the target algorithm resources; When the back-end quantization result is that resources are scarce, and the front-end quantization result is that the non-front-end resources are sufficient, the medium-precision algorithm resources deployed on the front-end image acquisition device, the low-precision algorithm resources deployed on the front-end image acquisition device, the medium-precision algorithm resources deployed on the back-end server, and the low-precision algorithm resources deployed on the back-end server are used as the target algorithm resources; Performing privacy protection processing on the collected video data based on the detection result of the target algorithm resource; The privacy protection processing of the collected video data based on the detection result of the target algorithm resource includes: Based on the detection result of the target algorithm resource, a privacy target to be blocked is obtained; based on the privacy target to be blocked, an overlay range is determined to perform privacy protection processing.

2. The privacy protection method based on multi-algorithm resource scheduling according to claim 1 is characterized in that: Based on the received privacy protection task request, quantify the remaining performance of the back-end server and the front-end image acquisition device to obtain a target quantification result, including: Based on the received privacy protection task request, quantify the remaining performance of the backend server to obtain a backend quantification result; the backend quantification result includes any one of sufficient resources, ordinary resources and tight resources; When the backend quantization result indicates that the resources are sufficient, a first target quantization result is obtained; When the back-end quantization result indicates that the resources are normal or the resources are tight, the front-end quantization index of the front-end image acquisition device is obtained, and based on the front-end quantization index, the remaining performance of the front-end image acquisition device is quantified to obtain the front-end quantization result, and the second target quantization result is obtained based on the front-end quantization result and the back-end quantization result.

3. The privacy protection method based on multi-algorithm resource scheduling according to claim 2 is characterized in that: quantifying the remaining performance of the backend server based on the received privacy protection task request to obtain a backend quantification result; The backend quantitative result includes any one of sufficient resources, ordinary resources and scarce resources, including: Based on the received privacy protection task request, obtaining backend quantitative indicators of the backend server; Performing weighted sum calculation on each of the quantitative indicators based on a preset load weight ratio to obtain a backend quantitative result; Based on a preset quantification degree standard, the backend quantification result is determined as any one of sufficient resources, ordinary resources, and scarce resources.

4. The privacy protection method based on multi-algorithm resource scheduling according to claim 3 is characterized in that: The backend quantitative indicators include remaining neural network computing power, remaining memory and remaining CPU resources; Wherein, when one type of algorithm task is started, the remaining neural network computing power is calculated based on the number of the algorithm tasks and the single-type environment computing power consumption of the algorithm tasks; When multiple types of algorithm tasks are enabled, the remaining neural network computing power is calculated based on the number of the algorithm tasks and the mixed environment conservative consumption value of the currently enabled algorithm tasks; The remaining memory and the remaining CPU resources are obtained based on a query command in the operating system.

5. The privacy protection method based on multi-algorithm resource scheduling according to claim 2 is characterized in that: Determining a target algorithm resource based on the credibility of the algorithm resource matching the target quantization result includes: When the first target quantization result is obtained, the algorithm resource with the highest credibility is used as the target algorithm resource, and the algorithm resource with the highest credibility is a high-precision algorithm resource deployed on the back-end server; When the second target quantization result is obtained, a target algorithm resource is selected from the algorithm resources that are not of the highest credibility.

6. The privacy protection method based on multi-algorithm resource scheduling according to claim 1 is characterized in that: The collected video data is subjected to privacy protection processing based on the detection result of the target algorithm resource, including: When the back-end quantification result is that resources are tight, and the front-end quantification result is that the non-front-end resources are sufficient, the video data collected by the front-end image acquisition device is obtained, and the video data is detected based on the target algorithm resources deployed on the back-end server to obtain the back-end detection result; and the front-end detection result is obtained after the target algorithm resources deployed on the front-end image acquisition device detect the video data; In the front-end detection results and the back-end detection results, based on the detection result with the largest number of detected targets, the privacy target to be shielded is obtained; Based on the intersection of the target areas in each of the front-end detection results and the back-end detection results, the superposition range of the privacy target to be shielded is obtained.

7. A privacy protection device based on multi-algorithm resource scheduling, characterized in that: The device is applied to a video monitoring system, which includes a front-end image acquisition device and a back-end server; The device is applicable to the back-end server; the device comprises: A residual performance quantification module, used to quantify the residual performance of the back-end server and the front-end image acquisition device based on the received privacy protection task request to obtain a target quantification result; A matching algorithm resource module, used to determine a target algorithm resource based on the credibility of the algorithm resource matching the target quantization result; the credibility of the algorithm resource is pre-set based on the accuracy and deployment location of the algorithm resource; the deployment location includes the back-end server and the front-end image acquisition device; Wherein, determining the target algorithm resource based on the credibility of the algorithm resource matching the target quantization result includes: The target quantization result includes a back-end quantization result and a front-end quantization result; when the back-end quantization result indicates that the resource is ordinary or the resource is tight, and the front-end quantization result indicates that the front-end resource is sufficient, the high-precision algorithm resource deployed on the front-end image acquisition device is used as the target algorithm resource; When the back-end quantization result is that the resources are ordinary, and the front-end quantization result is that the non-front-end resources are sufficient, the medium-precision algorithm resources deployed on the front-end image acquisition device are used as the target algorithm resources; When the back-end quantization result is that resources are scarce, and the front-end quantization result is that the non-front-end resources are sufficient, the medium-precision algorithm resources deployed on the front-end image acquisition device, the low-precision algorithm resources deployed on the front-end image acquisition device, the medium-precision algorithm resources deployed on the back-end server, and the low-precision algorithm resources deployed on the back-end server are used as the target algorithm resources; A privacy protection processing module is used to perform privacy protection processing on the collected video data based on the detection result of the target algorithm resource; wherein the privacy protection processing on the collected video data based on the detection result of the target algorithm resource includes: Based on the detection result of the target algorithm resource, a privacy target to be blocked is obtained; based on the privacy target to be blocked, an overlay range is determined to perform privacy protection processing.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images