Low-resource-overhead method and system for face recognition algorithm scene application

Abstract

The invention belongs to the technical field of computer vision and artificial intelligence, and provides a low-resource-overhead method and system for face recognition algorithm scene application, and the method comprises the steps: carrying out the preprocessing of a video through an edge service host, and carrying out the preliminary filtering, screening and duplicate removal of face recognition, and generating a face feature value; the background face recognition server receives the face feature values, compares and recognizes the face feature values, and generates a recognition result; the background face recognition server returns a recognition result to the edge service host and records face recognition data at the same time; and the edge service host receives the identification result and displays the identification result through the local display device. By solving the problem of high bandwidth occupation, improving the computing power efficiency, reducing the equipment overhead and eliminating the information leakage risk, the performance and safety of the face recognition technology in the visual image AI field are remarkably improved, flexible deployment and multi-protocol access are supported, and the compatibility and expansibility of the system are improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and artificial intelligence technology, and in particular relates to a method and system with low resource overhead for face recognition algorithm scenario application. Background Art

[0002] In existing visual image AI application scenarios, the technical architecture of cloud and end, front-end and back-end is usually adopted. The end device is usually a functional physical host running Android, Linux, or Windows operating system, and the back-end device is usually a containerized cloud server or a local physical server. Typical applications include face recognition access control host and face recognition management background, traffic and passenger flow analysis streaming media host and algorithm computing platform, object recognition and behavior analysis front-end video collector and back-end algorithm computing platform.

[0003] In the existing visual image AI face recognition algorithm scenarios, there are mainly two architectural models: the "front-end video acquisition-back-end comparison and recognition-back-end management and recording" model and the "front-end video acquisition-front-end preprocessing-front-end algorithm processing-front-end comparison and recognition-back-end management and recording" model.

[0004] The main similarities between the two models are: video preprocessing and algorithm computing power are integrated in the same physical functional host device (face recognition all-in-one machine) or background server, which can facilitate understanding, reduce technical difficulty and facilitate technical implementation in small-scale applications.

[0005] In the application of visual image AI technology, the main problems faced by the "front-end video acquisition-back-end comparison and recognition-back-end management and recording" model include: as the video resolution increases and the concurrent pressure increases, the network bandwidth is tight, causing video stream frame drops, freezes, overloads and other problems, affecting other businesses and increasing the network burden. At the same time, the back-end centrally handles video preprocessing and algorithms, wasting computing resources.

[0006] In contrast, although the "front-end video acquisition-front-end preprocessing-front-end algorithm processing-front-end comparison and recognition-back-end management and recording" model disperses the processing to the front-end and reduces bandwidth pressure, it faces information leakage and data security risks, as well as the processing capacity and aging of front-end equipment, which may lead to increased recognition time and decreased equipment performance.

[0007] In summary, the system will become slower and slower with continuous use, and the cost of upgrading hardware equipment will also increase. Summary of the invention

[0008] To solve the above problems, the present invention provides a method and system with low resource overhead for face recognition algorithm scenario applications. It adopts the "cloud-edge-end" technical architecture, builds "dynamic threads and dynamic engines" as the core method concept based on edge applications, creates threads and engine pools, and supports face recognition systems with multiple protocols and multi-channel camera modules, thereby improving the resource utilization of terminal hardware infrastructure.

[0009] The technical solution adopted by the present invention is:

[0010] A low resource cost method for face recognition algorithm scenario application, the method is as follows:

[0011] The edge service host performs pre-processing of the video and preliminary filtering, screening and deduplication of face recognition to generate facial feature values;

[0012] The backend face recognition server receives the face feature value, compares and identifies the face feature value, and obtains a recognition result; the backend face recognition server returns the recognition result to the edge service host, and records the face recognition data at the same time;

[0013] The edge service host receives the recognition result and displays the recognition result through a local display device.

[0014] Furthermore, the edge service host obtains video by accessing multiple protocols and multiple camera modules.

[0015] Furthermore, the edge service host performs pre-processing on the video and preliminary filtering, screening and deduplication of face recognition to generate face feature values, including:

[0016] Video data stream decoding: decoding the video into original frame images;

[0017] Face detection: detecting the face position in the original frame image;

[0018] Filtering and screening of valid portraits: filtering and screening faces according to the face positions, and eliminating invalid, duplicate or low-quality portrait images;

[0019] Anti-tilt calculation: Anti-tilt calculation and posture correction are performed based on the face position information;

[0020] Key point detection: detect key feature points in the rectified face image;

[0021] Generate facial feature values: Extract and generate facial feature values ​​based on key feature point information.

[0022] Furthermore, the edge service host and the background face recognition server transmit information through a trunk network.

[0023] Further, the background face recognition server receives the face feature value, and compares and identifies the face feature value to obtain a recognition result; the background face recognition server returns the recognition result to the edge service host, and records the face recognition data, including:

[0024] Comparison and recognition: comparing the facial feature values ​​with the feature values ​​in the database;

[0025] Generate recognition results: Generate recognition results based on the comparison results;

[0026] Delivery result event: Return the recognition result to the edge service host.

[0027] Furthermore, the face detection also includes portrait micro-motion detection, which is performed simultaneously to ensure that a real face is detected.

[0028] Furthermore, the filtering and screening of the effective portraits includes:

[0029] Blurred portrait judgment, edge truncation portrait judgment, flat portrait judgment, portrait judgment with excessive tilt angle, portrait micro-motion judgment and same face filtering.

[0030] Furthermore, the display device is replaced with a screen of a local face recognition all-in-one machine to perform human-computer interaction and display the recognition result.

[0031] A low-resource-overhead system for face recognition algorithm scenario applications, characterized in that the system includes a local display device, an edge service host and a background face recognition server; the edge service host is connected to the background face recognition server via a trunk network; the local display device is connected to the edge service host.

[0032] Furthermore, the edge service host is used to pre-process the video and perform preliminary filtering, screening and deduplication of face recognition, generate facial feature values, and transmit the facial feature values ​​to the background face recognition server; the background face recognition server is used to compare and identify the facial feature values, and return the recognition results to the edge service host, and the edge service host displays the recognition results through the local display device.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1. Reduce network bandwidth usage: By moving high-throughput services forward, the trunk bandwidth usage is significantly reduced. Especially in large-scale applications, it effectively solves common problems such as frame drops, freezes, and overloads in the video platform scheduling process, thereby greatly alleviating network pressure.

[0035] 2. Improve operational efficiency: The preprocessing of the face recognition algorithm by the edge server significantly improves the operational efficiency of the face recognition server, especially when computing resources are precious, with less CPU / GPU consumption to achieve the same performance. In the case of high concurrency business, the speed of the face recognition business system is increased by 25-50%.

[0036] 3. Centralized management and data security: All data is stored in the backend face recognition server for centralized management, which is convenient for maintenance and real-time update. At the same time, since the data is stored in a remote controlled computer room, the risk of face information leakage and data security is greatly reduced.

[0037] 4. Decoupling local device performance requirements: By concentrating computing power on the background face recognition server, the problem of increasing performance requirements of the local face recognition all-in-one machine as the face database grows is effectively decoupled. By taking advantage of the elastic deployment of the server, computing power resources are allocated on demand and by time period, and edge service hosts are introduced with relatively small investment to achieve the goal of reducing resource overhead.

[0038] 5. Compatibility and scalability: The method and system of the present invention are compatible with the current mainstream technical architecture. Whether it is the "front-end video acquisition-back-end comparison and recognition-back-end management and recording" model or the "front-end video acquisition-front-end preprocessing-front-end algorithm processing-front-end comparison and recognition-back-end management and recording" model, it can be transformed and upgraded by adding edge service hosts, so that the existing monitoring infrastructure can also quickly iterate to complete the visual image AI face recognition algorithm scenario application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A schematic flow chart of a method for low resource overhead in a face recognition algorithm scenario application according to an embodiment of the present invention is shown;

[0041] Figure 2 A schematic diagram of a data processing flow of a method with low resource overhead for face recognition algorithm scenario application according to an embodiment of the present invention is shown;

[0042] Figure 3 A schematic diagram showing a method for low resource overhead in a face recognition algorithm scenario application according to an embodiment of the present invention is shown;

[0043] Figure 4A schematic diagram of the structural principle of low resource overhead for a face recognition algorithm scenario application according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] like Figure 1 As shown, the present invention provides a method for low resource overhead in face recognition algorithm scenario application, the method comprising the following steps:

[0046] S1: The edge service host obtains videos by accessing multiple protocols and multiple camera modules, and performs pre-processing and preliminary filtering, screening and deduplication of face recognition on the videos to generate face feature values, including:

[0047] like Figure 2 and Figure 3 As shown, the "thread pool, engine pool" method is adopted in the edge service host to handle different recognition tasks and manage multiple face recognition engines by constructing dynamic multi-threading and multi-engine support.

[0048] The face recognition engine includes: face detection, filtering and screening of effective portraits, anti-tilt calculation, key point detection, generation of face feature values, comparison recognition and generation of recognition results. Among them, the edge service host is used to complete the tasks of face detection, filtering and screening of effective portraits, anti-tilt calculation, key point detection and generation of face feature values.

[0049] S11: Video data stream decoding: decoding the video into original frame images.

[0050] S12: Face detection: Detect the face position in the original frame image; wherein, while performing the face detection, portrait micro-motion detection can also be performed to ensure that the detected face is a real face.

[0051] S13: filtering and screening of valid portraits: filtering and screening faces according to the face positions, and eliminating invalid, repeated or low-quality portrait images;

[0052] The filtering of the effective portraits includes:

[0053] S13-1: Blurred portrait judgment refers to determining whether a head image is clear enough for effective processing or recognition.

[0054] The specific method is: by converting the original frame image into a grayscale image G(x,y), using the Laplace operator to calculate the sharpness image S(x,y), and calculating the variance of the sharpness image To judge the clarity of the image, if the variance value is If the variance value is between 100 and 10,000, the image is considered normal; if the variance value is lower than 100 or higher than 10,000, the image is considered blurred or has interference and is removed without subsequent face recognition processing, thus filtering out blurred portraits. The calculation formula is as follows:

[0055]

[0056] Where: W and H are the pixel width and height of the sharpness image S(x,y); It is the variance of the sharpened image, which reflects the distribution range and variation of pixel values ​​in the image. A higher variance means richer details and more obvious edges in the image.

[0057] S13-2: edge truncation portrait judgment, which refers to detecting whether a portrait is partially truncated or out of bounds;

[0058] The specific method is: determine a reliable face frame according to the face position, and calculate the minimum boundary distance Gap. The calculation formula is as follows:

[0059] Gap=min(x,y,W-(x+w),H-(y+h))

[0060] Among them, W and H are the total width and height of the pixels of the entire image, w and h are the height and width of the face frame, and x and y are the coordinates of the starting point p(x,y) in the upper left corner of the face frame.

[0061] If Gap≥0, the portrait is considered to be within the entire image range and is a valid image. To exclude edge optical imaging distortion, the filter pixel threshold t can be set, 0<t<min(W / 2,H / 2). By judging Gap≥t, a typical t can be 10-100 pixels. This means that portrait images with face frames at the edge of the entire image can be excluded, and portrait images with complete face information can be screened out to avoid portraits being truncated by the edge.

[0062] S13-3: Planar portrait judgment refers to determining whether a portrait in a picture is planar.

[0063] The specific method is: use image tool algorithms to process the original frame image, such as Canny edge detection, remove the edge image I (x, y) that does not contain the main face information, and then calculate the variance of all remaining background pixels.

[0064]

[0065] like If it is less than or equal to 30, it can be judged as a flat portrait, and the picture should be filtered out.

[0066] S13-4: Judging a portrait with a large tilt angle refers to detecting whether the portrait image is unsuitable for further processing or recognition due to the tilt of the head or the large rotation angle. The facial posture calculation method is as follows:

[0067] The key point detection algorithm is used to obtain the positions of the left and right eyes and the tip of the nose. left =(x l ,y1),E right =(x r ,y r )N center =(x n ,y n ).

[0068] The vectors from the left eye to the right eye and from the right eye to the left eye are calculated as:

[0069]

[0070] Calculate the vectors from the left eye to the tip of the nose and the right eye to the tip of the nose as:

[0071]

[0072] The two angles θ1 and θ2 between the three points are calculated:

[0073]

[0074] Calculate the difference between the two angles to get Δθ:

[0075] Δθ=|θ1-θ2|

[0076] If Δθ=0, it means that the portrait is completely facing forward; if Δθ is empty, at least one key eye information is missing. Under normal circumstances, Δθ is between 0° and 90°. A threshold t is set according to the characteristics of face tilt and rotation. When Δθ≤t is determined, the face is a usable frontal portrait image. Setting t to 30° indicates that a triangle of two eyes and one nose is a typical portrait image with typical face information, thereby filtering out portrait images with excessive tilt and rotation.

[0077] S13-5: Portrait micro-motion judgment refers to the changes in the left and right eyes, left and right mouth, left and right cheeks, and forehead areas. By setting key points and calculating whether the relative position relationship between the key points has changed, it is judged whether the portrait has micro-motion. Portraits with micro-motion are considered to be closer to real and effective people, avoiding static pictures, masks, dolls, print advertisements, and clothing logos from being calculated in the subsequent face recognition process, thus achieving a filtering effect.

[0078] The specific method is: among the typical 68 facial detection key points, select the left and right eyes, left and right mouths, left and right cheeks, and forehead areas to set n key point coordinates containing x and y. In the image at time t = 0, calculate the center point of the n key points in the selected area:

[0079]

[0080] Then the selected n key points and the center point The total distance D between t :

[0081] D t =d1+d2+…+d n

[0082] At two different times T , D T+1 , under normal circumstances, the interval between the two is 100-1000 milliseconds, and the normalized amplitude of the change is obtained part :

[0083]

[0084] δ part ≤1, when it exceeds 0.2, it is considered that there is a change in expression in the local area. part When <0.2, set δ part = 0. Face available: left eye δ l_eye , right eyeδ r_eye , left mouth δ l_mout h , right mouth δ r_mout h , left cheek l_cheek , right cheekδ r_cheek 、Forehead head , the variation range of the 7 main regions is superimposed to obtain ∈:

[0085] ∈=δ l_eye +δ r_eye +δ lmout h +δ r_mout h +δ l_cheek +δ r_cheek +δ headSince facial expressions are often accompanied by the linkage of multiple facial muscles, when ∈≥0.35, it can be determined that there is micro-motion on the face.

[0086] S13-6: Same face filtering means that in a continuous video stream, multiple identical portraits are continuously tracked, and the same portrait data will not be repeatedly processed multiple times in a short period of time.

[0087] The specific method is: initially obtain the portrait image at time t0 and mark the face M a , the face frame contains the starting point P0(x, y). If the distance between the position P1(x, y) and P0(x, y) of the face frame of the portrait image at the next sampling time t1 does not exceed d, it is considered to be the same continuous face M a , continue to compare the pixel distance at the previous moment until the position of the face frame P(x, y) n Normally leave the entire image, or stay in the video stream for n seconds, then release M a , re-track the face M in the portrait image b As a restart of the portrait processing at t0, the sampling interval can be preset to 500 milliseconds and the distance d to 200 pixels, thereby achieving the effect of deduplication.

[0088] The above main processes of blurred portrait judgment, edge truncation portrait judgment, flat portrait judgment, portrait judgment with excessive tilt angle, portrait micro-motion judgment and same face filtering together form the face recognition preprocessing mechanism, which processes most low-quality, artifact, invalid and repeated portrait images through filtering, screening and deduplication. It has been verified that this method can reduce computing power and bandwidth by 65% ​​to 85% in different scenarios. This means the high availability of face recognition at the million level, which has obvious economic value and far-reaching social significance for large enterprises and social security management.

[0089] S14: Anti-tilt calculation: anti-tilt calculation and posture correction are performed according to the face position information;

[0090] S15: Key point detection: detecting key feature points in the corrected face image;

[0091] S16: Generate facial feature values: extract and generate facial feature values ​​based on key feature point information.

[0092] S2: The backend face recognition server receives the face feature value through the trunk network, compares and identifies the face feature value, and generates a recognition result; the backend face recognition server returns the recognition result to the edge service host, and records the face recognition data, including:

[0093] S21: Comparison and recognition: comparing the facial feature value with the feature value in a database;

[0094] S22: Generate recognition result: Generate recognition result according to the comparison result;

[0095] S23: Delivery result event: returning the identification result to the edge service host through the trunk network.

[0096] S3: The edge service host receives the recognition result and displays the recognition result through a local display device; wherein the display device can also be replaced with a screen of a local face recognition all-in-one machine to perform human-computer interaction and display the recognition result.

[0097] Based on the above method, the embodiment of the present invention also provides a low resource overhead system for face recognition algorithm scenario application corresponding to the above method, such as Figure 4 As shown, the system includes: a local display device, an edge service host and a background face recognition server; the edge service host and the background face recognition server are connected through a trunk network; the local display device is connected to the edge service host.

[0098] The edge service host is used to pre-process the video and perform preliminary filtering, screening and deduplication of face recognition, generate facial feature values, and transmit them to the background face recognition server through the backbone network; among them, the edge service host can access the video through multiple protocols and multiple camera modules.

[0099] The background face recognition server is used to compare and identify the facial feature values ​​and return the recognition results to the edge service host, which displays the recognition results through the local display device.

[0100] The method and system involved in the present invention are compatible with the existing architecture models: the "front-end video acquisition-back-end comparison and identification-back-end management and recording" model and the "front-end video acquisition-front-end preprocessing-front-end algorithm processing-front-end comparison and identification-back-end management and recording" model. By adding edge service hosts for transformation and upgrading, the existing monitoring infrastructure can also be quickly iterated to complete the visual image AI face recognition algorithm scenario application.

[0101] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A low resource cost method for face recognition algorithm scene application, characterized in that: The method comprises: The edge service host performs pre-processing of the video and preliminary filtering, screening and deduplication of face recognition to generate facial feature values; The backend face recognition server receives the face feature value, compares and identifies the face feature value, and generates a recognition result; the backend face recognition server returns the recognition result to the edge service host, and records the face recognition data at the same time; The edge service host receives the recognition result and displays the recognition result through a local display device.

2. The method according to claim 1, characterized in that The edge service host obtains video by accessing multiple protocols and multiple camera modules.

3. The method according to claim 2, characterized in that The edge service host performs pre-processing of the video and preliminary filtering, screening and deduplication of face recognition to generate face feature values, including: Video data stream decoding: decoding the video into original frame images; Face detection: detecting the face position in the original frame image; Filtering and screening of valid portraits: filtering and screening faces according to the face positions, and eliminating invalid, duplicate or low-quality portrait images; Anti-tilt calculation: Anti-tilt calculation and posture correction are performed based on the face position information; Key point detection: detect key feature points in the rectified face image; Generate facial feature values: Extract and generate facial feature values ​​based on key feature point information.

4. The method according to claim 1, characterized in that: The edge service host and the background face recognition server transmit information through the trunk network.

5. The method according to claim 1, characterized in that The backend face recognition server receives the face feature value, and compares and identifies the face feature value to obtain a recognition result; The backend face recognition server returns the recognition result to the edge service host and records the face recognition data, including: Comparison and recognition: comparing the facial feature values ​​with the feature values ​​in the database; Generate recognition results: Generate recognition results based on the comparison results; Delivery result event: Return the recognition result to the edge service host.

6. The method according to claim 4, characterized in that The face detection also includes portrait micro-motion detection, which is performed simultaneously to ensure that a real face is detected.

7. The method according to claim 4, characterized in that The filtering and screening of the effective portraits includes: Blurred portrait judgment, edge truncation portrait judgment, flat portrait judgment, portrait judgment with excessive tilt angle, portrait micro-motion judgment and same face filtering.

8. The method according to claim 1, characterized in that The display device is replaced with the screen of the local face recognition all-in-one machine to perform human-computer interaction and display the recognition results.

9. A low resource overhead system for face recognition algorithm scene application, characterized in that: The system includes a local display device, an edge service host and a background face recognition server; the edge service host is connected to the background face recognition server via a trunk network; and the local display device is connected to the edge service host.

10. The system according to claim 9, characterized in that The edge service host is used to pre-process the video and perform preliminary filtering, screening and deduplication of face recognition, generate face feature values, and transmit the face feature values ​​to the background face recognition server; The background face recognition server is used to compare and identify the face feature values, and return the recognition result to the edge service host, and the edge service host displays the recognition result through the local display device.

Images