A face filtering model training method, a face recognition method, and a device

Abstract

The application provides a face filtering model training method, a face recognition method and device. In the training process, the face filtering model training method uses the prediction feature layer of the SSD network to extract feature maps in sample pictures and performs bicubic interpolation on the pixel points in each feature map, superimposes the features of each interpolated feature map, and outputs a picture containing a predicted face position. The face recognition method includes: a background filter filtering background video frames in video frames to obtain first type video frames, a face filtering model filtering out video frames without faces to obtain second type video frames, a repeated person filter filtering out video frames containing repeated faces in the second type video frames to obtain a target video frame, and a preset face recognition model identifying the identity of a target person in the target video frame. The application can improve the face recognition efficiency of the face recognition model and the video frame filtering efficiency of the video frame filtering model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a training method for a face filtering model, a face recognition method, and an apparatus. Background Technology

[0002] In order to achieve large-scale video analysis in surveillance video face recognition systems, existing technologies take two approaches. On the one hand, they focus on the algorithm model itself, reducing the computational load of video analysis by pruning unnecessary parameters in the neural network to make the model lighter. On the other hand, they use lightweight algorithms to preprocess the video stream data and use filters to quickly filter out a large number of video frames that are irrelevant to the target event, thereby dynamically reducing the number of video frames that actually reach the face recognition network model and enabling fast online analysis of multiple video streams.

[0003] However, due to the distance between faces and surveillance cameras in surveillance video scenarios, as well as noise interference, it becomes very difficult for models in video frame filtering to detect target faces in some video frames. This makes it impossible to accurately capture target faces in surveillance videos, resulting in some video frames containing target faces being filtered out, leading to missed detections and reducing the accuracy of the model in filtering video frames. At the same time, there are multiple models in the video frame filtering mechanism, and there is a resource competition relationship between these models. Furthermore, the actual scene is dynamically changing, and existing face recognition models cannot adjust the filtering mechanism according to scene changes, thus reducing the efficiency and accuracy of face recognition models in recognizing faces in video frames. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a training method for a face filtering model, a face recognition method, and an apparatus to solve the problems of low filtering efficiency and low face recognition efficiency of existing video frame filtering models.

[0005] One aspect of the present invention provides a method for training a face filtering model, the method comprising the following steps: Obtain a training sample set, which includes multiple samples, each sample including a sample image, add real face location information to each sample image and use the face as a label; A preset neural network is obtained. The preset neural network uses an SSD network to extract prediction feature maps of multiple sizes from the sample image. After interpolation, dimension unification, and superposition of one or more prediction feature maps of each sample image, the data is input into a linear activation function to obtain a fused feature map. Based on the fused feature map, an image containing the predicted face location is identified. The preset neural network is trained using the training sample set. The position loss function and the confidence loss function are weighted and summed to construct the total loss function, which is then used to iteratively update the parameters of the preset neural network to obtain the face filtering model.

[0006] In some embodiments, the preset neural network uses an SSD network to extract predicted feature maps of multiple sizes from the sample image. After interpolating, unifying the dimensions, and superimposing one or more predicted feature maps for each sample image, the results are input into a linear activation function to obtain a fused feature map. Based on the fused feature map, an image containing the predicted face location is identified, including: The sample image is resized to 300×300 and then input into the face filtering model. The SSD network in the face filtering model uses three prediction feature layers, conv4_3, conv5_3 and fc_6, to extract prediction feature maps from the sample image. The first prediction feature map extracted by conv4_3 is obtained by interpolating the pixels using bicubic interpolation and extracting the image features of the first prediction feature map after interpolation using a 3×3×512 convolution kernel. The first feature map is obtained by normalizing the first prediction feature map after interpolation. By deconvolution, the dimension of the second predicted feature map extracted by conv5_3 is made to be consistent with the dimension of the first predicted feature map. Bicubic interpolation is used to interpolate the pixels in the second predicted feature map after deconvolution. A 3×3×512 convolution kernel is used to extract the image features in the second predicted feature map after interpolation. The second predicted feature map is obtained after normalization of the interpolated second predicted feature map. By deconvolution, the dimension of the third predicted feature map extracted by fc_6 is made to be consistent with the dimension of the first predicted feature map. Bicubic interpolation is used to interpolate the pixels in the third predicted feature map after deconvolution. A 3×3×512 convolution kernel is used to extract the image features in the third predicted feature map after interpolation. The third predicted feature map is obtained after normalization of the interpolated third predicted feature map. The features in the first feature map, the second feature map, and the third feature map are superimposed and input into a linear activation function to obtain a first fused feature map. The first fused feature map is convolutionally pooled and output to obtain a second fused feature map containing multiple prediction boxes. Prediction boxes with confidence scores below a threshold and prediction boxes with an overlap rate with the real face location box above a threshold are filtered out, and the image containing the predicted face location is output.

[0007] Another aspect of the present invention provides a face recognition method, comprising: Acquire multiple video streams and assign a unique number to each video stream; Each video stream is input into a video frame filtering model in numerical order. The video frame filtering model includes a background frame filter, a face filtering model as described in the face filtering model training method, and a duplicate person filter, all connected in sequence. The background frame filter uses differential detection to identify a first type of video frame containing moving targets. The first type of video frames are input into the face filtering model and filtered to obtain a second type of video frames containing predicted face locations. The duplicate person filter uses the DSST algorithm to filter video frames containing duplicate faces in the second type of video frames, and only one target video frame is output for duplicate faces. The target video frame is input into a preset face recognition model to obtain the identity information of the target person; The identity information of the target personnel is marked in each video frame of the corresponding video stream and stored in the database.

[0008] In some embodiments, the background frame filter uses differential detection to identify a first class of video frames containing moving targets, including: One video frame is selected from the video stream as the background video frame, and the rest are designated as third-type video frames. The gray values ​​of the pixels in the third-type video frames are sequentially subtracted from the gray values ​​of the pixels in the background video frames to obtain the difference images corresponding to each third-type video frame. The difference images are then binarized, with pixels whose values ​​are less than a set threshold assigned a value of 0 as background points, and pixels whose values ​​are greater than or equal to the threshold assigned a value of 255 as foreground points. This yields the binarized images of the third-type video frames. After performing connectivity analysis on the foreground points in the binarized images, the first-type video frames containing moving targets are output.

[0009] In some embodiments, the grayscale values ​​of pixels in the third type of video frame are sequentially differentially compared with the grayscale values ​​of pixels in the background video frame to obtain a difference image corresponding to each third type of video frame, including: The formula for calculating the difference image is: ; in, This represents the difference image. This represents the grayscale value of a pixel in the third type of video frame. This represents the grayscale value of a pixel in the background video frame. This represents the coordinates of a pixel in the third type of video frame. This represents the coordinates of a pixel in the background video frame. This represents the coordinates of a pixel in the difference image.

[0010] In some embodiments, after inputting the first type of video frames into the face filtering model and filtering to obtain a second type of video frames containing predicted face locations, the method further includes: During processing, the first type of video frames are constructed into a differential queue, and the third type of video frames are constructed into a video frame queue for later use. When the number of first-type video frames in the differential queue is greater than the number of third-type video frames in the video frame queue, the video frames that do not contain faces detected by the face filtering model are updated to the background video frames to continue processing the remaining video frames in the sample video stream.

[0011] In some embodiments, before the preset face recognition model identifies the target video frame to obtain the target person's identity information, the method further includes: The target video frame is preprocessed, including: face alignment, face lighting compensation, grayscale transformation, histogram equalization, normalization, geometric correction, median filtering, and sharpening.

[0012] In some embodiments, the preset face recognition model identifies the target person's identity information by recognizing the target video frame, including: The preset face recognition model extracts the facial features of the target person in the target video frame and matches them with the facial features of multiple known identity information persons stored in the database. If the matching degree between the facial features of the target person and the facial features of a known identity information person in the database is higher than the matching degree threshold, then the target person and the known identity information person are considered to be the same person, thereby obtaining the identity information of the target person.

[0013] On the other hand, the present invention also provides an electronic device including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory, wherein when the computer instructions are executed by the processor, the device implements the steps of the above method.

[0014] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0015] The beneficial effects of the present invention are at least as follows: The training method, face recognition method, and apparatus for the face filtering model described in this invention filter out background video frames from video frames and output first-class video frames containing moving targets through a background frame filter. The face filtering model filters the first-class video frames and outputs second-class video frames containing faces. The duplicate face filtering model filters out video frames containing duplicate faces from the second-class video frames. For the same face, only one target video frame is output. The preset face recognition model identifies the identity information of the people in the target video frames. By filtering step by step, the workload of the preset face recognition model is reduced, while the video frame filtering efficiency of the video frame filtering model is improved.

[0016] Furthermore, by interpolating the predicted feature maps extracted from the predicted feature layer in the SSD network using bicubic interpolation, the facial features in the predicted feature maps are enhanced, preventing some video frames from being filtered out due to unclear faces, and improving the filtering accuracy of the video frame filtering model.

[0017] Furthermore, when the number of first-class video frames in the differential queue is greater than the number of third-class video frames in the video frame queue, the background video frames are updated. Through the feedback mechanism, the background video frames in the background frame filter are adaptively updated, which alleviates the problem of video frame accumulation caused by the different processing speeds of various models, reduces the hardware resource competition between different filters, and improves the recognition efficiency of the face recognition model and the filtering efficiency of the video frame filtering model.

[0018] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0019] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings: Figure 1 This is a schematic diagram of the structure of the video frame filtering model and the preset face recognition model according to an embodiment of the present invention.

[0021] Figure 2 This is a fusion processing flow of three predicted feature maps of the face filtering model described in an embodiment of the present invention.

[0022] Figure 3This is a schematic diagram of the feature fusion structure of the three prediction feature layers of the SSD network according to an embodiment of the present invention.

[0023] Figure 4 This is a flowchart of the background frame filter according to an embodiment of the present invention.

[0024] Figure 5 This is a face recognition method according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0026] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0027] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0028] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0029] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0030] The video surveillance industry is developing rapidly, with the number of cameras in scenarios such as campuses, communities, and factories constantly increasing. However, investigating events from large amounts of surveillance video is highly inefficient. Therefore, video surveillance scenarios utilize numerous intelligent analysis technologies, such as object detection, object tracking, and facial recognition. Among these, facial recognition is the most complex task. Deep learning-based facial recognition technology, with its complex network model structure, significantly improves prediction accuracy, but this leads to high hardware costs. Even when running on high-performance GPUs, the model's processing speed is insufficient to support the real-time analysis of large amounts of surveillance video streams.

[0031] In surveillance video face recognition systems, to achieve large-scale video analysis using high-accuracy face recognition models on hardware devices with limited computing resources, existing technologies address two main issues. First, they focus on the algorithm model itself, pruning unnecessary parameters in neural networks to reduce the computational load of video analysis. Second, they utilize lightweight algorithms to preprocess the video stream data, using filters to quickly remove a large number of video frames irrelevant to the target event, thereby dynamically reducing the number of video frames actually reaching the face recognition network model and enabling rapid online analysis of multiple video streams. However, because target events occur infrequently and target faces repeatedly appear in consecutive video frames, a large number of irrelevant video frames exist in the video stream. Even with lightweight face recognition models, frame-by-frame analysis is still required, wasting significant computing and storage resources and potentially leading to a decrease in recognition accuracy.

[0032] Existing face recognition filtering methods generally use techniques such as background subtraction detection, face detection, and face tracking to filter video frames. In practical applications, surveillance cameras are usually installed in specific locations and maintain a fixed shooting angle to monitor for faces. However, the target face is often far from the surveillance camera, and interference from various noise factors makes direct detection of the target face very difficult. Some video frames are filtered out because the model cannot detect small target faces, resulting in a decrease in the overall filtering efficiency of the filtering system. In addition, actual monitoring scenarios are dynamically changing, and an unreasonable filtering mechanism design cannot guarantee the service quality of multiple models, making it difficult for the system to operate stably. Therefore, this invention provides a training method for a face filtering model, a face recognition method, and an apparatus to solve the problems of low filtering efficiency and low recognition rate of existing face filtering mechanisms.

[0033] One aspect of the present invention provides a method for training a face filtering model, the method comprising steps S101-S103: S101: Obtain the training sample set, which includes multiple samples. Each sample includes a sample image. Add real face location information to each sample image and use the face as a label.

[0034] S102: Obtain a preset neural network. The preset neural network uses an SSD network to extract prediction feature maps of multiple sizes from the sample images. After interpolation, dimensionality unification, and superposition of one or more prediction feature maps for each sample image, the data is input into a linear activation function to obtain a fused feature map. Based on the fused feature map, an image containing the predicted face location is identified.

[0035] S103: The preset neural network is trained using a training sample set. The position loss function and the confidence loss function are weighted and summed to construct the total loss function, which is then used to iteratively update the parameters of the preset neural network to obtain a face filtering model.

[0036] In step S101, the locations of real faces of various types, such as blurred, making expressions, and occluded faces, are outlined in each sample image to obtain real face location information, and the faces are used as labels.

[0037] In step S102, the sample image size is adjusted to 300×300 and then input into the face filtering model. The SSD network in the face filtering model extracts predicted feature maps from the sample image using three prediction feature layers: conv4_3, conv5_3, and fc_6. For example... Figure 2 As shown, bicubic interpolation is used to interpolate the pixels in the first predicted feature map extracted by conv4_3, and a 3×3×512 convolution kernel is used to extract the image features in the interpolated first predicted feature map. After normalization, the first feature map is obtained. Deconvolution is used to make the dimension of the second predicted feature map extracted by conv5_3 consistent with the dimension of the first predicted feature map. Bicubic interpolation is then used to interpolate the pixels in the deconvolutioned second predicted feature map, and a 3×3×512 convolution kernel is used to extract the image features in the interpolated second predicted feature map. After normalization, the second feature map is obtained. By deconvolution, the dimension of the third predicted feature map extracted by fc_6 is made to be consistent with the dimension of the first predicted feature map. Bicubic interpolation is used to interpolate the pixels in the third predicted feature map after deconvolution. A 3×3×512 convolution kernel is used to extract the image features in the interpolated third predicted feature map. The third feature map is obtained after normalization of the interpolated third predicted feature map.

[0038] like Figure 3 As shown, the features from the first, second, and third feature maps are superimposed and input into a linear activation function to obtain a first fused feature map. The first fused feature map is then convolutionally pooled to output a second fused feature map containing multiple predicted bounding boxes. Predicted bounding boxes with confidence levels below a threshold and those with an overlap rate higher than a threshold with the actual face location are filtered out, resulting in an image containing the predicted face location.

[0039] In this network, the first predicted feature map extracted by conv4_3 has a size of 38×38×512, the second predicted feature map extracted by conv5_3 has a size of 19×19×512, and the third predicted feature map extracted by fc_6 has a size of 19×19×1024. Deconvolution is used to reduce the size of both the second and third predicted feature maps to 38×38×512. Different convolutional layers in the SSD network extract high-semantic, low-resolution high-level predicted feature maps and low-semantic, high-resolution low-level predicted feature maps, respectively. After bicubic interpolation of each predicted feature map, it is input into the ReLU activation function to obtain the first fused feature map. Convolutional pooling of the first fused feature map outputs a second fused feature map with a size of 1×1×128. Fusing the high-level and low-level predicted feature maps allows the model to learn more layers of image features. The prediction boxes with confidence levels below the threshold in the second fused feature map are filtered out by a confidence threshold. The unfiltered prediction boxes are decoded to obtain the parameters of the real face box location. Non-maximum suppression is used to remove prediction boxes with an overlap rate greater than the threshold with the real face box. The final prediction box location is the location of the face in the video frame. At the same time, the face filtering model scores the faces in the output image.

[0040] In some embodiments, the expression for bicubic interpolation is: ; ; ; in, This represents the interpolated pixel coordinates. Let x represent the x-coordinate of the pixel to be interpolated, y represent the y-coordinate of the pixel to be interpolated, and x represent the x-coordinate of the pixel to be interpolated. i This represents the x-coordinate of the neighboring points of the pixel to be interpolated. The ordinate represents the y-coordinate of the neighboring points of the pixel to be interpolated, i represents the row of the neighboring points of the pixel to be interpolated, j represents the column of the neighboring points of the pixel to be interpolated, and a represents the hyperparameter.

[0041] In step S103, the position loss function and the confidence loss function are weighted and summed to construct the total loss function. At the same time, the Adam optimizer is used to update the gradient and find a set of parameters that can minimize the structural risk. The face filtering model is obtained by updating the preset neural network through the total loss function and the Adam optimizer.

[0042] Another aspect of the present invention provides a face recognition method, such as... Figure 5 As shown, steps S201 to S204 are included: S201: Acquire multiple video streams and assign a unique number to each video stream.

[0043] S202: Input each video stream into the video frame filtering model in numerical order, such as... Figure 1 As shown, the video frame filtering model includes a background frame filter, a face filtering model as described in the face filtering model training method, and a duplicate person filter, all connected in sequence. The background frame filter uses differential detection to identify first-class video frames containing moving targets. The first-class video frames are input into the face filtering model, which uses an SSD network to extract predicted feature maps of multiple sizes from the first-class video frames. After interpolation, dimensionality unification, and superposition of one or more predicted feature maps for each first-class video frame, a fused feature map is obtained by inputting it into a linear activation function. Based on the fused feature map, the predicted face positions in the first-class video frames are identified, and video frames without faces are removed to obtain second-class video frames containing predicted face positions. The duplicate person filter uses the DSST algorithm to filter video frames containing duplicate faces in the second-class video frames, retaining only one target video frame for each duplicate face in the output.

[0044] S203: Input the target video frame into the preset face recognition model to obtain the identity information of the target person.

[0045] S204: Mark the target personnel's identity information in each video frame of the corresponding video stream and store it in the database.

[0046] In step S201, multiple video streams captured by different cameras are numbered to facilitate sorting the video streams according to their numbers before sequentially inputting them into the video frame filtering model. Simultaneously, the video stream to which each face and corresponding person's identity information belongs can be determined based on the number.

[0047] In step S202, as Figure 4 As shown, after grayscale processing of each input video stream, it is input into the video frame filtering model. The video frame filtering model randomly selects one video frame as the background video frame for the background frame filter, and the remaining video frames are used as third-class video frames. The background video frame is denoted as... The third type of video frame is denoted as The difference image is obtained by performing a difference operation between the third type of video frame and the background video frame. The formula for calculating the difference image is: ; in, Represents a difference image. This represents the grayscale value of a pixel in a third type of video frame. This represents the grayscale value of a pixel in the background video frame. This represents the coordinates of a pixel in a third type of video frame. This represents the coordinates of a pixel in the background video frame. This represents the coordinates of a pixel in the difference image.

[0048] Set a threshold T, and then process the difference images one by one. The pixels on the image are binarized to obtain a binarized image. .

[0049] ; Difference image Pixels with a grayscale value greater than or equal to T are set to a grayscale value of 255 as foreground pixels, and pixels with a grayscale value less than T are set to a grayscale value of 0 as background pixels, thus obtaining a binarized image. For binarized images Connectivity analysis is performed on the foreground points in the binarized image to determine whether the image composed of the foreground points is continuous. If the image composed of the foreground points is continuous and uninterrupted, the binarized image is output as an image containing the complete moving target. This refers to the first type of video frame containing moving targets.

[0050] In some embodiments, the face filtering model uses an SSD network to extract predicted feature maps of multiple sizes from the first type of video frames. After interpolating, unifying the dimensions, and superimposing one or more predicted feature maps for each first type of video frame, the model inputs a linear activation function to obtain a fused feature map. Based on the fused feature map, the model identifies the predicted face locations in the first type of video frames. After removing video frames without faces from the first type of video frames to obtain the second type of video frames, the model further includes steps S211~S212: S211: During the processing, the first type of video frames are constructed into a differential queue, and the third type of video frames are constructed into a video frame queue for use.

[0051] S212: When the number of first-class video frames in the differential queue is greater than the number of third-class video frames in the video frame queue, update the video frames that do not contain faces detected by the face filtering model to background video frames so as to continue processing the remaining video frames in the video stream.

[0052] In steps S211-S212, when the number of input video frames and output first-type video frames in the background frame filter is equal, and the face filtering model does not output second-type video frames, it indicates that the background frame filter has failed. This failure causes the number of first-type video frames in the difference queue to exceed the number of third-type video frames in the video frame queue, reducing model processing efficiency. When the number of first-type video frames in the difference queue exceeds the number of third-type video frames in the video frame queue, the video frames detected by the face filtering model that do not contain faces are updated to background video frames to ensure the accuracy of the background frames selected by the background frame filter, and the remaining video frames in the sample video stream continue to be processed. This improves the accuracy of the background frame filter in filtering background frames and reduces resource waste caused by video frame accumulation.

[0053] The second type of video frames output by the face filtering model are input into the duplicate people filter. The duplicate people filter uses the DSST algorithm to filter second-type video frames containing identical faces. Based on the face filtering model's face scoring in the second-type video frames, the duplicate people filter outputs a second-type video frame whose face score meets a set threshold. Subsequent second-type video frames containing identical faces are not output. In video surveillance scenarios, a face may appear in the video frame for several seconds, corresponding to more than 100 frames. In reality, face recognition does not need to compare all these video frames; therefore, processing only one video frame containing a face is sufficient to obtain the person information in the remaining video frames containing identical faces. Filtering out video frames containing duplicate faces through the duplicate people filter reduces the workload of the face recognition model and improves its recognition efficiency.

[0054] In step S203, before constructing the target video frames output by the duplicate person filter into a tracking queue and inputting them into the preset face recognition model, the target video frames need to be preprocessed. Since the target video frames output by the video frame filtering model are subject to various limitations and random interference, they cannot be used directly. Therefore, preprocessing of the target video frames is necessary before inputting them into the preset face recognition model. Preprocessing includes: face alignment, face lighting compensation, grayscale transformation, histogram equalization, normalization, geometric correction, median filtering, and sharpening.

[0055] In some embodiments, a preset face recognition model uses the FaceNet network to identify faces in a target video frame and outputs the target person's identity information.

[0056] In some embodiments, a preset face recognition model extracts the facial features of the target person in the target video frame and matches them with the facial features of multiple known identity information persons stored in the database. If the matching degree between the facial features of the target person and the facial features of a known identity information person in the database is higher than the matching degree threshold, then the target person and the known identity information person are considered to be the same person, thereby obtaining the identity information of the target person.

[0057] Facial feature extraction, also known as facial representation, focuses on specific features of the face. It involves modeling facial features. Methods for facial feature extraction include: One is a knowledge-based representation method, which obtains feature data for face classification based on the shape descriptions of facial organs and the distance characteristics between them. Its feature components typically include Euclidean distance, curvature, and angles between feature points. The face is composed of local features such as the eyes, nose, mouth, and chin; the geometric description of these local features and the structural relationships between them is called geometric features. Another method is a representation method based on algebraic features or statistical learning. The basic idea is to transform the high-dimensional description of the face in the spatial domain into a low-dimensional description in the frequency domain or other spaces. This representation method includes linear projection representation and nonlinear projection representation.

[0058] In some embodiments, the duplicate people filter retains one target video frame for duplicate faces and then feeds the result back to the face filtering model, marking duplicate faces in the face filtering model. After the preset face recognition model identifies the target person's identity information, it feeds the identification result back to the duplicate people filter and assigns identity information to faces in the duplicate people filter that match the predicted faces. The preset face recognition model feeds the identification result back to the duplicate people filter, which in turn feeds the filtering result back to the face filtering model. Through this hierarchical feedback, the identity information of the relevant target persons in each video frame containing a face can be directly viewed in the face filtering model.

[0059] In step S204, the target person's identity information is marked in each video frame of the corresponding video stream. This is used to establish a correspondence between each video stream number and the target person's identity information and face position in the corresponding video stream, so as to facilitate the determination of the video stream to which each target person belongs.

[0060] In summary, the training method, face recognition method, and apparatus for the face filtering model described in this invention filter out background video frames from video frames and output first-class video frames containing moving targets through a background frame filter. The face filtering model filters the first-class video frames and outputs second-class video frames containing faces. The duplicate face filtering model filters out video frames containing duplicate faces from the second-class video frames. For the same face, only one target video frame is output. The preset face recognition model identifies the identity information of the people in the target video frames. By filtering step by step, the workload of the preset face recognition model is reduced, while the video frame filtering efficiency of the video frame filtering model is improved.

[0061] Furthermore, by interpolating the predicted feature maps extracted from the predicted feature layer in the SSD network using bicubic interpolation, the facial features in the predicted feature maps are enhanced, preventing some video frames from being filtered out due to unclear faces, and improving the filtering accuracy of the video frame filtering model.

[0062] Furthermore, when the number of first-class video frames in the differential queue is greater than the number of third-class video frames in the video frame queue, the background video frames are updated. Through the feedback mechanism, the background video frames in the background frame filter are adaptively updated, which alleviates the problem of video frame accumulation caused by the different processing speeds of various models, reduces the hardware resource competition between different filters, and improves the recognition efficiency of the face recognition model and the filtering efficiency of the video frame filtering model.

[0063] Corresponding to the above method, the present invention also provides an apparatus comprising a computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and the apparatus performing the steps of the method as described above when the computer instructions are executed by the processor.

[0064] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0065] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0066] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0067] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0068] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A training method for a face filtering model, characterized in that, The method includes the following steps: Obtain a training sample set, which includes multiple samples, each sample including a sample image, add real face location information to each sample image and use the face as a label; A preset neural network is obtained. The preset neural network uses an SSD network to extract prediction feature maps of multiple sizes from the sample image. After interpolation, dimension unification, and superposition of one or more prediction feature maps of each sample image, the data is input into a linear activation function to obtain a fused feature map. Based on the fused feature map, an image containing the predicted face location is identified. The preset neural network is trained using the training sample set. The position loss function and the confidence loss function are weighted and summed to construct the total loss function, which is then used to iteratively update the parameters of the preset neural network to obtain a face filtering model. The preset neural network uses an SSD network to extract predicted feature maps of multiple sizes from the sample images. After interpolation, dimensionality unification, and superposition of one or more predicted feature maps for each sample image, a fused feature map is obtained by inputting it into a linear activation function. Based on the fused feature map, an image containing the predicted face location is identified, including: The sample image is resized to 300×300 and then input into the face filtering model. The SSD network in the face filtering model uses three prediction feature layers, conv4_3, conv5_3 and fc_6, to extract prediction feature maps from the sample image. The first prediction feature map extracted by conv4_3 is obtained by interpolating the pixels using bicubic interpolation and extracting the image features of the first prediction feature map after interpolation using a 3×3×512 convolution kernel. The first feature map is obtained by normalizing the first prediction feature map after interpolation. By deconvolution, the dimension of the second predicted feature map extracted by conv5_3 is made to be consistent with the dimension of the first predicted feature map. Bicubic interpolation is used to interpolate the pixels in the second predicted feature map after deconvolution. A 3×3×512 convolution kernel is used to extract the image features in the second predicted feature map after interpolation. The second predicted feature map is obtained after normalization of the interpolated second predicted feature map. By deconvolution, the dimension of the third predicted feature map extracted by fc_6 is made to be consistent with the dimension of the first predicted feature map. Bicubic interpolation is used to interpolate the pixels in the third predicted feature map after deconvolution. A 3×3×512 convolution kernel is used to extract the image features in the third predicted feature map after interpolation. The third predicted feature map is obtained after normalization of the interpolated third predicted feature map. The features in the first feature map, the second feature map, and the third feature map are superimposed and input into a linear activation function to obtain a first fused feature map. The first fused feature map is convolutionally pooled and output to obtain a second fused feature map containing multiple prediction boxes. Prediction boxes with confidence scores below a threshold and prediction boxes with an overlap rate with the real face location box above a threshold are filtered out, and the image containing the predicted face location is output.

2. A face recognition method, characterized in that, include: Acquire multiple video streams and assign a unique number to each video stream; Each video stream is input into a video frame filtering model in numerical order. The video frame filtering model includes a background frame filter, a face filtering model as described in claim 1, and a duplicate person filter, which are connected in sequence. The background frame filter uses differential detection to identify a first type of video frame containing moving targets. The first type of video frames are input into the face filtering model and filtered to obtain a second type of video frames containing predicted face positions. The duplicate person filter uses the DSST algorithm to filter video frames containing duplicate faces in the second type of video frames, and only one target video frame is output for duplicate faces. The target video frame is input into a preset face recognition model to obtain the identity information of the target person; The identity information of the target personnel is marked in each video frame of the corresponding video stream and stored in the database; The preset face recognition model uses the duplicate person filter to progressively match the target person's identity information to the predicted face location obtained by the face filtering model and labels the identity information.

3. The face recognition method according to claim 2, characterized in that, The background frame filter uses differential detection to identify a first class of video frames containing moving targets, including: One video frame is selected from the video stream as the background video frame, and the rest are designated as third-type video frames. The gray values ​​of the pixels in the third-type video frames are sequentially subtracted from the gray values ​​of the pixels in the background video frames to obtain the difference images corresponding to each third-type video frame. The difference images are then binarized, with pixels whose values ​​are less than a set threshold assigned a value of 0 as background points, and pixels whose values ​​are greater than or equal to the threshold assigned a value of 255 as foreground points. This yields the binarized images of the third-type video frames. After performing connectivity analysis on the foreground points in the binarized images, the first-type video frames containing moving targets are output.

4. The face recognition method according to claim 3, characterized in that, The grayscale values ​​of pixels in the third type of video frame are sequentially subtracted from the grayscale values ​​of pixels in the background video frame to obtain the difference image corresponding to each third type of video frame, including: The formula for calculating the difference image is: ； in, This represents the difference image. This represents the grayscale value of a pixel in the third type of video frame. This represents the grayscale value of a pixel in the background video frame. This represents the coordinates of a pixel in the third type of video frame. This represents the coordinates of a pixel in the background video frame. This represents the coordinates of a pixel in the difference image.

5. The face recognition method according to claim 4, characterized in that, After inputting the first type of video frames into the face filtering model and filtering to obtain the second type of video frames containing the predicted face location, the process further includes: During processing, the first type of video frames are constructed into a differential queue, and the third type of video frames are constructed into a video frame queue for later use. When the number of first-type video frames in the differential queue is greater than the number of third-type video frames in the video frame queue, the video frames that do not contain faces detected by the face filtering model are updated to the background video frames to continue processing the remaining video frames in the sample video stream.

6. The face recognition method according to claim 2, characterized in that, Before the preset face recognition model identifies the target person's identity information by recognizing the target video frame, it also includes: The target video frame is preprocessed, including: face alignment, face lighting compensation, grayscale transformation, histogram equalization, normalization, geometric correction, median filtering, and sharpening.

7. The face recognition method according to claim 6, characterized in that, The preset face recognition model identifies the target person's identity information by recognizing the target video frame, including: The preset face recognition model extracts the facial features of the target person in the target video frame and matches them with the facial features of multiple known identity information persons stored in the database. If the matching degree between the facial features of the target person and the facial features of a known identity information person in the database is higher than the matching degree threshold, then the target person and the known identity information person are considered to be the same person, thereby obtaining the identity information of the target person.

8. A face recognition device, comprising a processor and a memory, characterized in that, The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images