Face mis-detection optimization method and device, storage medium, equipment and program product

Abstract

The application discloses a face mis-detection optimization method and device, a storage medium, equipment and a program product. The method comprises the following steps: decomposing a to-be-tested video to form video image sequence frames arranged in time sequence; performing feature extraction on each frame of video image in the video image sequence frames to obtain face features of each frame of video image, wherein the face features at least comprise left eye feature information, right eye feature information and mouth information; performing feature matching on the face features of each frame of video image and face feature templates in a face feature template library based on a trained matching network model; if the face features are matched with the face feature templates in the face feature template library, it is determined that the face in a face detection result corresponding to a current frame of video image in the video image sequence frames is correct, and the face feature template library is updated according to the face features of the current frame of video image, so that the robustness and stability of face detection can be improved.

Description

Technical Field

[0001] This application relates to the field of digital image processing technology, specifically to a method, apparatus, storage medium, device, and program product for optimizing false face detection. Background Technology

[0002] Face detection refers to distinguishing between human and non-human faces using a model, outputting a binary classification result. False detection occurs when a non-human sample is mistakenly identified as a face. While face detection technology has rapidly advanced with continuous technological advancements, the problem of false detection persists, making complete elimination impossible. This is especially true in video face tracking, where features like beautification, makeup, and stickers are applied, placing extremely high demands on the accuracy of the face. Even a single false detection can result in noticeable abrupt changes in the decoration or face rendering, significantly impacting the user experience. Therefore, face detection in video requires more stable faces, eliminating false detections of face changes during decoration or rendering. Furthermore, while frontal faces clearly show facial features, profile faces, due to their angle, lack some facial information in 2D images. Therefore, profile face detection presents a greater challenge for models than frontal face detection.

[0003] Given the importance of face recognition applications, addressing false detections has become a crucial issue in building a stable face recognition system. However, due to the diversity of scenes, lighting, occlusion, and facial angles, improving the accuracy of face detection models by adding negative samples to overcome false detections is extremely difficult and requires a massive amount of engineering work. It also places higher demands on the model's computing power and memory, which is unfavorable for real-time face recognition applications with speed and memory constraints. Therefore, a fast and accurate method to avoid false detections is essential for current video face recognition systems. Furthermore, since most existing technologies judge based on single-frame images, ignoring the continuity between video frames, false detection errors are easily introduced, reducing accuracy. Summary of the Invention

[0004] This application provides a method, apparatus, storage medium, device, and program product for optimizing false face detection, which can improve the robustness and stability of face detection.

[0005] On one hand, embodiments of this application provide a method for optimizing false face detection. The method includes: decomposing a video to be tested to form a video image sequence frame arranged in chronological order; extracting features from each video image in the video image sequence frame to obtain facial features of each video image frame, wherein the facial features include at least left eye feature information, right eye feature information, and mouth information; performing feature matching between the facial features of each video image frame and facial feature templates in a facial feature template library based on a trained matching network model; if the facial features match the facial feature templates in the facial feature template library, then determining that the face in the face detection result corresponding to the current video image in the video image sequence frame is correct, and updating the facial feature template library according to the facial features of the current video image frame.

[0006] On the other hand, embodiments of this application provide a face false detection optimization device, the device comprising:

[0007] The decomposition unit is used to decompose the video under test into a sequence of video image frames arranged in chronological order.

[0008] The extraction unit is used to extract features from each frame of the video image sequence to obtain the facial features of each frame of the video image. The facial features include at least left eye feature information, right eye feature information and mouth information.

[0009] The matching unit is used to perform feature matching between the facial features of each frame of the video image and the facial feature templates in the facial feature template library based on the trained matching network model.

[0010] The processing unit is configured to determine that the face in the face detection result corresponding to the current frame video image in the video image sequence is correct if the face feature matches a face feature template in the face feature template library, and update the face feature template library according to the face feature of the current frame video image.

[0011] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the face false detection optimization method as described in any of the above embodiments.

[0012] On the other hand, embodiments of this application provide a computer device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the face false detection optimization method as described in any of the above embodiments by calling the computer program stored in the memory.

[0013] On the other hand, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the face false detection optimization method as described in any of the above embodiments.

[0014] This application embodiment decomposes the video to be tested to form a video image sequence frame arranged in chronological order; features are extracted from each video image in the video image sequence frame to obtain the facial features of each video image frame, the facial features including at least left eye feature information, right eye feature information and mouth information; based on a trained matching network model, the facial features of each video image frame are matched with facial feature templates in the facial feature template library; if the facial features match the facial feature templates in the facial feature template library, it is determined that the face in the face detection result corresponding to the current video image frame in the video image sequence frame is correct, and the facial feature template library is updated according to the facial features of the current video image frame. This application embodiment calculates the facial features of each frame in a video image sequence and performs feature matching based on the facial features of each frame with a facial feature template. If a match is found, the template library is updated using the facial features of the current frame. This innovative approach combines video temporal information with an online template library update method, improving the robustness and stability of face detection. Furthermore, by combining and judging each frame in the video image sequence with the accumulated facial information in real time, the face tracking process is monitored in real time to promptly detect incorrectly tracked faces and provide feedback to the face system for timely correction of the face bounding box position. This prevents the accumulation of errors from affecting the display effect of subsequent special effects such as stickers and makeup. It also achieves accurate face filtering with minimal computing power, meeting the requirements of practical business applications. Attached Figure Description

[0015] Figure 1 A flowchart illustrating the face false detection optimization method provided in this application embodiment.

[0016] Figure 2 This is a schematic diagram of a first application scenario for the face false detection optimization method provided in the embodiments of this application.

[0017] Figure 3 This is a schematic diagram of a second application scenario for the face false detection optimization method provided in the embodiments of this application.

[0018] Figure 4 This is a schematic diagram of a third application scenario for the face false detection optimization method provided in the embodiments of this application.

[0019] Figure 5 This is a schematic diagram of the structure of the face false detection optimization device provided in the embodiments of this application.

[0020] Figure 6A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] This application provides a method, apparatus, storage medium, device, and program product for optimizing face false detection. Specifically, the face false detection optimization method of this application can be executed by a computer device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, smart TV, smart speaker, wearable smart device, smart vehicle terminal, etc. The terminal can also include a client, which can be a video client, browser client, instant messaging client, or mini-program, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0023] The embodiments of this application can be applied to various scenarios such as video processing, image processing, data security, artificial intelligence, and intelligent transportation.

[0024] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0025] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0026] Intelligent transportation fully utilizes next-generation information technologies such as the Internet of Things, spatial sensing, cloud computing, and mobile internet across the entire transportation sector. It comprehensively applies theories and tools from transportation science, systems methods, artificial intelligence, and knowledge mining. With the goals of comprehensive perception, deep integration, proactive service, and scientific decision-making, it builds a real-time dynamic information service system, deeply mines transportation-related data, forms problem analysis models, and enhances the industry's ability to optimize resource allocation, improve public decision-making capabilities, enhance industry management capabilities, and improve public service capabilities. This promotes safer, more efficient, more convenient, more economical, more environmentally friendly, and more comfortable operation and development of transportation, and drives the transformation and upgrading of transportation-related industries.

[0027] With the rapid development of technology, face detection technology is also constantly innovating. From traditional feature-based face detection to deep learning methods based on CNNs, the accuracy of face detection has been continuously improving. However, regardless of whether it is based on traditional methods or deep learning methods, false detections will always exist in the face detection process. The occurrence of false detections has a significant impact on face detection applications, especially in complex application scenarios where the probability of false detections is even higher. Currently, many methods improve the accuracy of the face detection model by adding negative samples of false detections to the training set. However, this approach is not a panacea. In various complex and variable scenarios, achieving very high accuracy in face detection requires increased computation and memory consumption, which puts pressure on the computing power and speed of real-time applications.

[0028] This paper presents a face false detection optimization method based on a depth camera. The method first requires obtaining color and depth images of the same scene. Face detection and facial landmark localization are performed on the color image first. Then, the depth image and multiple face detection modules are combined to determine whether the initial face region in the face image is a false detection. This accelerates detection efficiency, improves the accuracy of facial landmark localization, and avoids the error accumulation caused by judging each landmark individually, thus reducing the false detection rate. However, this method uses both color and depth images to determine whether a face region is a false detection. Acquiring depth images typically requires specific depth acquisition equipment, which is unavailable in most everyday applications. Therefore, this method cannot be universally applied across multiple scenarios.

[0029] A related technology provides a method for filtering false face detections. This method includes a facial key point detection model that detects key points in a color image, such as the center points of the left and right eyes, the center point of the nose, and the left and right corners of the mouth. It then uses the absolute distance relationships between these key points and a preset threshold to determine whether the detected face in the color image is a false face. This method utilizes the positions of key points on key facial features and, based on prior knowledge of the distribution of facial features, uses the distances between the detected key points to determine whether the face conforms to the prior distribution. This method is simple in concept, easy to operate, and very fast, without needing to consider additional power consumption. However, due to various changes in facial angles and expressions, the distribution of facial features changes, and the constraint distances between facial features also change significantly. Therefore, using only the distances between facial features to determine whether a face is a false face is inaccurate when the facial angle and expression change.

[0030] This paper presents a method for optimizing face false detection. The method first uses a face detector to pre-detect faces, then uses a facial landmark algorithm to locate key points on the face, and extracts depth features from these key point locations to form new features. These features are then used to train a binary classifier to determine whether a detected face is a false detection. This method prioritizes locating facial landmarks on the face using the facial landmark algorithm, extracts corresponding depth features based on the key point locations, and combines these features to train a binary classification model for false face detection. This method is relatively stable and can fully utilize the high-level semantic information of the face to determine whether a face behind a wall is a correct face. However, when dealing with profile views, half of the face is invisible, making the extracted depth features from this portion invalid and thus interfering with the model.

[0031] One method for reducing false detection rates in face detection is provided by related technologies. This method first acquires the current frame image and the previous frame image. For the face region detected in the current frame, the corresponding cropped region from the previous frame at the same coordinates is cropped. Using the frame difference relationship, moving objects are selected from multiple target blocks in the current frame image and correctly identified as faces, while static objects are identified as false background images. This significantly reduces the interference caused by environmental diversity and lowers the false detection rate. While this method utilizes the relationship between consecutive frames and uses frame difference to remove false backgrounds, the frame difference method is extremely sensitive to external environments such as lighting changes and background movement, making it prone to misjudgments and exhibiting poor robustness. Furthermore, this method has stringent requirements for the scene, necessitating a fixed camera, which is unattainable in most everyday applications, thus limiting its versatility.

[0032] In view of the problems existing in the above-mentioned related technologies, this application provides a method, device, storage medium, equipment and program product for optimizing face false detection in video image sequences. The method involves decomposing the video to be tested to form a video image sequence frame arranged in chronological order, and extracting features from each frame of the video image sequence frame to obtain the face features of each frame. The face features include at least left eye feature information, right eye feature information and mouth information. Then, based on a trained matching network model, the face features of each frame of the video image are matched with face feature templates in the face feature template library. If the face features match the face feature templates in the face feature template library, it is determined that the face in the face detection result corresponding to the current frame of the video image sequence frame is correct, and the face feature template library is updated according to the face features of the current frame of the video image sequence frame. This application embodiment calculates the facial features of each frame in a video image sequence and performs feature matching based on the facial features of each frame with a facial feature template. If a match is found, the template library is updated using the facial features of the current frame. This innovative approach combines video temporal information with an online template library update method, improving the robustness and stability of face detection. Furthermore, by combining and judging each frame in the video image sequence with the accumulated facial information in real time, the face tracking process is monitored in real time to promptly detect incorrectly tracked faces and provide feedback to the face system for timely correction of the face bounding box position. This prevents the accumulation of errors from affecting the display effect of subsequent special effects such as stickers and makeup. It also achieves accurate face filtering with minimal computing power, meeting the requirements of practical business applications.

[0033] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0034] This application provides a method for optimizing false face detection. This method can be executed by a terminal or a server, or by both a terminal and a server. This application uses the example of the method being executed by a server to illustrate the optimization method for false face detection.

[0035] Please see Figures 1 to 4 , Figure 1 This is a flowchart illustrating the face false detection optimization method provided in an embodiment of this application. Figures 2 to 4 This is a schematic diagram illustrating an application scenario for the face false detection optimization method provided in this application embodiment. The method may include the following steps 110 to 140:

[0036] Step 110: Decompose the video to be tested to form a video image sequence frame arranged in chronological order.

[0037] For example, the video to be tested contains images of faces. The video is broken down into multiple frames in chronological order to form a video image sequence. This method ensures the reasonableness of the video decomposition.

[0038] Step 120: Extract features from each frame of the video image sequence to obtain the facial features of each frame of the video image. The facial features include at least left eye feature information, right eye feature information, and mouth information.

[0039] In some embodiments, step 120 can be implemented by the following steps 121 to 124 (not shown in the figure), specifically:

[0040] Step 121: Perform facial landmark detection on each frame of the video image sequence to obtain the facial landmarks of each frame of the video image. The facial landmarks include at least the left eye landmark, the right eye landmark, and the mouth landmark.

[0041] For example, for each frame in the input video image sequence, facial landmark tracking is first performed on each frame to calculate the facial landmarks. Based on the obtained facial landmarks, the left eye landmark, right eye landmark, and mouth landmark are extracted from each video image. For example, as... Figure 2 As shown, taking 106 facial key points as an example, key points are extracted from the left eye, right eye, and mouth areas to obtain the key points of the left eye, right eye, and mouth.

[0042] Step 122: Determine the bounding rectangle of the face region in each frame of the video image based on the facial key points. The bounding rectangle of the face region includes at least the bounding rectangle of the left eye, the bounding rectangle of the right eye, and the bounding rectangle of the mouth.

[0043] For example, such as Figure 2 As shown, the bounding rectangle 10 of the face region in each frame of the video image is determined based on the facial key points. The bounding rectangle 10 of the face region includes at least the bounding rectangle 11 of the left eye, the bounding rectangle 12 of the right eye, and the bounding rectangle 13 of the mouth. For example, the bounding rectangle of each region block is calculated based on the coordinates corresponding to the key points of the left eye, the right eye, and the mouth. For example, the bounding rectangle of the face region can be expressed as the following formula (1):

[0044] min_x=min(x i (i = 1, 2Λn)

[0045] max_x=max(x i (i = 1, 2Λn)

[0046] min_y = min(y i (i = 1, 2Λn)

[0047] max_y = max(y i ,i=1,2Λn)(1);

[0048] The coordinates of the bounding rectangle of the face region are encoded as [min_x,min_y,max_x,max_y]. min_x and min_y represent the x-axis and y-axis coordinates of the upper left corner of the bounding rectangle of the face region, respectively, while max_x and max_y represent the x-axis and y-axis coordinates of the lower right corner of the bounding rectangle of the face region.

[0049] For example, based on the minimum x-axis coordinate among all coordinates of the left eye keypoints, determine the x-axis coordinate of the top-left corner of the left eye's bounding rectangle; based on the minimum y-axis coordinate among all coordinates of the left eye keypoints, determine the y-axis coordinate of the top-left corner of the left eye's bounding rectangle; based on the maximum x-axis coordinate among all coordinates of the left eye keypoints, determine the x-axis coordinate of the bottom-right corner of the left eye's bounding rectangle; based on the maximum y-axis coordinate among all coordinates of the left eye keypoints, determine the y-axis coordinate of the bottom-right corner of the left eye's bounding rectangle.

[0050] For example, based on the minimum x-axis coordinate among all coordinates of the right eye keypoints, determine the x-axis coordinate of the top-left corner of the right eye's circumscribed rectangle; based on the minimum y-axis coordinate among all coordinates of the right eye keypoints, determine the y-axis coordinate of the top-left corner of the right eye's circumscribed rectangle; based on the maximum x-axis coordinate among all coordinates of the right eye keypoints, determine the x-axis coordinate of the bottom-right corner of the right eye's circumscribed rectangle; based on the maximum y-axis coordinate among all coordinates of the right eye keypoints, determine the y-axis coordinate of the bottom-right corner of the right eye's circumscribed rectangle.

[0051] For example, based on the minimum x-axis coordinate among all the coordinates of the mouth key points, determine the x-axis coordinate of the top-left corner of the bounding rectangle of the mouth; based on the minimum y-axis coordinate among all the coordinates of the mouth key points, determine the y-axis coordinate of the top-left corner of the bounding rectangle of the mouth; based on the maximum x-axis coordinate among all the coordinates of the mouth key points, determine the x-axis coordinate of the bottom-right corner of the bounding rectangle of the mouth; based on the maximum y-axis coordinate among all the coordinates of the mouth key points, determine the y-axis coordinate of the bottom-right corner of the bounding rectangle of the mouth.

[0052] Step 123: Crop each frame of the video image according to the bounding rectangle of the face region to obtain the face region image corresponding to each frame of the video image. The face region image includes at least the left eye image, the right eye image, and the mouth image.

[0053] For example, after determining the outer bounding rectangle 11 for the left eye, the outer bounding rectangle 12 for the right eye, and the outer bounding rectangle 13 for the mouth in each frame of video image, cropping can be performed based on the outer bounding rectangle 11 for the left eye, the outer bounding rectangle 12 for the right eye, and the outer bounding rectangle 13 for the mouth to obtain the face region image corresponding to each frame of video image. The face region image includes at least the left eye image, the right eye image, and the mouth image.

[0054] In some embodiments, before cropping each frame of the video image according to the bounding rectangle of the face region, the method further includes: expanding the region according to a preset multiple of the width and height of the bounding rectangle of the face region to obtain an expanded bounding rectangle of the region, wherein the expanded bounding rectangle of the region is used to indicate the cropping region of each frame of the video image.

[0055] The step of cropping each frame of the video image according to the bounding rectangle of the face region to obtain the face region image corresponding to each frame of the video image includes: cropping each frame of the video image according to the expanded bounding rectangle of the region to obtain the face region image corresponding to each frame of the video image.

[0056] For example, to prevent regional errors caused by inaccurate facial key points, when cropping regions, the bounding boxes of each region can be expanded to a certain extent according to preset multiples of the width and height of the bounding box of each region. For example, the preset multiple corresponding to the upper left corner can be 0.1 times, and the preset multiple corresponding to the lower right corner can be 1.2 times. For example, the expanded bounding box of the region can be expressed as the following formula (2):

[0057]

[0058] Wherein, `width` represents the width of the bounding rectangle of the original face region, determined by the difference between the x-coordinates of the bottom-right and top-left corners of the original bounding rectangle; `height` represents the height of the bounding rectangle of the original bounding rectangle, determined by the difference between the y-coordinates of the bottom-right and top-left corners of the original bounding rectangle. The coordinate encoding of the bounding rectangle of the expanded face region is [new_min_x, new_min_y, new_max_x, new_max_y], where `new_min_x` and `new_min_y` represent the x-coordinates of the top-left corner and the y-coordinates of the expanded face region bounding rectangle, respectively, and `new_max_x` and `new_max_y` represent the x-coordinates of the bottom-right corner and the y-coordinates of the expanded face region bounding rectangle.

[0059] For example, by reducing the x-axis coordinate of the top-left corner of the bounding rectangle of the face region before expansion by 0.1 times its original width, and by reducing the y-axis coordinate of the top-left corner of the bounding rectangle of the face region before expansion by 0.1 times its original height, we can obtain the x-axis and y-axis coordinates of the top-left corner of the bounding rectangle of the face region after expansion, which are new_min_x and new_min_y.

[0060] For example, by increasing the x-axis coordinate of the bottom right corner of the bounding rectangle of the face region before expansion by 1.2 times its original width, and by increasing the y-axis coordinate of the bottom right corner of the bounding rectangle of the face region before expansion by 1.2 times its original height, we can obtain the x-axis and y-axis coordinates of the bottom right corner of the bounding rectangle of the face region after expansion, which are new_max_x and new_max_y.

[0061] From this, three new bounding boxes for the left eye, right eye, and mouth can be calculated, where box = [new_min_x, new_min_y, new_max_x, new_max_y]. The regions corresponding to these three bounding boxes are then cropped to obtain the face region image for each frame of the video image. This face region image includes at least the left eye image, the right eye image, and the mouth image.

[0062] Step 124: Extract features from the face region image corresponding to each frame of the video image to obtain the face features of each frame of the video image.

[0063] For example, the left eye image, right eye image, and mouth image corresponding to the three regions cropped in step 124 are used to extract facial features to obtain the facial features of each frame of video image.

[0064] For example, the purpose of extracting features from the left eye image, right eye image, and mouth image is because the black and white areas of the eyes and the red areas of the lips are the most prominent parts of a face image. If there are false positives, such as only skin or ears, or if the face detection is distorted due to incorrect prediction of facial landmarks, these false positives will not contain both the eye and mouth areas. Based on this purpose and consideration, in this embodiment, the three key regions corresponding to the left eye image, right eye image, and mouth image obtained above are used as feature extraction regions to represent the facial feature information of this video image.

[0065] In some embodiments, the step of extracting features from the face region image corresponding to each frame of the video image to obtain the face features of each frame of the video image includes: obtaining the color histogram features of the face region image corresponding to each frame of the video image, wherein the color histogram features include a first color histogram feature corresponding to the left eye image, a second color histogram feature corresponding to the right eye image, and a third color histogram feature corresponding to the mouth image; and concatenating the first color histogram feature, the second color histogram feature, and the third color histogram feature corresponding to each frame of the video image to obtain the face features of each frame of the video image.

[0066] For example, when extracting features, considering that face applications typically need to run in real-time on mobile devices, false face detection can only be a very small part of the entire face system. Therefore, the computational load involved must be small and the calculation speed must be fast. Thus, the face features used for false face detection cannot be overly complex, to avoid increasing the computational load excessively and affecting the overall speed of the face system. Meanwhile, the above... Figure 2 The most obvious feature among the three candidate feature regions (left eye image, right eye image, and mouth image) is color. Therefore, based on the above two considerations, the color histogram feature can be used as the face feature of the face region image in this embodiment of the application.

[0067] In some embodiments, obtaining the color histogram features of the face region image corresponding to each frame of the video image includes: uniformly dividing each color channel in the color space of each image in the face region image corresponding to each frame of the video image into n sub-intervals within the grayscale value range of [0, 255], wherein each sub-interval corresponds to an interval scale in the color histogram; calculating the number of pixels of each color channel in each image in the face region image corresponding to each frame of the video image in each sub-interval, and determining the number of pixels as the feature value of the corresponding interval scale in the color histogram; normalizing the feature value of the corresponding interval scale in the color histogram to the range of [0, 1] to obtain the color histogram features of each image in the face region image corresponding to each frame of the video image.

[0068] For example, in the RGB color space, the values ​​of each color channel in each image of a face region are distributed from 0 to 255. However, the human eye's ability to distinguish colors is limited. In the color space, when the distance between two colors is less than a certain value, the human eye cannot distinguish the difference between them and considers them to be the same color. Therefore, color gradation can be performed. On the one hand, it reduces the difficulty for the human eye to distinguish colors, and on the other hand, it reduces the dimensionality of features, thus reducing the computational load for subsequent calculations. In the embodiments of this application, the grayscale value range is evenly divided into n sub-ranges, for example, 16 sub-ranges. Every 16 color values ​​are gradually divided into one sub-range, such as [0,15], [16,31], ..., [240,255]. Thus, in the RGB space, a total of 48 ranges can be divided into 16*3 ranges. Then, for the color gradation from... Figure 2 The regions of the left eye image, right eye image, and mouth image obtained by cropping are used to calculate the number of pixels in each color channel of each image in each sub-interval. The number of pixels is determined as the feature value of the corresponding interval scale in the color histogram. The histogram result (the feature value of the corresponding interval scale in the color histogram) is normalized to the range [0,1] to obtain the color histogram feature of each image in the face region image corresponding to each frame of video image. For example, the formed histogram is as follows: Figure 3 The content shown is as follows. In a single image, an n-by-3 dimensional color histogram feature vector can be obtained. Through repeated calculations, features are extracted from each image sequentially, and each n-by-3 dimensional color histogram feature vector is concatenated in RGB color channel order to form a 3n-by-1 dimensional long vector. The color histogram features of each image (left eye image, right eye image, mouth image) are statistically calculated. Then, the three 3n color histogram features are concatenated to obtain the facial features of each frame of the video image.

[0069] For example, the distribution of regions corresponding to the left eye image, right eye image, and mouth image is analyzed as follows: Figure 3 The histogram calculation shown yields three 48-dimensional histogram feature vectors. These three feature vectors are then concatenated to form a 1*144-dimensional feature vector. This 1*144-dimensional feature vector is used as the facial feature of the currently processed video image. This completes the facial feature extraction process.

[0070] Step 130: Based on the trained matching network model, perform feature matching between the facial features of each frame of the video image and the facial feature templates in the facial feature template library.

[0071] In some embodiments, before performing feature matching between the facial features of each frame of the video image and the facial feature templates in the facial feature template library based on the trained matching network, the method further includes: constructing a facial feature template library based on the facial features of the first N frames of the video image sequence, wherein the facial feature template library contains facial feature templates corresponding to each face in the video image sequence, and N≥3.

[0072] For example, for ease of description, the facial feature template of a single face is denoted as φ, which is also a 1*144 dimensional feature vector; while the facial feature vector extracted from the video image is denoted as F, which is also a 1*144 dimensional feature vector.

[0073] For example, if N is set to 3, in order to maintain sufficient stability in face false detection when constructing the initial face feature template library, it is assumed that the first 3 video images detected by the face system are correct faces, and the face features of the first 3 video images can be directly added to the face feature template library for template update. Assuming that the face features extracted from the first 3 video images are denoted as F0, F1, and F2 respectively, the face feature template can be constructed using the following formula (3):

[0074] φ=a0*F0+a1*F1+a2*F2(3);

[0075] Where a0, a1, and a2 represent the weights of the facial features in each frame view image, such as 0.2, 0.3, and 0.5.

[0076] For example, N can be set to be greater than 3. If N is set to 5, the first 5 video images detected by the face recognition system are considered to be correct faces by default. The facial features of the first 3 video images can be directly added to the face feature template library for template updates. Assume that the facial features extracted from the first 3 video images are denoted as F0, F1, F2, F3, and F4, respectively, and the weights of the facial features of each frame view image are a0, a1, a2, a3, and a4, respectively. The corresponding face feature template calculation method can refer to formula (3), which is to first calculate the product of the facial features and weights of each frame view image, and then add the products to obtain the face feature template.

[0077] For example, due to differences in skin color, clothing, accessories, etc., the facial feature template library for a single face will have a large error when applied to different people. Therefore, a facial feature template is configured independently for each face in the video image. The lifespan of a facial feature template for a single face only exists when the face is retained in the video image. Once the face goes out of bounds, the facial feature template corresponding to the out-of-bounds face will be released.

[0078] In some embodiments, the method further includes: obtaining facial features of the current frame video image in the video image sequence; if the similarity value between the facial features of the current frame video image and the facial feature templates in the facial feature template library is greater than a first threshold and less than a second threshold, then reconstructing the facial feature template library based on the facial features of the current frame video image and the facial features of the previous two frames of video images.

[0079] For example, in practical applications, when a face undergoes significant changes, such as drastic changes in lighting or turning the head to form a profile, the features of the current frame view image are significantly different from those of a previous period. If the face feature template library still retains the feature information of the original frame, it will be very unfriendly to the judgment in the current changing environment. In view of this consideration, the entire face false detection filtering system will simultaneously retain the face feature information of the current frame video image and the two frames of video images before the current frame, for a total of 3 frames. During the above template matching process, if the similarity value between the face features of the current frame video image and the corresponding face feature template in the face feature template library is greater than the first threshold and less than the second threshold, it can be considered that there is a possibility of feature distance change. At this time, it is necessary to recalculate the face feature template φ corresponding to the face contained in the current frame video image in the face feature template library, as shown in formula (4):

[0080] φ=a0*F t-2 +a1*F t-1 +a2*F t (4);

[0081] Where a0, a1, and a2 represent the weights of the facial features in each frame view image; F t F represents the facial features of the current frame of the video image. t-1 F represents the facial features of the previous frame of the video image in the current frame. t-2 This represents the facial features of the two previous video frames of the current frame.

[0082] For example, a facial feature template library can be reconstructed based on the facial features of the current frame video image and the facial features of the previous M frames. M can be set to a value greater than 2. If M is set to 4, assuming the facial feature of the current frame video image is F... t The facial features extracted from the first four frames of the video image are denoted as F. t-1 F t-2 F t-3 F t-4 The weights of the facial features in each frame view image are a0, a1, a2, a3, and a4, respectively. The corresponding facial feature template can be calculated using formula (4), which involves first calculating the product of the facial features and weights in each frame view image, and then adding the products together to obtain the facial feature template.

[0083] In some embodiments, the method further includes: if the similarity value between the facial features of the current frame video image and the facial feature templates in the facial feature template library is greater than the second threshold, then updating the facial feature template library based on the facial features of the current frame video image.

[0084] For example, if the similarity value between the facial features of the current frame video image and the corresponding facial feature template in the facial feature template library is greater than the second threshold, it indicates that the matching result between the facial features of the current frame video image and the corresponding facial feature template in the facial feature template library is sufficiently similar. Therefore, the facial features of the current frame video image are used to update the corresponding facial feature template in the facial feature template library online, and this, combined with continuous frame information, improves the robustness of the model in judging false facial detections. For example, the update formula is expressed as the following formula (5):

[0085] φ'=b0*φ+b1*F t (5);

[0086] Where b0 represents the weight corresponding to the original face feature template φ, and b1 represents the face feature F of the current frame video image. t The corresponding weights, for example, b0 = 0.9, b1 = 0.1.

[0087] For example, given an initial face feature template library and face feature vectors extracted from each frame of a video image sequence, it is necessary to perform feature matching between the face feature vector of each frame and the face feature templates retained in the face feature template library to determine whether they match. That is, to determine whether the face feature vector of each frame is similar to the face feature templates retained in the face feature template library. If they match, it means that the face in the face detection result corresponding to the current frame of the video image sequence is the correct face. Then, the face feature template library can be updated using the above formula (5) to ensure the effectiveness and timeliness of the face feature template library. If they do not match, it means that the face predicted in the current frame of the video image has been falsely detected.

[0088] In some embodiments, before performing feature matching between the facial features of each frame of the video image and the facial feature templates in the facial feature template library based on the trained matching network model, the method further includes: obtaining a training sample set, the training sample set including facial feature samples of sample video images and facial feature template samples in the facial feature template sample library; and training the matching network model based on the training sample set to obtain a trained matching network model.

[0089] For example, in order to ensure the accuracy and robustness of the matching, this application uses an ultra-lightweight neural network as the matching network model for matching degree training. Compared with the traditional method of calculating the matching degree by distance, the matching degree predicted by the ultra-lightweight neural network as the matching network model has stronger robustness to changes in lighting, image quality, etc.

[0090] For example, such as Figure 4 As shown, assuming Figure 4 Feature 1 in the image represents a facial feature sample extracted from the sample video image. Assuming... Figure 4 Feature 2 in the image represents the face feature template sample corresponding to the sample video image. The extracted feature 1 (face feature sample) and feature 2 (corresponding face feature template sample) are concatenated into a 2-channel feature map. The feature map is fed into a matching network model composed of convolutional neural network modules, and a similarity value (similarity score) between [0,1] is output. This score is used to judge the matching degree between feature 1 and feature 2. Usually, the first threshold can be set to 0.5. If the output similarity value is less than the first threshold, it is considered that feature 1 and feature 2 are not similar. If the output similarity value is greater than the first threshold, it is considered that feature 1 and feature 2 are similar. At the same time, a second threshold can also be set. For example, the second threshold can be set to 0.8. When the output similarity value is less than the second threshold (e.g., 0.8), it is considered that there is a risk of change. The face feature template library can be recalculated and updated using the above formula (4) to ensure that the model can robustly identify false faces when changes are caused by the environment, etc.

[0091] In some embodiments, obtaining the training sample set includes: creating sample video images based on multiple sample video image sequence frames, the sample video images including positive sample video images and negative sample video images, wherein each positive sample video image has a corresponding positive facial feature sample, and each negative sample video image has a corresponding negative facial feature sample; creating facial feature template samples from a facial feature template sample library based on the multiple sample video image sequence frames; concatenating the positive facial feature samples of the positive sample video images with the facial feature template samples to obtain a first concatenated feature map, and labeling the first concatenated feature map with a first similarity label to obtain a first training sample pair; concatenating the negative facial feature samples of the negative sample video images with the facial feature template samples to obtain a second concatenated feature map, and labeling the first concatenated feature map with a second similarity label to obtain a second training sample pair; and obtaining a training sample set based on the first training sample pair and the second training sample pair.

[0092] In some embodiments, training the matching network model based on the training sample set to obtain a trained matching network model includes: inputting the first concatenated feature map from the training sample set into the matching network model to obtain a first predicted matching result, and training the matching network model based on the first predicted matching result; inputting the first concatenated feature map from the training sample set into the matching network model to obtain a second predicted matching result, and training the matching network model based on the second predicted matching result; and ending the training of the matching network model when the first predicted matching result matches the first similarity label and the second predicted matching result matches the second similarity label.

[0093] In some embodiments, the step of inputting the first spliced ​​feature map from the training sample set into the matching network model to obtain a first predicted matching result, and training the matching network model based on the first predicted matching result, includes: inputting the first spliced ​​feature map from the training sample set into the matching network model for similarity matching, so as to output a first similarity value between the positive face feature sample and the face feature template sample in the first spliced ​​feature map; if the first similarity value is less than a first threshold, the obtained first predicted matching result is predicted to be that the positive face feature sample and the face feature template sample are not similar, and it is determined that the first predicted matching result does not match the first similarity label; if it is determined that the first predicted matching result does not match the first similarity label, the face feature template sample library is updated again, and the first spliced ​​feature map is updated according to the updated face feature template sample library, and the step of inputting the first spliced ​​feature map from the training sample set into the matching network model for similarity matching is returned, until the first similarity value is greater than or equal to the first threshold, the obtained first predicted matching result is predicted to be that the positive face feature sample and the face feature template sample are similar, so that the first predicted matching result matches the first similarity label.

[0094] In some embodiments, the method further includes: if the first similarity value is greater than the first threshold and less than the second threshold, then updating the face feature template sample library again, updating the first spliced ​​feature map according to the updated face feature template sample library again, and returning to the step of inputting the first spliced ​​feature map in the training sample set into the matching network model for similarity matching, until the first similarity value is greater than or equal to the second threshold, and ending the training of the matching network model.

[0095] In some embodiments, the step of inputting the first spliced ​​feature map from the training sample set into the matching network model to obtain a second predicted matching result, and training the matching network model based on the second predicted matching result, includes inputting the second spliced ​​feature map from the training sample set into the matching network model for similarity matching, so as to output a second similarity value between the negative face feature sample and the face feature template sample in the second spliced ​​feature map; if the second similarity value is greater than or equal to a first threshold, the obtained second predicted matching result is predicted to be similar between the negative face feature sample and the face feature template sample, and it is determined that the second predicted matching result does not match the second similarity label; if it is determined that the second predicted matching result does not match the second similarity label, the face feature template sample library is updated again, and the second spliced ​​feature map is updated according to the updated face feature template sample library, and the step of inputting the second spliced ​​feature map from the training sample set into the matching network model for similarity matching is returned, until the second similarity value is less than the first threshold, the obtained second predicted matching result is predicted to be dissimilar between the negative face feature sample and the face feature template sample, so that the second predicted matching result matches the second similarity label.

[0096] During the training process, the training sample set needs to be created using multiple video image sequence frames. Figure 4 Feature 1 in the diagram represents the facial feature samples extracted from the sample video images. Here, both positive and negative sample video images need to be extracted simultaneously. The positive sample video image corresponds to the facial features of the three facial regions extracted under the normal and accurate predicted facial key point samples. The negative sample video image corresponds to the facial features of the three non-face regions extracted when the facial key point prediction fails. Here, to simulate prediction failure, one part can be obtained by randomly and significantly shifting the face bounding box to obtain the face region for facial key point prediction, and the face region extracted in this way can be used as the negative facial feature sample. Another part can be obtained by randomly cropping the image bounding box on the background image to simulate false detection face bounding boxes, and then extracting the features of the above three regions after performing facial key point prediction, which can be used as the negative facial feature sample.

[0097] Assumption Figure 4 Feature 2 in the sample video image represents the face feature template sample corresponding to the sample video image. The creation of this face feature template sample requires that all face features are extracted from the correct face key points of the three face regions. Here, feature 2 can be generated by using the face features of adjacent random n frames using formula (3), formula (4), or formula (4) to simulate the generation and updating process of the face feature template library in actual applications.

[0098] Positive face feature samples (feature 1 positive samples), negative face feature samples (feature 1 negative samples), and face feature template samples (feature 2 positive samples) are generated in the above manner. The feature 1 and feature 2 samples are then randomly combined to form... Figure 4 The input feature maps are arranged in two ways. The first concatenated feature map consists of positive face feature samples (feature 1 positive samples) and face feature template samples (feature 2 positive samples), with a first similarity label of label=1. The second concatenated feature map consists of negative face feature samples (feature 1 negative samples) and face feature template samples (feature 2 positive samples), with a second similarity label of label=0. Training sample pairs of input features and labels are formed in this way. Figure 4 The matching network model is trained for classification. Then, the trained matching network model can be used to predict the matching degree between facial features and templates.

[0099] Step 140: If the facial features match the facial feature templates in the facial feature template library, then the face in the face detection result corresponding to the current frame video image in the video image sequence is determined to be correct, and the facial feature template library is updated according to the facial features of the current frame video image.

[0100] In some embodiments, the method further includes: if the face feature does not match the face feature template in the face feature template library, then determining that the face in the face detection result corresponding to the current frame video image in the video image sequence is a false detection.

[0101] In some embodiments, the method further includes: feeding back the falsely detected face detection results to correct the bounding rectangle of the face region based on the falsely detected face detection results.

[0102] For example, given an initial face feature template library and face feature vectors extracted from each frame of a video image sequence, it is necessary to perform feature matching between the face feature vector of each frame and the face feature templates retained in the face feature template library to determine whether they match. That is, to determine whether the face feature vector of each frame is similar to the face feature templates retained in the face feature template library. If they match, it means that the face in the face detection result corresponding to the current frame of the video image sequence is the correct face. Then, the face feature template library can be updated using the above formula (5) to ensure the effectiveness and timeliness of the face feature template library. If they do not match, it means that the face predicted in the current frame of the video image has been falsely detected.

[0103] This application proposes a fast and effective optimization method for face false detection. It innovatively employs a new template update strategy to combine and judge each frame in the video image sequence with the accumulated face information in real time. This allows for real-time monitoring during face tracking, timely detection of incorrectly tracked faces, and feedback to the face system for timely correction of face predictions. The method proposed in this application is simple to operate, computationally inefficient, and does not affect the speed and performance of the face system. Simultaneously, it robustly monitors face results in real time, preventing the accumulation of errors between frames from affecting subsequent applications of face technology, such as the display effects of stickers and makeup. This application proposes a fast and effective video face false detection filtering method with extremely low computational cost, ensuring the stability of the face system. The proposed new face feature representation method is not only computationally inefficient but also highly representative, providing effective feature representation for accurate face false detection judgment. The proposed new feature template matching algorithm can quickly and accurately improve matching results and has good generalization ability to the effects of lighting and deformation.

[0104] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0105] This application embodiment decomposes the video to be tested to form a video image sequence frame arranged in chronological order; features are extracted from each video image in the video image sequence frame to obtain the facial features of each video image frame, the facial features including at least left eye feature information, right eye feature information and mouth information; based on a trained matching network model, the facial features of each video image frame are matched with facial feature templates in the facial feature template library; if the facial features match the facial feature templates in the facial feature template library, it is determined that the face in the face detection result corresponding to the current video image frame in the video image sequence frame is correct, and the facial feature template library is updated according to the facial features of the current video image frame. This application embodiment calculates the facial features of each frame in a video image sequence and performs feature matching based on the facial features of each frame with a facial feature template. If a match is found, the template library is updated using the facial features of the current frame. This innovative approach combines video temporal information with an online template library update method, improving the robustness and stability of face detection. Furthermore, by combining and judging each frame in the video image sequence with the accumulated facial information in real time, the face tracking process is monitored in real time to promptly detect incorrectly tracked faces and provide feedback to the face system for timely correction of the face bounding box position. This prevents the accumulation of errors from affecting the display effect of subsequent special effects such as stickers and makeup. It also achieves accurate face filtering with minimal computing power, meeting the requirements of practical business applications.

[0106] To facilitate better implementation of the face false detection optimization method of this application, this application also provides a face false detection optimization device. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of the structure of the face false detection optimization device provided in an embodiment of this application. The face false detection optimization device 200 may include:

[0107] The decomposition unit 210 is used to decompose the video under test to form a video image sequence frame arranged in chronological order.

[0108] The extraction unit 220 is used to extract features from each frame of the video image sequence to obtain the facial features of each frame of the video image. The facial features include at least left eye feature information, right eye feature information and mouth information.

[0109] Matching unit 230 is used to perform feature matching between the facial features of each frame of the video image and the facial feature templates in the facial feature template library based on the trained matching network model;

[0110] The processing unit 240 is configured to determine that the face in the face detection result corresponding to the current frame video image in the video image sequence is correct if the face feature matches a face feature template in the face feature template library, and update the face feature template library according to the face feature of the current frame video image.

[0111] In some embodiments, the extraction unit 220 is specifically configured to: perform facial keypoint detection on each frame of the video image sequence to obtain facial keypoints for each frame of the video image, wherein the facial keypoints include at least a left eye keypoint, a right eye keypoint, and a mouth keypoint; determine a bounding rectangle of the face region for each frame of the video image based on the facial keypoints, wherein the bounding rectangle of the face region includes at least a bounding rectangle of the left eye, a bounding rectangle of the right eye, and a bounding rectangle of the mouth; crop each frame of the video image based on the bounding rectangle of the face region to obtain a face region image corresponding to each frame of the video image, wherein the face region image includes at least a left eye image, a right eye image, and a mouth image; and extract features from the face region image corresponding to each frame of the video image to obtain the face features of each frame of the video image.

[0112] In some embodiments, when the extraction unit 220 extracts features from the face region image corresponding to each frame of the video image to obtain the face features of each frame of the video image, it can be used to: obtain the color histogram features of the face region image corresponding to each frame of the video image, wherein the color histogram features include a first color histogram feature corresponding to the left eye image, a second color histogram feature corresponding to the right eye image, and a third color histogram feature corresponding to the mouth image; and concatenate the first color histogram feature, the second color histogram feature, and the third color histogram feature corresponding to each frame of the video image to obtain the face features of each frame of the video image.

[0113] In some embodiments, when the extraction unit 220 obtains the color histogram features of the face region image corresponding to each frame of the video image, it can be used to: uniformly divide each color channel in the color space of each image in the face region image corresponding to each frame of the video image into n sub-intervals within the gray value range of [0, 255], wherein each sub-interval corresponds to an interval scale in the color histogram; calculate the number of pixels of each color channel in each image in the face region image corresponding to each frame of the video image in each sub-interval, and determine the number of pixels as the feature value of the corresponding interval scale in the color histogram; normalize the feature value of the corresponding interval scale in the color histogram to the range of [0, 1] to obtain the color histogram features of each image in the face region image corresponding to each frame of the video image.

[0114] In some embodiments, before cropping each frame of the video image according to the bounding rectangle of the face region, the extraction unit 220 is further configured to: expand the region according to a preset multiple of the width and height of the bounding rectangle of the face region to obtain an expanded bounding rectangle of the region, wherein the expanded bounding rectangle of the region is used to indicate the cropping region of each frame of the video image.

[0115] When the extraction unit 220 crops each frame of the video image according to the bounding rectangle of the face region to obtain the face region image corresponding to each frame of the video image, it can be used to: crop each frame of the video image according to the expanded bounding rectangle of the region to obtain the face region image corresponding to each frame of the video image.

[0116] In some embodiments, the face false detection optimization device 200 further includes a construction unit:

[0117] The construction unit can be used to construct a face feature template library based on the face features of the first N frames of video images in the video image sequence, wherein the face feature template library contains a face feature template corresponding to each face in the video image sequence, and N≥3.

[0118] In some embodiments, the construction unit is further configured to: obtain the facial features of the current frame video image in the video image sequence; if the similarity value between the facial features of the current frame video image and the facial feature templates in the facial feature template library is greater than a first threshold and less than a second threshold, then reconstruct the facial feature template library based on the facial features of the current frame video image and the facial features of the previous two frames of video images.

[0119] In some embodiments, the construction unit is further configured to: update the face feature template library based on the face features of the current frame video image if the similarity value between the face features of the current frame video image and the face feature templates in the face feature template library is greater than the second threshold.

[0120] In some embodiments, the face false detection optimization device 200 further includes a training unit:

[0121] The training unit is used to acquire a training sample set, which includes facial feature samples from sample video images and facial feature template samples from a facial feature template sample library; and to train the matching network model based on the training sample set to obtain a trained matching network model.

[0122] In some embodiments, when acquiring a training sample set, the training unit is specifically configured to: create sample video images based on multiple sample video image sequence frames, the sample video images including positive sample video images and negative sample video images, wherein each positive sample video image has a corresponding positive facial feature sample, and each negative sample video image has a corresponding negative facial feature sample; create facial feature template samples from a facial feature template sample library based on the multiple sample video image sequence frames; concatenate the positive facial feature samples of the positive sample video images with the facial feature template samples to obtain a first concatenated feature map, and label the first concatenated feature map with a first similarity label to obtain a first training sample pair; concatenate the negative facial feature samples of the negative sample video images with the facial feature template samples to obtain a second concatenated feature map, and label the first concatenated feature map with a second similarity label to obtain a second training sample pair;

[0123] A training sample set is obtained based on the first training sample pair and the second training sample pair.

[0124] In some embodiments, when the training unit trains the matching network model based on the training sample set to obtain a trained matching network model, it specifically performs the following steps: inputting the first concatenated feature map from the training sample set into the matching network model to obtain a first predicted matching result, and training the matching network model based on the first predicted matching result; inputting the first concatenated feature map from the training sample set into the matching network model to obtain a second predicted matching result, and training the matching network model based on the second predicted matching result; and ending the training of the matching network model when the first predicted matching result matches the first similarity label and the second predicted matching result matches the second similarity label.

[0125] In some embodiments, when the training unit inputs the first spliced ​​feature map from the training sample set into the matching network model to obtain a first predicted matching result, and trains the matching network model based on the first predicted matching result, it can be used to: input the first spliced ​​feature map from the training sample set into the matching network model for similarity matching, so as to output a first similarity value between the positive face feature sample and the face feature template sample in the first spliced ​​feature map; if the first similarity value is less than a first threshold, the obtained first predicted matching result is that the positive face feature sample and the face feature template sample are predicted to be dissimilar, and the first predicted matching result is determined to be... If the predicted matching result does not match the first similarity label, and the face feature template sample library is updated again, the first spliced ​​feature map is updated according to the updated face feature template sample library, and the step of inputting the first spliced ​​feature map in the training sample set into the matching network model for similarity matching is returned, until the first similarity value is greater than or equal to the first threshold, the first predicted matching result is predicted to be similar to the face feature positive sample and the face feature template sample, so that the first predicted matching result matches the first similarity label.

[0126] In some embodiments, the training unit is further configured to: if the first similarity value is greater than the first threshold and less than the second threshold, update the face feature template sample library again, update the first spliced ​​feature map according to the updated face feature template sample library again, and return to the step of inputting the first spliced ​​feature map in the training sample set into the matching network model for similarity matching, until the first similarity value is greater than or equal to the second threshold, and then end the training of the matching network model.

[0127] In some embodiments, when the training unit inputs the first spliced ​​feature map from the training sample set into the matching network model to obtain a second predicted matching result, and trains the matching network model based on the second predicted matching result, it can be used to: input the second spliced ​​feature map from the training sample set into the matching network model for similarity matching, so as to output a second similarity value between the negative face feature sample and the face feature template sample in the second spliced ​​feature map; if the second similarity value is greater than or equal to a first threshold, the obtained second predicted matching result is a prediction that the negative face feature sample and the face feature template sample are similar, and the first... If the second predicted matching result does not match the second similarity label, the face feature template sample library is updated again, and the second spliced ​​feature map is updated according to the updated face feature template sample library. The process of inputting the second spliced ​​feature map in the training sample set into the matching network model for similarity matching is repeated until the second similarity value is less than the first threshold. In this case, the second predicted matching result is that the negative face feature sample and the face feature template sample are not similar, so that the second predicted matching result matches the second similarity label.

[0128] In some embodiments, the processing unit 240 can also be used to: if the face feature does not match the face feature template in the face feature template library, determine that the face in the face detection result corresponding to the current frame video image in the video image sequence is a false detection.

[0129] In some embodiments, the processing unit 240 may also be used to: feed back the falsely detected face detection results to correct the bounding rectangle of the face region based on the falsely detected face detection results.

[0130] Each unit in the aforementioned face false detection optimization device can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each unit.

[0131] The face false detection optimization device 200 can be integrated into a terminal or server that has storage and a processor and thus computing power, or the face false detection optimization device 200 can be the terminal or server.

[0132] Optionally, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0133] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device can be a terminal, and the terminal can be a smart vehicle terminal. For example... Figure 6 As shown, the computer device 300 may include: a communication interface 301, a memory 302, a processor 303, and a communication bus 304. The communication interface 301, memory 302, and processor 303 communicate with each other via the communication bus 304. The communication interface 301 is used for data communication between the computer device 300 and external devices. The memory 302 can be used to store software programs and modules, and the processor 303 runs the software programs and modules stored in the memory 302, such as the software programs for the corresponding operations in the foregoing method embodiments.

[0134] Optionally, the processor 303 can invoke software programs and modules stored in the memory 302 to perform the following operations:

[0135] The video to be tested is decomposed to form a sequence of video image frames arranged in chronological order. Feature extraction is performed on each frame of the video image sequence to obtain facial features for each frame. The facial features include at least left eye feature information, right eye feature information, and mouth information. Based on a trained matching network model, the facial features of each frame of the video image are matched with facial feature templates in a facial feature template library. If the facial features match the facial feature templates in the facial feature template library, it is determined that the face in the face detection result corresponding to the current frame of the video image sequence is correct, and the facial feature template library is updated according to the facial features of the current frame of the video image.

[0136] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the face false detection optimization method of the embodiments of this application; for the sake of brevity, these will not be elaborated further here.

[0137] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding process in the face false detection optimization method of this application embodiment. For simplicity, further details are omitted here.

[0138] This application also provides a computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding process in the face false detection optimization method of this application. For brevity, further details are omitted here.

[0139] It should be understood that the processor in this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0140] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0141] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0142] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] In addition, the functional units in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0146] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0147] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for optimizing false face detection, characterized in that, The method includes: The video to be tested is decomposed to form a sequence of video image frames arranged in chronological order; Feature extraction is performed on each frame of the video image sequence to obtain facial features for each frame. This includes: detecting facial key points in each frame of the video image sequence to obtain facial key points; determining a bounding rectangle for the facial region of each frame based on the facial key points, wherein the bounding rectangle includes at least a bounding rectangle for the left eye, a bounding rectangle for the right eye, and a bounding rectangle for the mouth; cropping each frame of the video image based on the bounding rectangle to obtain a facial region image corresponding to each frame, wherein the facial region image includes at least a left eye image, a right eye image, and a mouth image; and extracting features from the facial region image corresponding to each frame of the video image to obtain facial features for each frame, wherein the facial features include at least left eye feature information, right eye feature information, and mouth information. A face feature template library is constructed based on the face features of the first N frames of the video image sequence. The face feature template library contains face feature templates corresponding to each face in the video image sequence, where N≥3. The lifecycle of a face feature template corresponding to a single face is only retained during the time period when the face is in the video frame. Once the face goes out of bounds, the face feature template corresponding to the out-of-bounds face will be released. Based on the trained matching network model, feature matching is performed on the facial features of each frame of the video image and the facial feature templates in the facial feature template library; If the facial features match the facial feature templates in the facial feature template library, then the face in the face detection result corresponding to the current frame video image in the video image sequence is determined to be correct, and the facial feature template library is updated according to the facial features of the current frame video image; If the facial feature does not match the facial feature template in the facial feature template library, then the face in the face detection result corresponding to the current frame video image in the video image sequence is determined to be a false detection, and the false detection face detection result is fed back to correct the bounding rectangle of the face region based on the false detection face detection result.

2. The face false detection optimization method as described in claim 1, characterized in that, The step of extracting features from the face region image corresponding to each frame of the video image to obtain the face features of each frame of the video image includes: Obtain the color histogram features of the face region image corresponding to each frame of the video image. The color histogram features include a first color histogram feature corresponding to the left eye image, a second color histogram feature corresponding to the right eye image, and a third color histogram feature corresponding to the mouth image. The first color histogram feature, the second color histogram feature, and the third color histogram feature corresponding to each frame of the video image are concatenated to obtain the face features of each frame of the video image.

3. The face false detection optimization method as described in claim 2, characterized in that, The step of obtaining the color histogram features of the face region image corresponding to each frame of the video image includes: In each frame of the video image corresponding to the face region image, each color channel in the color space of each image is uniformly divided into n sub-intervals within the gray value range of [0, 255], where each sub-interval corresponds to an interval scale in the color histogram. Calculate the number of pixels in each color channel of each image in each sub-interval of the face region image corresponding to each frame of the video image, and determine the number of pixels as the feature value of the corresponding interval scale in the color histogram; The feature values ​​of the corresponding interval scale in the color histogram are normalized to the range [0,1] to obtain the color histogram features of each image in the face region image corresponding to each frame of the video image.

4. The face false detection optimization method as described in claim 1, characterized in that, Before cropping each frame of the video image based on the bounding rectangle surrounding the face region, the method further includes: The region is expanded according to a preset multiple of the width and height of the bounding rectangle of the face region to obtain an expanded bounding rectangle. The expanded bounding rectangle is used to indicate the cropping area of ​​each frame of the video image. The step of cropping each frame of the video image based on the bounding rectangle of the face region to obtain the face region image corresponding to each frame of the video image includes: Each frame of the video image is cropped based on the expanded bounding rectangle to obtain the face region image corresponding to each frame of the video image.

5. The face false detection optimization method as described in claim 4, characterized in that, The method further includes: Obtain the facial features of the current frame video image in the video image sequence; If the similarity value between the facial features of the current frame video image and the facial feature templates in the facial feature template library is greater than a first threshold and less than a second threshold, then the facial feature template library is reconstructed based on the facial features of the current frame video image and the facial features of the previous two frames of video images.

6. The face false detection optimization method as described in claim 5, characterized in that, The method further includes: If the similarity value between the facial features of the current frame video image and the facial feature templates in the facial feature template library is greater than the second threshold, then the facial feature template library is updated according to the facial features of the current frame video image.

7. The face false detection optimization method as described in any one of claims 1-4, characterized in that, Before performing feature matching between the facial features of each frame of the video image and the facial feature templates in the facial feature template library based on the trained matching network model, the method further includes: Obtain a training sample set, which includes facial feature samples from sample video images and facial feature template samples from a facial feature template sample library; The matching network model is trained based on the training sample set to obtain the trained matching network model.

8. The face false detection optimization method as described in claim 7, characterized in that, The acquisition of the training sample set includes: Sample video images are generated based on multiple sample video image sequence frames. The sample video images include positive sample video images and negative sample video images. Each positive sample video image has a corresponding positive facial feature sample, and each negative sample video image has a corresponding negative facial feature sample. Based on the multiple sample video image sequence frames, facial feature template samples are created from the facial feature template sample library; The positive facial feature samples of the positive sample video image are concatenated with the facial feature template samples to obtain a first concatenated feature map, and the first similarity label of the first concatenated feature map is labeled to obtain a first training sample pair. The negative facial feature samples of the negative sample video image are concatenated with the facial feature template samples to obtain a second concatenated feature map, and the second similarity label of the first concatenated feature map is labeled to obtain a second training sample pair. A training sample set is obtained based on the first training sample pair and the second training sample pair.

9. The face false detection optimization method as described in claim 8, characterized in that, The step of training the matching network model based on the training sample set to obtain the trained matching network model includes: The first concatenated feature map from the training sample set is input into the matching network model to obtain a first predicted matching result, and the matching network model is trained based on the first predicted matching result. The first concatenated feature map from the training sample set is input into the matching network model to obtain a second predicted matching result, and the matching network model is trained based on the second predicted matching result. The training of the matching network model ends when the first predicted matching result matches the first similarity label and the second predicted matching result matches the second similarity label.

10. The face false detection optimization method as described in claim 9, characterized in that, The step of inputting the first concatenated feature map from the training sample set into the matching network model to obtain a first predicted matching result, and training the matching network model based on the first predicted matching result, includes: The first spliced ​​feature map in the training sample set is input into the matching network model for similarity matching, so as to output the first similarity value between the positive face feature sample and the face feature template sample in the first spliced ​​feature map; If the first similarity value is less than the first threshold, the first prediction matching result is that the positive sample of the face feature and the face feature template sample are not similar, and it is determined that the first prediction matching result does not match the first similarity label. If it is determined that the first predicted matching result does not match the first similarity label, the face feature template sample library is updated again, and the first spliced ​​feature map is updated according to the updated face feature template sample library. Then, the process returns to the step of inputting the first spliced ​​feature map in the training sample set into the matching network model for similarity matching until the first similarity value is greater than or equal to the first threshold. The first predicted matching result is that the positive face feature sample and the face feature template sample are similar, so that the first predicted matching result matches the first similarity label.

11. The face false detection optimization method as described in claim 10, characterized in that, The method further includes: If the first similarity value is greater than the first threshold and less than the second threshold, then the face feature template sample library is updated again, and the first spliced ​​feature map is updated according to the updated face feature template sample library. Then, the step of inputting the first spliced ​​feature map in the training sample set into the matching network model for similarity matching is returned. The training of the matching network model ends only when the first similarity value is greater than or equal to the second threshold.

12. The face false detection optimization method as described in claim 9, characterized in that, The step involves inputting the first concatenated feature map from the training sample set into the matching network model to obtain a second predicted matching result, and then training the matching network model based on the second predicted matching result. The second spliced ​​feature map in the training sample set is input into the matching network model for similarity matching, so as to output the second similarity value between the negative facial feature sample and the facial feature template sample in the second spliced ​​feature map; If the second similarity value is greater than or equal to the first threshold, the second prediction matching result is that the negative sample of the face feature and the face feature template sample are similar, and it is determined that the second prediction matching result does not match the second similarity label. If it is determined that the second predicted matching result does not match the second similarity label, the face feature template sample library is updated again, and the second spliced ​​feature map is updated according to the updated face feature template sample library. Then, the step of inputting the second spliced ​​feature map in the training sample set into the matching network model for similarity matching is returned. Only when the second similarity value is less than the first threshold, the second predicted matching result is that the predicted negative face feature sample and the face feature template sample are not similar, so that the second predicted matching result matches the second similarity label.

13. A face false detection optimization device, characterized in that, The device includes: The decomposition unit is used to decompose the video under test into a sequence of video image frames arranged in chronological order. An extraction unit is configured to extract features from each frame of the video image sequence to obtain facial features for each frame, including: detecting facial key points in each frame of the video image sequence to obtain facial key points in each frame; determining a bounding rectangle of the facial region in each frame of the video image based on the facial key points, wherein the bounding rectangle of the facial region includes at least a bounding rectangle of the left eye, a bounding rectangle of the right eye, and a bounding rectangle of the mouth; cropping each frame of the video image based on the bounding rectangle of the facial region to obtain a facial region image corresponding to each frame of the video image, wherein the facial region image includes at least a left eye image, a right eye image, and a mouth image; and extracting features from the facial region image corresponding to each frame of the video image to obtain facial features for each frame of the video image, wherein the facial features include at least left eye feature information, right eye feature information, and mouth information. The construction unit is used to construct a face feature template library based on the face features of the first N frames of video images in the video image sequence. The face feature template library contains face feature templates corresponding to each face in the video image sequence, where N≥3. The lifecycle of a face feature template corresponding to a single face is only retained during the time period when the face is in the video frame. Once the face goes out of bounds, the face feature template corresponding to the out-of-bounds face will be released. The matching unit is used to perform feature matching between the facial features of each frame of the video image and the facial feature templates in the facial feature template library based on the trained matching network model. The processing unit is configured to determine that the face in the face detection result corresponding to the current frame video image in the video image sequence is correct if the face feature matches a face feature template in the face feature template library, and update the face feature template library according to the face feature of the current frame video image; The processing unit is further configured to: if the face feature does not match the face feature template in the face feature template library, determine that the face in the face detection result corresponding to the current frame video image in the video image sequence is a false detection, and feed back the false face detection result to correct the bounding rectangle of the face region according to the false face detection result.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to execute the face false detection optimization method as described in any one of claims 1-12.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program, and the processor executing the face false detection optimization method according to any one of claims 1-12 by calling the computer program stored in the memory.

16. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the face false detection optimization method according to any one of claims 1-12.

Images