Face area recognition method, device, electronic device and storage medium

By performing frame-by-frame processing on the video and utilizing compressed domain information, combined with high-resolution face detection and motion vector information, the problem of flickering in the face detection model was solved, achieving accurate, stable, and efficient recognition of face regions in the video.

CN117746470BActive Publication Date: 2024-11-19XINGIN INFORMATION TECH (WUHAN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211622758.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-16
Publication Date
2024-11-19
Estimated Expiration
2042-12-16

AI Technical Summary

Technical Problem

In existing technologies, face detection models are prone to flickering when identifying regions of interest, leading to inaccurate region recognition and affecting video rendering quality.

Method used

By segmenting the video into frames, using compression domain information to determine the frame type and reference frame, and combining high-resolution face detection and motion vector information, the face region of each frame of video image can be accurately identified, avoiding flickering.

Benefits of technology

It achieves accurate and stable recognition of facial regions in videos, reduces computational costs, improves recognition efficiency, and prevents flickering.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117746470B_ABST
    Figure CN117746470B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, electronic device and storage medium for face region recognition, wherein the method comprises: performing frame processing on a video to be processed to obtain at least one frame of video image; parsing each frame of the at least one frame of video image to obtain at least one compression domain information, wherein at least one compression domain information corresponds to at least one frame of video image; determining the frame type and reference frame image of each frame of video image according to each compression domain information in the at least one compression domain information; determining the recognition method of each frame of video image according to the frame type of each frame of video image; performing face region recognition on each frame of video image according to the recognition method of each frame of video image and the reference frame image of each frame of video image to obtain the face region of each frame of video image. Using this method, the face region of each frame in the video can be accurately and stably recognized without flickering.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of face region recognition, and in particular to a face region recognition method, device, electronic equipment and storage medium. Background Art

[0002] In the process of identifying the region of interest (ROI), the commonly used face detection model will have some flickering when inferring the face area. The reason for these problems is that during the recognition process, there are some frames that are not recognized, causing the recognition frame to flicker. This will lead to inaccurate region recognition and flickering quality of the ROI area, which will affect subsequent video rendering. Therefore, there is an urgent need for a stable face recognition method for video coding ROI to stably identify the face area in each frame and prevent flickering. Summary of the invention

[0003] In order to solve the above-mentioned problems existing in the prior art, the embodiments of the present application provide a face area recognition method, device, electronic device and storage medium, which can accurately and stably identify the face area of ​​each frame in the video and prevent flickering from occurring.

[0004] In a first aspect, an embodiment of the present application provides a method for recognizing a face region, the method comprising:

[0005] Perform frame processing on the video to be processed to obtain at least one frame of video image;

[0006] Parsing each video frame in at least one video frame to obtain at least one compression domain information, wherein the at least one compression domain information corresponds to the at least one video frame;

[0007] Determine a frame type and a reference frame image of each frame of the video image according to each compression domain information of at least one compression domain information;

[0008] Determine a recognition method for each frame of the video image according to the frame type of each frame of the video image;

[0009] According to the recognition method of each frame of video image and the reference frame image of each frame of video image, face region recognition is performed on each frame of video image to obtain the face region of each frame of video image.

[0010] In a possible implementation manner, the frame type includes: a frame type without reference and a frame type with reference.

[0011] In a possible implementation, when the frame type is a reference frame type, according to the recognition method of each frame of video image and the reference frame of each frame of video image, face region recognition is performed on each frame of video image to obtain the face region of each frame of video image, including:

[0012] Acquire first compression domain information of a face region in a reference frame image of each frame of video image;

[0013] According to the first compression domain information, obtaining the region offset;

[0014] Performing a regional offset process on the face region in the reference frame image of each frame of the video image according to the regional offset amount to obtain a first offset region;

[0015] Acquire second compression domain information of a first correction region in each frame of video image, wherein a position of the first correction region in each frame of video image is the same as a position of the first offset region in a reference frame image of each frame of video image;

[0016] Determining the credibility of the first correction region according to the second compression domain information;

[0017] When the credibility is credible, the first correction area is determined as the face area of ​​each frame of the video image.

[0018] In a possible implementation manner, obtaining the region offset according to the first compression domain information includes:

[0019] Determine, according to the first compression domain information, a motion vector in a face region in a reference frame image of each frame of the video image;

[0020] The average value of the motion vectors is used as the region offset.

[0021] In a possible implementation manner, determining the credibility of the first correction region according to the second compression domain information includes:

[0022] Determine, according to the second compression domain information, a percentage value occupied by the Intra block in the first correction area;

[0023] determining an average residual of the motion vectors in the first correction region based on the second compressed domain information;

[0024] When the percentage value is less than the first threshold value and the average residual is less than the second threshold value, the credibility of the first correction region is determined to be credible.

[0025] In a possible implementation manner, the reference frame types include: a unidirectional reference frame type and a bidirectional reference frame type.

[0026] In a possible implementation, when the frame type is an intra-frame coding type, according to a recognition method for each frame of video image and a reference frame of each frame of video image, face region recognition is performed on each frame of video image to obtain a face region of each frame of video image, including:

[0027] Determine a first resolution according to the original resolution of the video to be processed;

[0028] Perform face detection on each frame of video image according to the first resolution to determine a face area of ​​each frame of video image.

[0029] In a possible implementation manner, the first resolution and the original resolution can be expressed by formula ①:

[0030]

[0031] Among them, A1 represents the first resolution, A P Indicates native resolution.

[0032] In a possible implementation manner, the non-reference frame type includes: an intra-frame coding type.

[0033] In a second aspect, an embodiment of the present application provides a face region recognition device, comprising:

[0034] A frame segmentation module, used for performing frame segmentation processing on the video to be processed to obtain at least one frame of video image;

[0035] A parsing module, configured to parse each frame of at least one frame of video image to obtain at least one compression domain information, wherein the at least one compression domain information corresponds to the at least one frame of video image one by one, and determine a frame type and a reference frame image of each frame of video image according to each compression domain information in the at least one compression domain information;

[0036] The processing module is used to determine the recognition method of each video frame according to the frame type of each video frame, and to perform face area recognition on each video frame according to the recognition method of each video frame and the reference frame image of each video frame to obtain the face area of ​​each video frame.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method of the first aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method of the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and is computer-operable to cause the computer to execute the method of the first aspect.

[0040] The implementation of the present application has the following beneficial effects:

[0041] In the implementation mode of the present application, the video to be processed is divided into frames, and then the frame type and reference frame are determined by the compression domain information of each frame, and then the corresponding recognition method is further determined by the frame type. Then, the matched recognition method is combined with the corresponding reference frame and compression domain information to accurately recognize the face of each frame. Thus, the compression domain information obtained in the video encoding ROI process is directly used, combined with the reference frame of each frame, to accurately and stably recognize the face area in each frame of the video, and prevent the occurrence of flickering. In addition, with this method, there is no need to recalculate the relationship between the two frames, which can reduce the computing cost and improve the recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the implementation modes of the present application, the drawings required for use in the description of the implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A schematic diagram of the hardware structure of a face region recognition device provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of a flow chart of a face region recognition method provided in an embodiment of the present application;

[0045] Figure 3 A reference frame image superimposed with motion vector information in a compressed domain provided in an embodiment of the present application;

[0046] Figure 4 A functional module composition block diagram of a face area recognition device provided in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0050] Reference to "embodiment" herein means that a particular feature, result, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0051] See also Figure 1 , Figure 1 A hardware structure diagram of a face region recognition device provided in an embodiment of the present application. The face region recognition device 100 includes at least one processor 101 , a communication line 102 , a memory 103 and at least one communication interface 104 .

[0052] In this embodiment, the processor 101 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0053] The communication link 102 may include a path to transmit information between the above components.

[0054] The communication interface 104 may be any transceiver-like device (eg, antenna, etc.) for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.

[0055] The memory 103 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0056] In this embodiment, the memory 103 can exist independently and be connected to the processor 101 via the communication line 102. The memory 103 can also be integrated with the processor 101. The memory 103 provided in the embodiment of the present application can generally be non-volatile. Among them, the memory 103 is used to store computer-executable instructions for executing the scheme of the present application, and the execution is controlled by the processor 101. The processor 101 is used to execute the computer-executable instructions stored in the memory 103, so as to implement the method provided in the following embodiment of the present application.

[0057] In an optional implementation, the computer-executable instructions may also be referred to as application code, which is not specifically limited in this application.

[0058] In an optional embodiment, the processor 101 may include one or more CPUs, such as Figure 1 CPU0 and CPU1 in.

[0059] In an optional embodiment, the face region recognition device 100 may include multiple processors, such as Figure 11 and 107. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0060] In an optional embodiment, if the face region recognition device 100 is a server, for example, it can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. Then the face region recognition device 100 can also include an output device 105 and an input device 106. The output device 105 communicates with the processor 101 and can display information in a variety of ways. For example, the output device 105 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 106 communicates with the processor 101 and can receive user input in a variety of ways. For example, the input device 106 can be a mouse, a keyboard, a touch screen device, or a sensor device.

[0061] The above-mentioned face region recognition device 100 can be a general device or a special device. The embodiment of the present application does not limit the type of the face region recognition device 100.

[0062] The following is a description of a face region recognition method disclosed in this application:

[0063] See also Figure 2 , Figure 2 A flow chart of a face region recognition method provided in an embodiment of the present application. The face region recognition method comprises the following steps:

[0064] 201: Perform frame processing on the video to be processed to obtain at least one frame of video image.

[0065] 202: Analyze each video frame of at least one video frame to obtain at least one compression domain information.

[0066] In this embodiment, the at least one compression domain information corresponds to at least one frame of video image. Specifically, the compression domain information may include: frame type, reference frame, motion vector information, number of Intra blocks, number of Skip blocks, average value of residuals, and other information of each frame of video image. The compression domain information is generated in the process of decoding the video to be processed, and decoding the video is a necessary step in the video encoding ROI processing process. Therefore, when encoding the ROI of the video to be processed, the compression domain information generated during decoding can be directly obtained for subsequent use, which can improve the recognition efficiency of the method while reducing computing power consumption.

[0067] 203: Determine a frame type and a reference frame image of each frame of the video image according to each compression domain information of the at least one compression domain information.

[0068] In this embodiment, the frame type may include a frame type without reference and a frame type with reference. Among them, the frame type without reference may include an intra-frame coding type frame I frame, and this type of frame image is usually a scene switching frame, that is, switching from one scene to another scene, and serving as a frame at the beginning of another scene. For the I frame, as the first frame at the beginning of a scene, there is no other frame that can be used as a reference to assist in its parsing, that is, the reference frame of the I frame is empty. The frame type with reference may include: a unidirectional reference frame type frame P frame and a bidirectional reference frame type frame B frame. Among them, the reference frame of the P frame is a single frame, that is, another frame in the same scene, and the reference frame of the B frame is a double frame, that is, another two frames in the same scene.

[0069] In this embodiment, the frame type and reference frame image of each frame of video image are recorded in its corresponding compression domain information. Based on this, the frame type and reference frame of the frame of video image can be determined by querying the compression domain information corresponding to the frame of video image.

[0070] 204: Determine a recognition method for each frame of the video image according to the frame type of each frame of the video image.

[0071] 205: According to the recognition method of each frame of video image and the reference frame image of each frame of video image, perform face region recognition on each frame of video image to obtain the face region of each frame of video image.

[0072] In this embodiment, since there is no reference frame for the I frame, the face region can only be recognized based on its own information. At the same time, since the I frame will be used as a reference frame for other frames of video images in the same scene, in order to ensure the accuracy of face recognition in subsequent frames of video images, in this embodiment, the first resolution can be determined based on the original resolution of the video to be processed, and then face detection is performed on the video image of the I frame based on the first resolution to determine the face region of the video image of the I frame.

[0073] Specifically, the first resolution and the original resolution can be expressed by formula ②:

[0074]

[0075] Among them, A1 represents the first resolution, A P Indicates native resolution.

[0076] That is, the first resolution is any value between half of the original resolution and the original resolution. Therefore, by performing face recognition on the video image of frame I at a higher resolution, while ensuring the recognition accuracy, it can provide a high-precision reference frame for the face recognition of subsequent frames of video images, thereby further improving the recognition accuracy of the face area of ​​the entire video to be processed.

[0077] In this embodiment, for the P frame, the first compression domain information of the face area in the reference frame image of the video image of the P frame can be first obtained, and then the area offset can be obtained according to the first compression domain information. Then, the face area in the reference frame image of each frame of the video image is subjected to area offset processing according to the area offset to obtain the first offset area. Then, the second compression domain information of the first correction area in the video image of the P frame is obtained, wherein the position of the first correction area in each frame of the video image is the same as the position of the first offset area in the reference frame image of each frame of the video image. Finally, the credibility of the first correction area is determined according to the second compression domain information, and when the credibility is credible, the first correction area is determined as the face area of ​​the video image of the P frame.

[0078] Specifically, in this embodiment, if the reference frame of the video image of the P frame is an I frame, the face area identified by face detection in the I frame can be used as the face area in the reference frame image of the video image of the P frame, and then the compressed domain information of the area is obtained. If the reference frame of the video image of the P frame is not an I frame, the reference frame can be processed first until a video image whose reference frame is an I frame is found, thereby determining the face area of ​​each frame step by step until the face area of ​​the reference frame of the video image of the P frame is determined, and then the next step of processing is performed.

[0079] In this embodiment, the motion vector in the face region of the reference frame image of the video image of the P frame can be determined based on the first compression domain information, and then the average value of the motion vector is used as the region offset. Specifically, the motion vector information may include direction and size, and each pixel in the image has a corresponding motion vector for identifying the motion trend of the pixel. Figure 3 As shown, Figure 3A reference frame image superimposed with motion vector information in the compressed domain is shown, wherein the area framed by the box is the face area. It can be seen that each pixel in the area has a motion vector, which is used to identify the motion trend of the pixel in the next video, and then the average value of the motion vector in the area can be regarded as the motion trend of the area in the next video, that is, the area offset. Then, by moving the face area by the area offset, the predicted position of the face area in the video image of the P frame can be obtained, that is, the first offset area.

[0080] Then, in this embodiment, the first offset area is projected into the video image of the P frame to determine the area with the same position in the video image of the P frame as the first correction area, which is the area of ​​the suspected face area in the video image of the P frame.

[0081] In this embodiment, after determining the first correction region, it can be verified whether it is a face region, that is, the credibility of the region can be determined. Specifically, the number of Intra blocks and the number of Skip blocks in the first correction region can be determined based on the compression and information of the first correction region, that is, the second compression domain information, and then the percentage value of the Intra blocks in the first correction region can be determined. The percentage value can be expressed by formula ③:

[0082]

[0083] Among them, P I Indicates percentage value, L I represents the number of Intra blocks in the first correction area, L S Indicates the number of Skip blocks in the first correction area.

[0084] At the same time, the second compression domain information is queried to determine the average residual of the motion vector in the first correction area.

[0085] In this embodiment, when the percentage value is less than the first threshold and the average residual is less than the second threshold, the credibility of the first corrected area can be determined to be credible, that is, the first corrected area can be determined to be the face area of ​​the video image of the P frame.

[0086] In this embodiment, for B frames, since they have two corresponding reference frames, the two reference frames of the B frame can be processed separately according to the above-mentioned processing method for P frames, and then the most credible of the two first correction areas determined by the two reference frames is used as the face area of ​​the video image of the B frame. Generally speaking, when the percentage value is lower than the first threshold and the lower it is, and the average residual is lower than the second threshold and the lower it is, the higher its credibility is. In addition, the overlapping area or the minimum circumscribed area of ​​the two first correction areas can also be used as a new first correction area for subsequent credibility verification, and then the face area in the video image of the B frame is determined, and this application does not impose any restrictions on this.

[0087] Therefore, by performing face detection at high resolution to obtain the I frame of the face area as a reference frame, combined with the relationship of the reference frames in the compressed domain information, the face area of ​​each subsequent frame video image is determined, so that in the face recognition process of the entire video to be processed, only the I frame of each scene needs to be detected, and no face detection is required for other frames to achieve stable face detection of the entire video, saving a lot of computing costs while ensuring the successful recognition of the face area in each frame image and preventing the occurrence of flickering. At the same time, since only the I frame needs to be recognized, high-resolution face detection can be performed on the I frame, which can further ensure the accuracy of face area recognition while also ensuring recognition efficiency.

[0088] In summary, in the face area recognition method provided by the present invention, the video to be processed is divided into frames, and then the frame type and reference frame are determined by the compression domain information of each frame image, and then the corresponding recognition method is further determined by its frame type. Then, the matched recognition method is combined with the corresponding reference frame and compression domain information to accurately recognize the face of each frame image. Thus, the compression domain information obtained in the video encoding ROI process is directly used, combined with the reference frame of each frame image, to accurately and stably recognize the face area in each frame image in the video, and prevent the occurrence of flickering. In addition, through this method, there is no need to recalculate the relationship between the two frames, which can reduce the calculation cost and improve the recognition efficiency.

[0089] See also Figure 4 , Figure 4 This is a functional module block diagram of a face area recognition device provided in an embodiment of the present application. Figure 4 As shown, the face region recognition device 400 includes:

[0090] The frame segmentation module 401 is used to perform frame segmentation processing on the video to be processed to obtain at least one frame of video image;

[0091] The parsing module 402 is used to parse each frame of the at least one frame of video image to obtain at least one compression domain information, wherein the at least one compression domain information corresponds to the at least one frame of video image one by one, and determine the frame type and reference frame image of each frame of video image according to each compression domain information in the at least one compression domain information;

[0092] The processing module 403 is used to determine the recognition method of each video frame according to the frame type of each video frame, and perform face area recognition on each video frame according to the recognition method of each video frame and the reference frame image of each video frame to obtain the face area of ​​each video frame.

[0093] In an embodiment of the present invention, the frame type includes: a frame type without reference and a frame type with reference.

[0094] In an embodiment of the present invention, when the frame type is a reference frame type, in terms of performing face region recognition on each frame of video image according to the recognition method of each frame of video image and the reference frame of each frame of video image, and obtaining the face region of each frame of video image, the processing module 403 is specifically used to:

[0095] Acquire first compression domain information of a face region in a reference frame image of each frame of video image;

[0096] According to the first compression domain information, obtaining the region offset;

[0097] Performing a regional offset process on the face region in the reference frame image of each frame of the video image according to the regional offset amount to obtain a first offset region;

[0098] Acquire second compression domain information of a first correction region in each frame of video image, wherein a position of the first correction region in each frame of video image is the same as a position of the first offset region in a reference frame image of each frame of video image;

[0099] Determining the credibility of the first correction region according to the second compression domain information;

[0100] When the credibility is credible, the first correction area is determined as the face area of ​​each frame of the video image.

[0101] In the embodiment of the present invention, in terms of obtaining the region offset according to the first compression domain information, the processing module 403 is specifically configured to:

[0102] Determine, according to the first compression domain information, a motion vector in a face region in a reference frame image of each frame of the video image;

[0103] The average value of the motion vectors is used as the region offset.

[0104] In an embodiment of the present invention, in determining the credibility of the first correction region according to the second compression domain information, the processing module 403 is specifically configured to:

[0105] Determine, according to the second compression domain information, a percentage value occupied by the Intra block in the first correction area;

[0106] determining an average residual of the motion vectors in the first correction region based on the second compressed domain information;

[0107] When the percentage value is less than the first threshold value and the average residual is less than the second threshold value, the credibility of the first correction region is determined to be credible.

[0108] In an embodiment of the present invention, the reference frame types include: a unidirectional reference frame type and a bidirectional reference frame type.

[0109] In an embodiment of the present invention, when the frame type is a reference frame-free type, in terms of performing face region recognition on each frame of video image according to the recognition method of each frame of video image and the reference frame of each frame of video image, and obtaining the face region of each frame of video image, the processing module 403 is specifically used to:

[0110] Determine a first resolution according to the original resolution of the video to be processed;

[0111] Perform face detection on each frame of video image according to the first resolution to determine a face area of ​​each frame of video image.

[0112] In an embodiment of the present invention, the first resolution and the original resolution can be expressed by formula ④:

[0113]

[0114] Among them, A1 represents the first resolution, A P Indicates native resolution.

[0115] In an embodiment of the present invention, the non-reference frame type includes: an intra-frame coding type.

[0116] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes a transceiver 501, a processor 502 and a memory 503. They are connected via a bus 504. The memory 503 is used to store computer programs and data, and can transmit the data stored in the memory 503 to the processor 502.

[0117] The processor 502 is used to read the computer program in the memory 503 and perform the following operations:

[0118] Perform frame processing on the video to be processed to obtain at least one frame of video image;

[0119] Parsing each video frame in at least one video frame to obtain at least one compression domain information, wherein the at least one compression domain information corresponds to the at least one video frame;

[0120] Determine a frame type and a reference frame image of each frame of the video image according to each compression domain information of at least one compression domain information;

[0121] Determine a recognition method for each frame of the video image according to the frame type of each frame of the video image;

[0122] According to the recognition method of each frame of video image and the reference frame image of each frame of video image, face region recognition is performed on each frame of video image to obtain the face region of each frame of video image.

[0123] In an embodiment of the present invention, the frame type includes: a frame type without reference and a frame type with reference.

[0124] In an embodiment of the present invention, when the frame type is a reference frame type, in terms of performing face region recognition on each frame of video image according to the recognition method of each frame of video image and the reference frame of each frame of video image, and obtaining the face region of each frame of video image, the processor 502 is specifically used to perform the following operations:

[0125] Acquire first compression domain information of a face region in a reference frame image of each frame of video image;

[0126] According to the first compression domain information, obtaining the region offset;

[0127] Performing a regional offset process on the face region in the reference frame image of each frame of the video image according to the regional offset amount to obtain a first offset region;

[0128] Acquire second compression domain information of a first correction region in each frame of video image, wherein a position of the first correction region in each frame of video image is the same as a position of the first offset region in a reference frame image of each frame of video image;

[0129] Determining the credibility of the first correction region according to the second compression domain information;

[0130] When the credibility is credible, the first correction area is determined as the face area of ​​each frame of the video image.

[0131] In an embodiment of the present invention, in terms of obtaining the region offset according to the first compression domain information, the processor 502 is specifically configured to perform the following operations:

[0132] Determine, according to the first compression domain information, a motion vector in a face region in a reference frame image of each frame of the video image;

[0133] The average value of the motion vectors is used as the region offset.

[0134] In an embodiment of the present invention, in determining the credibility of the first correction region according to the second compression domain information, the processor 502 is specifically configured to perform the following operations:

[0135] Determine, according to the second compression domain information, a percentage value occupied by the Intra block in the first correction area;

[0136] determining an average residual of the motion vectors in the first correction region based on the second compressed domain information;

[0137] When the percentage value is less than the first threshold value and the average residual is less than the second threshold value, the credibility of the first correction region is determined to be credible.

[0138] In an embodiment of the present invention, the reference frame types include: a unidirectional reference frame type and a bidirectional reference frame type.

[0139] In an embodiment of the present invention, when the frame type is a type without a reference frame, in terms of performing face region recognition on each frame of video image according to a recognition method for each frame of video image and a reference frame of each frame of video image, and obtaining a face region of each frame of video image, the processor 502 is specifically configured to perform the following operations:

[0140] Determine a first resolution according to the original resolution of the video to be processed;

[0141] Perform face detection on each frame of video image according to the first resolution to determine a face area of ​​each frame of video image.

[0142] In an embodiment of the present invention, the first resolution and the original resolution can be expressed by formula ⑤:

[0143]

[0144] Among them, A1 represents the first resolution, A P Indicates native resolution.

[0145] In an embodiment of the present invention, the non-reference frame type includes: an intra-frame coding type.

[0146] It should be understood that the face area recognition device in this application may include a smart phone (such as an Android phone, an iOS phone, a Windows Phone phone, etc.), a tablet computer, a PDA, a laptop computer, a mobile Internet device MID (Mobile Internet Devices, referred to as: MID), a robot or a wearable device, etc. The above-mentioned face area recognition device is only an example, not an exhaustive list, including but not limited to the above-mentioned face area recognition device. In practical applications, the above-mentioned face area recognition device may also include: a smart vehicle terminal, a computer device, etc.

[0147] Through the description of the above implementation modes, those skilled in the art can clearly understand that the present invention can be implemented by combining software with a hardware platform. Based on such an understanding, all or part of the contribution of the technical solution of the present invention to the background technology can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation modes of the present invention or some parts of the implementation modes.

[0148] Therefore, the present application also provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement some or all steps of any face region recognition method described in the above method implementation. For example, the storage medium may include a hard disk, a floppy disk, an optical disk, a tape, a disk, a USB flash drive, a flash memory, etc.

[0149] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any face area recognition method recorded in the above method implementation.

[0150] It should be noted that, for the sake of simplicity, the aforementioned method implementations are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the implementations described in the specification are all optional implementations, and the actions and modules involved are not necessarily required by the present application.

[0151] In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device implementation described above is only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0154] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.

[0155] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0156] A person skilled in the art may understand that all or part of the steps in the various methods of the above-mentioned embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable memory, and the memory may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0157] The above is a detailed introduction to the implementation methods of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above implementation methods is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for recognizing a face region, characterized in that: The method comprises: Perform frame processing on the video to be processed to obtain at least one frame of video image; Parsing each frame of the at least one frame of video image to obtain at least one compression domain information, wherein the at least one compression domain information corresponds to the at least one frame of video image in one-to-one correspondence; Determine, according to each compression domain information in the at least one compression domain information, a frame type and a reference frame image of each frame of the video image, wherein the frame type includes: a frame type without reference and a frame type with reference; Determining a recognition method for each frame of the video image according to a frame type of each frame of the video image; According to the recognition method for each frame of video image and the reference frame image of each frame of video image, face region recognition is performed on each frame of video image to obtain the face region of each frame of video image, including: When the frame type is the reference frame type, determining the motion vector in the face area in the reference frame image of each frame of video image according to the first compression domain information of the face area in the reference frame image of each frame of video image; and taking the average value of the motion vector as the area offset; According to the region offset amount, performing region offset processing on the face region in the reference frame image of each frame of the video image to obtain a first offset region; Determine, according to the second compression domain information of the first correction area in each frame of the video image, a percentage value occupied by the Intra block in the first correction area and an average residual of the motion vector in the first correction area; When the percentage value is less than a first threshold value, and the average residual is less than a second threshold value, the first correction area is determined as the face area of ​​each frame of the video image.

2. The method according to claim 1, characterized in that The reference frame types include: unidirectional reference frame types and bidirectional reference frame types.

3. The method according to claim 1, characterized in that When the frame type is the non-reference frame type, the step of performing face region recognition on each frame of video image according to the recognition method for each frame of video image and the reference frame of each frame of video image to obtain the face region of each frame of video image includes: Determining a first resolution according to the original resolution of the video to be processed; Perform face detection on each frame of the video image according to the first resolution to determine a face area of ​​each frame of the video image.

4. The method according to claim 3, characterized in that The first resolution and the original resolution satisfy the following formula: Wherein, A1 represents the first resolution, A P Indicates the native resolution.

5. The method according to claim 1, characterized in that The non-reference frame type includes: an intra-frame coding type.

6. A face region recognition device, characterized in that: The device comprises: A frame segmentation module, used for performing frame segmentation processing on the video to be processed to obtain at least one frame of video image; A parsing module, configured to parse each frame of the at least one frame of video image to obtain at least one compression domain information, wherein the at least one compression domain information corresponds to the at least one frame of video image one by one, and determine a frame type and a reference frame image of each frame of video image according to each compression domain information in the at least one compression domain information, wherein the frame type includes: a frame type without reference and a frame type with reference; A processing module, used to determine a recognition method for each frame of video image according to a frame type of each frame of video image, and perform face region recognition on each frame of video image according to the recognition method for each frame of video image and a reference frame image of each frame of video image to obtain a face region of each frame of video image, including: When the frame type is the reference frame type, determining the motion vector in the face area in the reference frame image of each frame of video image according to the first compression domain information of the face area in the reference frame image of each frame of video image; and taking the average value of the motion vector as the area offset; According to the region offset amount, performing region offset processing on the face region in the reference frame image of each frame of the video image to obtain a first offset region; Determine, according to the second compression domain information of the first correction area in each frame of the video image, a percentage value occupied by the Intra block in the first correction area and an average residual of the motion vector in the first correction area; When the percentage value is less than a first threshold value, and the average residual is less than a second threshold value, the first correction area is determined as the face area of ​​each frame of the video image.

7. An electronic device, characterized in that: The method comprises a processor, a memory, a communication interface and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the one or more programs include instructions for executing the steps in the method described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 5.