Facial-region identification method and apparatus, and electronic device and storage medium

Abstract

A facial-region identification method and apparatus, and an electronic device and a storage medium. The method comprises: performing framing processing on a video to be processed, so as to obtain at least one video image frame; parsing each of the at least one video image frame, so as to obtain at least one piece of compressed-domain information, wherein the at least one piece of compressed-domain information corresponds to the at least one video image frame on a one-to-one basis; determining the frame type of each video image frame and a reference frame image thereof according to each of the at least one piece of compressed-domain information; determining an identification method for each video image frame according to the frame type of each video image frame; and performing facial-region identification on each video image frame according to the identification method for each video image frame and the reference frame image of each video image frame, so as to obtain a facial region in each video image frame. By means of the method, a facial region in each frame in a video can be precisely and stably identified, thereby preventing flickers.

Description

Face area recognition method, device, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 16, 2022, with application number 202211622758.X and application name “Face Area Recognition Method, Device, Electronic Device and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of face region recognition, and in particular to a face region recognition method, device, electronic device and storage medium. Background Art

[0003] When identifying regions of interest (ROIs), commonly used face detection models experience flickering when inferring face regions. This occurs because some frames are not recognized during the recognition process, causing the recognition frame to flicker. This can lead to inaccurate region recognition and flickering ROI quality, which in turn affects subsequent video rendering. Therefore, a stable face recognition method for video encoding ROIs is urgently needed to stably identify face regions in each frame and eliminate flickering.

[0004] Summary of the Invention

[0005] In order to solve the above-mentioned problems existing in the prior art, the embodiments of the present application provide a face area recognition method, device, electronic device and storage medium, which can accurately and stably identify the face area of ​​each frame in the video and prevent flickering from occurring.

[0006] In a first aspect, embodiments of the present application provide a method for recognizing a facial region, the method comprising:

[0007] Performing frame processing on the video to be processed to obtain at least one frame of video image;

[0008] Parsing each frame of the at least one frame of video image to obtain at least one compressed domain information, wherein the at least one compressed domain information corresponds one-to-one to the at least one frame of video image;

[0009] determining a frame type and a reference frame image of each frame of the video image according to each compressed domain information of the at least one compressed domain information;

[0010] Determining a recognition method for each frame of video image according to a frame type of each frame of video image;

[0011] According to the recognition method of each frame of video image and the reference frame image of each frame of video image, the face region of each frame of video image is recognized to obtain the face region of each frame of video image.

[0012] In a possible implementation manner, the frame type includes: a frame type without reference and a frame type with reference.

[0013] In a possible implementation, when the frame type is a reference frame type, face region recognition is performed on each video frame according to the recognition method for each video frame and the reference frame of each video frame to obtain the face region of each video frame, including:

[0014] Obtaining first compressed domain information of a face region in a reference frame image of each frame of video image;

[0015] Obtaining a region offset according to the first compression domain information;

[0016] Performing a region offset process on the face region in the reference frame image of each frame of video image according to the region offset amount to obtain a first offset region;

[0017] Acquire second compressed domain information of a first correction region in each frame of video image, wherein a position of the first correction region in each frame of video image is the same as a position of the first offset region in a reference frame image of each frame of video image;

[0018] determining the credibility of the first correction region based on the second compressed domain information;

[0019] When the credibility is credible, the first correction area is determined as the face area of ​​each frame of video image.

[0020] In a possible implementation, obtaining the region offset according to the first compression domain information includes:

[0021] determining, based on the first compression domain information, a motion vector within a face region in a reference frame image of each frame of the video image;

[0022] The average value of the motion vectors is used as the region offset.

[0023] In a possible implementation, determining the credibility of the first correction region according to the second compressed domain information includes:

[0024] Determining a percentage value occupied by the Intra block in the first correction area according to the second compression domain information;

[0025] determining an average residual of the motion vectors in the first correction region based on the second compressed domain information;

[0026] When the percentage value is smaller than the first threshold value and the average residual is smaller than the second threshold value, the credibility of the first correction region is determined to be credible.

[0027] In a possible implementation manner, the reference frame types include: a unidirectional reference frame type and a bidirectional reference frame type.

[0028] In a possible implementation, when the frame type is an intra-frame coding type, face region recognition is performed on each video frame according to a recognition method for each video frame and a reference frame for each video frame to obtain a face region for each video frame, including:

[0029] Determining a first resolution according to an original resolution of the video to be processed;

[0030] Perform face detection on each frame of video image according to the first resolution to determine a face area of ​​each frame of video image.

[0031] In a possible implementation, the first resolution and the original resolution can be expressed by formula ①:

[0032] Among them, A1 represents the first resolution, A P Indicates native resolution.

[0033] In a possible implementation manner, the non-reference frame type includes: an intra-frame coding type.

[0034] In a second aspect, an embodiment of the present application provides a face region recognition device, comprising:

[0035] A frame segmentation module, used for performing frame processing on the video to be processed to obtain at least one frame of video image;

[0036] a parsing module, configured to parse each frame of the at least one frame of video image to obtain at least one piece of compression domain information, wherein the at least one piece of compression domain information corresponds one-to-one to the at least one frame of video image, and determine a frame type and a reference frame image of each frame of the video image based on each piece of compression domain information in the at least one piece of compression domain information;

[0037] The processing module is used to determine the recognition method of each frame of video image according to the frame type of each frame of video image, and to perform face area recognition on each frame of video image according to the recognition method of each frame of video image and the reference frame image of each frame of video image to obtain the face area of ​​each frame of video image.

[0038] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, the processor being connected to a memory, the memory being used to store computer programs, and the processor being used to execute the computer programs stored in the memory, so that the electronic device performs the method of the first aspect.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method of the first aspect.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and is computer-operable to enable the computer to execute the method of the first aspect.

[0041] The implementation of the present application has the following beneficial effects:

[0042] In the embodiment of the present application, the video to be processed is divided into frames, and then the frame type and reference frame are determined by the compression domain information of each frame, and then the corresponding recognition method is further determined by its frame type. Then, the matched recognition method is combined with the corresponding reference frame and compression domain information to accurately recognize the face of each frame. Thus, the compression domain information obtained during the video encoding ROI process is directly used, combined with the reference frame of each frame, to accurately and stably recognize the face area in each frame of the video, and prevent the occurrence of flickering. In addition, this method does not need to recalculate the relationship between the two frames, which can reduce computing costs and improve recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0044] FIG1 is a schematic diagram of the hardware structure of a face region recognition device provided in an embodiment of the present application;

[0045] FIG2 is a flow chart of a face region recognition method according to an embodiment of the present application;

[0046] FIG3 is a reference frame image superimposed with motion vector information in a compressed domain, provided in an embodiment of the present application;

[0047] FIG4 is a block diagram of the functional modules of a face region recognition device provided in an embodiment of the present application;

[0048] FIG5 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0050] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0051] Reference herein to an "embodiment" means that a particular feature, result, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0052] 1 , which is a hardware schematic diagram of a face region recognition device according to an embodiment of the present invention. The face region recognition device 100 includes at least one processor 101 , a communication circuit 102 , a memory 103 , and at least one communication interface 104 .

[0053] In this embodiment, the processor 101 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0054] The communication link 102 may include a path for transmitting information between the above components.

[0055] The communication interface 104 may be any transceiver-like device (eg, antenna, etc.) for communicating with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc.

[0056] The memory 103 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0057] In this embodiment, the memory 103 can exist independently and be connected to the processor 101 via the communication line 102. The memory 103 can also be integrated with the processor 101. The memory 103 provided in the embodiments of the present application can generally be non-volatile. Among them, the memory 103 is used to store computer-executable instructions for executing the solution of the present application, and the execution is controlled by the processor 101. The processor 101 is used to execute the computer-executable instructions stored in the memory 103, thereby implementing the methods provided in the following embodiments of the present application.

[0058] In an optional implementation, the computer-executable instructions may also be referred to as application code, which is not specifically limited in this application.

[0059] In an optional embodiment, the processor 101 may include one or more CPUs, such as CPU0 and CPU1 in FIG. 1 .

[0060] In an optional embodiment, the face region recognition apparatus 100 may include multiple processors, such as the processor 101 and the processor 107 in FIG1 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0061] In an optional embodiment, if the facial region recognition device 100 is a server, for example, it can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The facial region recognition device 100 may also include an output device 105 and an input device 106. The output device 105 communicates with the processor 101 and can display information in a variety of ways. For example, the output device 105 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 106 communicates with the processor 101 and can receive user input in a variety of ways. For example, the input device 106 can be a mouse, keyboard, touch screen device, or sensor device.

[0062] The above-mentioned face region recognition device 100 can be a general device or a dedicated device. The embodiment of the present application does not limit the type of the face region recognition device 100.

[0063] The following describes a method for face region recognition disclosed in this application:

[0064] Refer to Figure 2, which is a flow chart of a face region recognition method provided by an embodiment of the present application. The face region recognition method includes the following steps:

[0065] 201: Perform frame processing on the video to be processed to obtain at least one frame of video image.

[0066] 202: Analyze each video frame in at least one video frame to obtain at least one compressed domain information.

[0067] In this embodiment, the at least one compressed domain information corresponds one-to-one with at least one frame of video image. Specifically, the compressed domain information may include information such as the frame type, reference frame, motion vector information, number of intra blocks, number of skip blocks, and average residual of each frame of video image. This compressed domain information is generated during the decoding process of the video to be processed, and decoding the video is a necessary step in the video encoding ROI processing process. Therefore, when encoding the ROI of the video to be processed, the compressed domain information generated during decoding can be directly obtained for subsequent use, which can improve the recognition efficiency of the method while reducing computing power consumption.

[0068] 203: Determine a frame type and a reference frame image of each frame of the video image according to each compression domain information of the at least one compression domain information.

[0069] In this embodiment, the frame type may include a frame type without reference and a frame type with reference. Among them, the frame type without reference may include an intra-frame coding type frame I frame. This type of frame image is usually a scene switching frame, that is, a frame that switches from one scene to another and serves as the beginning of another scene. For an I frame, as the first frame starting a scene, there are no other frames that can be used as references to assist in its parsing, that is, the reference frame of the I frame is empty. The frame type with reference may include: a unidirectional reference frame type frame P frame and a bidirectional reference frame type frame B frame. Among them, the reference frame of the P frame is a single frame, that is, another frame in the same scene, and the reference frame of the B frame is a double frame, that is, another two frames in the same scene.

[0070] In this embodiment, the frame type and reference frame image of each frame of video image are recorded in its corresponding compression domain information. Based on this, the frame type and reference frame of the frame of video image can be determined by querying the compression domain information corresponding to the frame of video image.

[0071] 204: Determine a recognition method for each frame of video image according to the frame type of each frame of video image.

[0072] 205: Performing facial region recognition on each video frame according to the recognition method for each video frame and the reference frame image of each video frame to obtain the facial region of each video frame.

[0073] In this embodiment, since I-frames lack a reference frame, facial region recognition can only be performed based on their own information. Furthermore, since I-frames serve as reference frames for other video frames in the same scene, to ensure accurate facial recognition in subsequent video frames, in this embodiment, a first resolution is determined based on the original resolution of the video to be processed. Face detection is then performed on the I-frame video image at this first resolution to determine the facial region within the I-frame video image.

[0074] Specifically, the first resolution and the original resolution can be expressed by formula ②:

[0075] Among them, A1 represents the first resolution, A P Indicates native resolution.

[0076] That is, the first resolution is any value between half of the original resolution and the original resolution. Therefore, by performing face recognition on the video image of frame I at a higher resolution, while ensuring the recognition accuracy, it can provide a high-precision reference frame for the face recognition of subsequent frames of video images, thereby further improving the recognition accuracy of the face area of ​​the entire video to be processed.

[0077] In this embodiment, for a P frame, first, first compression domain information of the face region in the reference frame image of the video image of the P frame can be obtained, and then the region offset can be obtained based on the first compression domain information. Then, region offset processing is performed on the face region in the reference frame image of each frame of the video image based on the region offset to obtain a first offset region. Then, second compression domain information of the first correction region in the video image of the P frame is obtained, wherein the position of the first correction region in each frame of the video image is the same as the position of the first offset region in the reference frame image of each frame of the video image. Finally, based on the second compression domain information, the credibility of the first correction region is determined, and if the credibility is credible, the first correction region is determined as the face region of the video image of the P frame.

[0078] Specifically, in this embodiment, if the reference frame of the P-frame video image is an I-frame, the face region identified by face detection in the I-frame can be used as the face region in the reference frame image of the P-frame video image, and compressed domain information of the region can be obtained. If the reference frame of the P-frame video image is not an I-frame, the reference frame can be processed first until a video image with an I-frame as the reference frame is found. The face region of each frame can then be determined step by step until the face region of the reference frame of the P-frame video image is determined, and then the next step of processing can be performed.

[0079] In this embodiment, the motion vector in the face area of ​​the reference frame image of the P frame video image can be determined based on the first compression domain information, and then the average value of the motion vector is used as the area offset. Specifically, the motion vector information may include direction and size, and each pixel in the image has a corresponding motion vector for identifying the motion trend of the pixel. As shown in Figure 3, Figure 3 shows a reference frame image superimposed with the motion vector information in the compression domain, wherein the area framed by the box is the face area. It can be seen that each pixel in the area has a motion vector for identifying the motion trend of the pixel in the next video, and then the average value of the motion vector in the area can be regarded as the motion trend of the area in the next video, that is, the area offset. Then, by moving the face area by the area offset, the predicted position of the face area in the P frame video image can be obtained, that is, the first offset area.

[0080] Then, in this embodiment, the first offset area is projected into the video image of the P frame to determine an area with the same position in the video image of the P frame as the first correction area, and the first correction area is the area of ​​the suspected face area in the video image of the P frame.

[0081] In this embodiment, once the first correction region is determined, its identity as a facial region can be verified, i.e., its credibility can be determined. Specifically, based on the compression sum information of the first correction region, i.e., the second compression domain information, the number of Intra blocks and Skip blocks in the first correction region can be determined. This percentage can then be determined. This percentage can be expressed using Formula ③:

[0082] Among them, P I Indicates percentage value, L I Indicates the number of Intra blocks in the first correction area, L S Indicates the number of Skip blocks in the first correction area.

[0083] At the same time, the second compressed domain information is queried to determine the average residual of the motion vector in the first correction area.

[0084] In this embodiment, when the percentage value is less than the first threshold and the average residual is less than the second threshold, the credibility of the first correction area can be determined to be credible, that is, the first correction area can be determined to be the face area of ​​the video image of the P frame.

[0085] In this embodiment, for B frames, since they have two corresponding reference frames, the two reference frames of the B frame can be processed separately according to the above-mentioned processing method for P frames, and then the two first correction areas determined by the two reference frames with the highest credibility are used as the face area of ​​the video image of the B frame. Generally speaking, when the percentage value is lower than the first threshold and the lower it is, and the average residual is lower than the second threshold and the lower it is, the higher its credibility is. In addition, the overlapping area or the minimum circumscribed area of ​​the two first correction areas can be used as a new first correction area for subsequent credibility verification, and then the face area in the video image of the B frame is determined. This application does not impose any restrictions on this.

[0086] Thus, by performing face detection at high resolution and obtaining an I-frame of the face area as a reference frame, and combining the relationship between the reference frames in the compressed domain information, the face area of ​​each subsequent frame of the video image is determined. This allows for face recognition in the entire video to be processed, requiring only the I-frame of each scene to be detected, without requiring face detection on other frames, to achieve stable face detection for the entire video. This saves significant computational cost while ensuring successful recognition of the face area in each frame and preventing flickering. Furthermore, since face recognition only requires the I-frame, high-resolution face detection can then be performed on the I-frame, further ensuring accuracy in face area recognition while also ensuring recognition efficiency.

[0087] In summary, in the face region recognition method provided by the present invention, the video to be processed is divided into frames, and then the frame type and reference frame are determined by the compression domain information of each frame image, and then the corresponding recognition method is further determined by its frame type. Then, the matched recognition method is combined with the corresponding reference frame and compression domain information to accurately recognize the face of each frame image. Thus, the compression domain information obtained in the video encoding ROI process is directly used, combined with the reference frame of each frame image, to accurately and stably recognize the face area in each frame image in the video, and prevent the occurrence of flickering. In addition, through this method, there is no need to recalculate the relationship between the two frames, which can reduce computing costs and improve recognition efficiency.

[0088] Refer to Figure 4, which is a functional module block diagram of a face region recognition device provided in an embodiment of the present application. As shown in Figure 4, the face region recognition device 400 includes:

[0089] The framing module 401 is used to perform frame processing on the video to be processed to obtain at least one frame of video image;

[0090] a parsing module 402 configured to parse each of the at least one frame of video images to obtain at least one piece of compression domain information, wherein the at least one piece of compression domain information corresponds one-to-one to the at least one frame of video images, and determine a frame type and a reference frame image of each frame of video images based on each piece of compression domain information in the at least one piece of compression domain information;

[0091] The processing module 403 is used to determine the recognition method of each video frame according to the frame type of each video frame, and perform face area recognition on each video frame according to the recognition method of each video frame and the reference frame image of each video frame to obtain the face area of ​​each video frame.

[0092] In an embodiment of the present invention, the frame type includes: a frame type without reference and a frame type with reference.

[0093] In an embodiment of the present invention, when the frame type is a reference frame type, in terms of performing face region recognition on each video frame according to the recognition method for each video frame and the reference frame of each video frame to obtain the face region of each video frame, the processing module 403 is specifically configured to:

[0094] Obtaining first compressed domain information of a face region in a reference frame image of each frame of video image;

[0095] Obtaining a region offset according to the first compression domain information;

[0096] Performing a region offset process on the face region in the reference frame image of each frame of video image according to the region offset amount to obtain a first offset region;

[0097] Acquire second compressed domain information of a first correction region in each frame of video image, wherein a position of the first correction region in each frame of video image is the same as a position of the first offset region in a reference frame image of each frame of video image;

[0098] determining the credibility of the first correction region based on the second compressed domain information;

[0099] When the credibility is credible, the first correction area is determined as the face area of ​​each frame of video image.

[0100] In an embodiment of the present invention, in terms of obtaining the region offset according to the first compressed domain information, the processing module 403 is specifically configured to:

[0101] determining, based on the first compression domain information, a motion vector within a face region in a reference frame image of each frame of the video image;

[0102] The average value of the motion vectors is used as the region offset.

[0103] In an embodiment of the present invention, in determining the credibility of the first correction region based on the second compression domain information, the processing module 403 is specifically configured to:

[0104] Determining a percentage value occupied by the Intra block in the first correction area according to the second compression domain information;

[0105] determining an average residual of the motion vectors in the first correction region based on the second compressed domain information;

[0106] When the percentage value is smaller than the first threshold value and the average residual is smaller than the second threshold value, the credibility of the first correction region is determined to be credible.

[0107] In an embodiment of the present invention, the reference frame types include: a unidirectional reference frame type and a bidirectional reference frame type.

[0108] In an embodiment of the present invention, when the frame type is a reference-free frame type, in terms of performing face region recognition on each frame of video image according to the recognition method for each frame of video image and the reference frame of each frame of video image, and obtaining the face region of each frame of video image, the processing module 403 is specifically configured to:

[0109] Determining a first resolution according to an original resolution of the video to be processed;

[0110] Perform face detection on each frame of video image according to the first resolution to determine a face area of ​​each frame of video image.

[0111] In an embodiment of the present invention, the first resolution and the original resolution can be expressed by formula ④:

[0112] Among them, A1 represents the first resolution, A P Indicates native resolution.

[0113] In an embodiment of the present invention, the non-reference frame type includes: an intra-frame coding type.

[0114] Referring to Figure 5 , Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in Figure 5 , electronic device 500 includes a transceiver 501, a processor 502, and a memory 503. These are connected via a bus 504. Memory 503 is used to store computer programs and data and can transmit data stored in memory 503 to processor 502.

[0115] The processor 502 is configured to read the computer program in the memory 503 and perform the following operations:

[0116] Performing frame processing on the video to be processed to obtain at least one frame of video image;

[0117] Parsing each frame of the at least one frame of video image to obtain at least one compressed domain information, wherein the at least one compressed domain information corresponds one-to-one to the at least one frame of video image;

[0118] determining a frame type and a reference frame image of each frame of the video image according to each compressed domain information of the at least one compressed domain information;

[0119] Determining a recognition method for each frame of video image according to a frame type of each frame of video image;

[0120] According to the recognition method of each frame of video image and the reference frame image of each frame of video image, the face region of each frame of video image is recognized to obtain the face region of each frame of video image.

[0121] In an embodiment of the present invention, the frame type includes: a frame type without reference and a frame type with reference.

[0122] In an embodiment of the present invention, when the frame type is a reference frame type, in terms of performing face region recognition on each video frame according to the recognition method for each video frame and the reference frame of each video frame to obtain the face region of each video frame, the processor 502 is specifically configured to perform the following operations:

[0123] Obtaining first compressed domain information of a face region in a reference frame image of each frame of video image;

[0124] Obtaining a region offset according to the first compression domain information;

[0125] Performing a region offset process on the face region in the reference frame image of each frame of video image according to the region offset amount to obtain a first offset region;

[0126] Acquire second compressed domain information of a first correction region in each frame of video image, wherein a position of the first correction region in each frame of video image is the same as a position of the first offset region in a reference frame image of each frame of video image;

[0127] determining the credibility of the first correction region based on the second compressed domain information;

[0128] When the credibility is credible, the first correction area is determined as the face area of ​​each frame of video image.

[0129] In the embodiment of the present invention, in terms of obtaining the region offset according to the first compressed domain information, the processor 502 is specifically configured to perform the following operations:

[0130] determining, based on the first compression domain information, a motion vector within a face region in a reference frame image of each frame of the video image;

[0131] The average value of the motion vectors is used as the region offset.

[0132] In the embodiment of the present invention, in determining the credibility of the first correction region according to the second compression domain information, the processor 502 is specifically configured to perform the following operations:

[0133] Determining a percentage value occupied by the Intra block in the first correction area according to the second compression domain information;

[0134] determining an average residual of the motion vectors in the first correction region based on the second compressed domain information;

[0135] When the percentage value is smaller than the first threshold value and the average residual is smaller than the second threshold value, the credibility of the first correction region is determined to be credible.

[0136] In an embodiment of the present invention, the reference frame types include: a unidirectional reference frame type and a bidirectional reference frame type.

[0137] In an embodiment of the present invention, when the frame type is a reference-free frame type, in terms of performing face region recognition on each video frame according to the recognition method for each video frame and the reference frame of each video frame to obtain the face region of each video frame, the processor 502 is specifically configured to perform the following operations:

[0138] Determining a first resolution according to an original resolution of the video to be processed;

[0139] Perform face detection on each frame of video image according to the first resolution to determine a face area of ​​each frame of video image.

[0140] In an embodiment of the present invention, the first resolution and the original resolution can be expressed by formula ⑤:

[0141] Among them, A1 represents the first resolution, A P Indicates native resolution.

[0142] In an embodiment of the present invention, the non-reference frame type includes: an intra-frame coding type.

[0143] It should be understood that the facial region recognition device in this application may include a smartphone (such as an Android phone, iOS phone, Windows Phone phone, etc.), a tablet computer, a PDA, a laptop computer, a mobile Internet device (MID), a robot, or a wearable device. The above facial region recognition device is only an example and not an exhaustive list, including but not limited to the above facial region recognition device. In actual applications, the above facial region recognition device may also include: a smart vehicle terminal, a computer device, etc.

[0144] Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by combining software with a hardware platform. Based on this understanding, all or part of the contribution of the technical solution of the present invention to the background art can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0145] Therefore, embodiments of the present application further provide a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement some or all of the steps of any of the face region recognition methods described in the above method embodiments. For example, the storage medium may include a hard disk, a floppy disk, an optical disk, a magnetic tape, a magnetic disk, a USB flash drive, a flash memory, etc.

[0146] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute part or all of the steps of any face area recognition method described in the above method embodiment.

[0147] It should be noted that for the aforementioned method implementations, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the implementations described in the specification are all optional implementations, and the actions and modules involved are not necessarily required for this application.

[0148] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0150] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0151] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0152] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0153] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0154] The above is a detailed introduction to the implementation methods of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above implementation methods is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for recognizing a face region, characterized in that: The method comprises: Perform frame processing on the video to be processed to obtain at least one frame of video image; Parsing each frame of the at least one frame of video image to obtain at least one compression domain information, wherein the at least one compression domain information corresponds to the at least one frame of video image in one-to-one correspondence; Determine a frame type and a reference frame image of each frame of the video image according to each compression domain information of the at least one compression domain information; Determining a recognition method for each frame of the video image according to a frame type of each frame of the video image; According to the recognition method for each frame of video image and the reference frame image of each frame of video image, face region recognition is performed on each frame of video image to obtain the face region of each frame of video image.

2. The method according to claim 1, characterized in that The frame types include: a frame type without reference and a frame type with reference.

3. The method according to claim 2, characterized in that When the frame type is the reference frame type, performing face region recognition on each frame of video image according to the recognition method for each frame of video image and the reference frame of each frame of video image to obtain the face region of each frame of video image includes: Acquire first compression domain information of a face area in a reference frame image of each frame of video image; Acquire a region offset according to the first compression domain information; Performing a region offset process on the face region in the reference frame image of each frame of the video image according to the region offset amount to obtain a first offset region; Acquire second compression domain information of a first correction region in each frame of video image, wherein a position of the first correction region in each frame of video image is the same as a position of the first offset region in a reference frame image of each frame of video image; determining the credibility of the first correction area according to the second compression domain information; When the credibility is credible, the first correction area is determined as the face area of ​​each frame of the video image.

4. The method according to claim 3, characterized in that The acquiring the region offset according to the first compression domain information includes: Determine, according to the first compression domain information, a motion vector in a face region in a reference frame image of each frame of the video image; The average value of the motion vectors is used as the region offset.

5. The method according to claim 3 or 4, characterized in that: The determining the credibility of the first correction area according to the second compression domain information includes: Determine, according to the second compression domain information, a percentage value occupied by the Intra block in the first correction area; determining an average residual of motion vectors in the first correction area according to the second compressed domain information; When the percentage value is smaller than a first threshold value, and the average residual is smaller than a second threshold value, it is determined that the credibility of the first correction region is credible.

6. The method according to claim 2, characterized in that The reference frame types include: unidirectional reference frame types and bidirectional reference frame types.

7. The method according to claim 2, characterized in that When the frame type is the non-reference frame type, the step of performing face region recognition on each frame of video image according to the recognition method for each frame of video image and the reference frame of each frame of video image to obtain the face region of each frame of video image includes: Determining a first resolution according to the original resolution of the video to be processed; Perform face detection on each frame of the video image according to the first resolution to determine a face area of ​​each frame of the video image.

8. The method according to claim 7, characterized in that The first resolution and the original resolution satisfy the following formula: Wherein, A1 represents the first resolution, A P Indicates the native resolution.

9. The method according to claim 2, characterized in that: The non-reference frame type includes: an intra-frame coding type.

10. A face region recognition device, characterized in that: The device comprises: A frame segmentation module, used for performing frame segmentation processing on the video to be processed to obtain at least one frame of video image; a parsing module, configured to parse each frame of the at least one frame of video image to obtain at least one compression domain information, wherein the at least one compression domain information corresponds to the at least one frame of video image one by one, and determine a frame type and a reference frame image of each frame of video image according to each compression domain information in the at least one compression domain information; A processing module is used to determine a recognition method for each frame of video image according to a frame type of each frame of video image, and to perform face region recognition on each frame of video image according to the recognition method for each frame of video image and a reference frame image of each frame of video image to obtain a face region of each frame of video image.

11. An electronic device, characterized in that: The method comprises a processor, a memory, a communication interface and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the one or more programs include instructions for executing the steps in the method described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 9.

Images