A face capture and recognition method in dense scenes

Abstract

The present invention relates to the technical field of face capture recognition, and discloses a face capture recognition method in a dense scene, the method comprising: performing face segmentation on a dense scene video frame to obtain a face area; super-resolution reconstruction on the face area; posture correction on the face area sub-image after super-resolution reconstruction; identity association recognition on the corrected face area sub-image to obtain a face capture recognition result. The present invention combines the motion information, brightness information and chromaticity component of the pixel to determine whether the pixel is a skin color pixel point, and uses the skin color pixel point as a seed point for region growth to achieve face area recognition, and uses a convex set projection method to extract the scale information of the pixel under different features, to form more abundant reconstruction projection information for high-resolution reconstruction of the face area, and performs posture correction and identity association recognition processing to obtain a face capture image sequence of the face in a dense scene.

Description

Technical Field

[0001] The present invention relates to the technical field of face capture and recognition, and in particular to a face capture and recognition method in a dense scene. Background Art

[0002] In recent years, face recognition technology has made great progress. Through deep learning technologies such as convolutional neural networks (CNNs), face recognition has achieved remarkable results in image classification, feature extraction, identity authentication and other fields. Especially in the field of security monitoring, face capture and recognition technology has gradually been applied to high-density crowd scenes, such as airports, stations, commercial centers, etc. However, as a specific application scenario, the railway station faces particularly complex technical challenges. The high-density scenes in railway stations are different from general low-density or controlled environments, and their face capture and recognition tasks face dynamically changing environments and complex visual conditions. Therefore, how to effectively improve the accuracy and real-time performance of face capture and recognition in railway stations has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present invention provides a face capture and recognition method in a dense scene, which combines the motion information, brightness information and chromaticity component of the pixel to judge whether the pixel is a skin color pixel, and uses the skin color pixel as a seed point to perform regional growth, and judges whether to continue growing according to the proportion of skin color pixels in the grown region, so as to avoid arms and other regions being used as face regions, realize face region recognition, extract the scale information of pixels under different features by convex set projection, construct richer reconstruction projection information to perform high-resolution reconstruction of low-resolution face regions, and perform posture correction and identity association recognition processing to obtain a face capture image sequence of the face in a dense scene.

[0004] To achieve the above object, the present invention provides a method for face capture and recognition in a dense scene, comprising the following steps:

[0005] S1: Acquire a dense scene video frame in a dense scene, perform face segmentation on the dense scene video frame, and obtain a face region, wherein the face segmentation adopts a region growing detection method;

[0006] S2: Perform super-resolution reconstruction on the face region to obtain a sub-image of the face region after super-resolution reconstruction, wherein the super-resolution reconstruction adopts an improved convex set projection algorithm;

[0007] S3: performing posture correction on the face region sub-image to obtain a corrected face region sub-image;

[0008] S4: Perform identity association recognition on the corrected face region sub-image to obtain a face capture recognition result.

[0009] As a further improvement method of the present invention:

[0010] Optionally, the step S1 of acquiring a dense scene video frame in a dense scene includes:

[0011] The dense scene is photographed using a video shooting device arranged in a dense scene, and the shooting result is used as a dense scene video frame, where the dense scene video frame is in the form of a pixel matrix, and the representation form of the obtained dense scene video frame is:

[0012] ;

[0013] ;

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] in:

[0023] represents the acquired dense scene video frame sequence, represents the acquired nth dense scene video frame, , N represents the total number of dense scene video frames acquired;

[0024] Represents the pixel in the xth row and yth column of the nth dense scene video frame Pixel information, , , X represents the number of pixel rows in the dense scene video frame, and Y represents the number of pixel columns in the dense scene video frame;

[0025] Represents dense scene video frames in sequence Medium Pixels Luminance information, blue chrominance component, and red chrominance component;

[0026] Represents dense scene video frames in sequence Medium Pixels exist Color values ​​of color channels;

[0027] Represents dense scene video frames Medium Pixels sports information; Represents dense scene video frames Medium Pixels The gray value of Represents dense scene video frames Medium Pixels The grayscale change value of the adjacent frame, Indicates the preset grayscale change threshold;

[0028] Represents the AND operator. If If both are 1, then is 1, otherwise is 0;

[0029] Perform face segmentation on dense scene video frames to obtain face regions in dense scene video frames, where dense scene video frames The face area in ,Face segmentation uses region growing detection method.

[0030] Optionally, for dense scene video frames Perform face segmentation to obtain dense scene video frames The face area in ,include:

[0031] S11: Dense scene video frames The skin color pixel point detection is performed on the pixels in the dense scene video frame to obtain the skin color pixel point detection result of the pixel. Medium Pixels The skin color pixel detection result is , It represents a dense scene video frame Medium Pixels is the skin color pixel, It represents a dense scene video frame Medium Pixels Not skin color pixels:

[0032] ;

[0033] in:

[0034] represents an exponential function with a natural constant as base;

[0035] Represents the chromaticity control parameter;

[0036] S12: taking any skin color pixel as a seed point;

[0037] S13: With the seed point as the center, scan eight neighboring pixels of the seed point in a clockwise direction, and merge the eight neighboring pixels and the seed point into a rectangular pixel area;

[0038] S14: Calculate the proportion of skin color pixels in the rectangular pixel area. If the proportion is lower than a preset threshold, return to step S12 to reselect seed points and output the current rectangular pixel area as a group of face pixel areas. Otherwise, use the skin color pixels and the neighboring pixels of the skin color pixels in the rectangular pixel area as seed points, return to step S13, and expand the rectangular pixel area.

[0039] S15: Repeat steps S12 to S14 until all skin color pixels are added to the face pixel area, and the output face pixel area constitutes the face area ,in Represents dense scene video frames The Individual face pixel area, Represents dense scene video frames In the embodiment of the present invention, the face pixel areas whose pixel numbers are lower than a preset threshold are filtered.

[0040] Optionally, the super-resolution reconstruction of the face region in step S2 includes:

[0041] The face area in the dense scene video frame is super-reconstructed, and the super-reconstruction adopts an improved convex set projection algorithm to obtain the face area sub-image after super-reconstruction. The super-resolution reconstruction process is:

[0042] S21: From the face area Extract the face pixel area, where the extracted The pixel area of ​​a face is :

[0043] ;

[0044] ;

[0045] in:

[0046] Represents the face pixel area Middle Row, column b pixel Pixel information, ,in Represents the face pixel area The number of pixel rows, Represents the face pixel area The number of pixel columns;

[0047] Represents the face pixel area in turn Medium Pixels Luminance information, blue chrominance component, red chrominance component and motion information;

[0048] S22: extracting pixel information of pixels in the face pixel area, and performing convex set projection on the pixel information to obtain reconstructed projection information of the pixels; wherein the pixel information The convex set projection process is:

[0049] S221: Using planar convex sets to align pixels The pixel coordinates are projected, where the projection result is :

[0050] ;

[0051] in:

[0052] Represents the normal vector of a convex set in a plane;

[0053] S222: Using convex sets to identify pixel information Projection is performed, where the projection result is :

[0054] ;

[0055] ;

[0056] in:

[0057] represents the projection formula of the j-th convex set, Indicates that As the projection formula The input value and the output projection result; ;

[0058] represents the center coordinates of the jth convex set;

[0059] represents the radius of the jth convex set;

[0060] represents the L1 norm;

[0061] S223: Reconstruction projection information of constituent pixels :

[0062] ;

[0063] in:

[0064] Showing face pixel area Medium Pixels The scale information, represents the scale factor, Set to 2;

[0065] T represents transpose, W represents the grayscale processing vector;

[0066] Indicates normalization processing, which processes the reconstructed projection information into a vector in the range of 0-1;

[0067] The reconstructed projection information of the pixel is the reconstructed projection gray value of the pixel;

[0068] S23: Super-resolution correction is performed on the face pixel area in combination with the reconstructed projection information of the pixels, and the super-resolution correction result is used as the face region sub-image after super-resolution reconstruction to form the face region A collection of face region sub-images ,in The face pixel area The corresponding face area sub-image.

[0069] Optionally, in step S23, super-resolution correction is performed on the face pixel region in combination with the reconstructed projection information of the pixels, and the super-resolution correction result is used as a face region sub-image after super-resolution reconstruction, including:

[0070] S231: Obtaining the resolution index of super-resolution correction ,in Represents the number of pixel rows of the super-resolution image. Indicates the number of pixel columns of the super-resolution corrected image;

[0071] S232: According to the resolution index The reconstructed projection information of the pixel is corrected; wherein the reconstructed projection information The correction formula is:

[0072] ;

[0073] in:

[0074] represents the correction factor;

[0075] Represents the reconstruction projection information The correction result of

[0076] S233: Using the corrected reconstructed projection information as the grayscale value of the pixel to form a grayscale face pixel area;

[0077] If the size of the grayscale face pixel area is lower than the resolution index, the grayscale face pixel area is enlarged to the size corresponding to the resolution index, and the bilinear interpolation method is used to complete the pixels, and the completion result is used as the face area sub-image after super-resolution reconstruction; otherwise, the grayscale face pixel area is directly used as the face area sub-image after super-resolution reconstruction, where The face pixel area The corresponding face area sub-image.

[0078] Optionally, performing posture correction on the face region sub-image in step S3 includes:

[0079] The posture correction is performed on the face area sub-image, where the face area sub-image The posture correction process is as follows:

[0080] S31: face area sub-image Mark the skin color pixels in the image, and mark other pixels as non-skin color pixels;

[0081] The face area sub-image The center of the face is taken as the face center, and the non-skin color pixel areas in the upper left part and the upper right part of the face center are taken as the left eye area and the right eye area;

[0082] S32: Connect the center of the left eye area and the center of the right eye area to obtain a face area sub-image The eye line connecting the two eyes; calculate the angle between the eye line and the horizontal line as the sub-image of the face area Roll angle of face pose ;

[0083] S33: Using the roll angle Construct the face area sub-image The affine matrix of :

[0084] ;

[0085] S34: According to the affine matrix For face area sub-image The pixel coordinates of the pixels in the image are affine transformed to obtain the face area sub-image after posture correction .

[0086] Optionally, the affine matrix For face area sub-image The pixel coordinates of the pixels in the image are transformed by an affine transformation, including:

[0087] Pixel coordinates The affine transformation formula is:

[0088] ;

[0089] ;

[0090] in:

[0091] is the pixel coordinate The affine transformation result of .

[0092] Optionally, performing identity association recognition on the corrected face region sub-image in step S4 includes:

[0093] The corrected face region sub-image is subjected to identity association recognition, wherein the identity association recognition process is as follows:

[0094] S41: Calculate the correlation coefficient between different corrected face region sub-images in adjacent dense scene video frames, where the dense scene video frames and In the figure, the rectified face region sub-image The correlation coefficient between :

[0095] ;

[0096] in:

[0097] express The pixel coordinates of the center;

[0098] express The mean value of the scale information of the pixels marked as skin color pixels in step S223;

[0099] represents the scale factor, Set to 2;

[0100] S42: taking two groups of corrected face region sub-images with correlation coefficients higher than a preset threshold as images of the same face appearing in adjacent dense scene video frames;

[0101] S43: repeating steps S41 to S42 to obtain a sequence of face region sub-images of different faces, wherein the sequence of face region sub-images represents a sequence of corrected face region sub-images of the same face in different dense scene videos;

[0102] A facial region sub-image sequence and a facial pixel region sequence of the same face are extracted as the face capture recognition result of the face in a dense scene.

[0103] In order to solve the above problem, the present invention provides an electronic device, the electronic device comprising:

[0104] A memory storing at least one instruction;

[0105] Communication interface, enabling electronic equipment to communicate; and

[0106] The processor executes the instructions stored in the memory to implement the above-mentioned face capture and recognition method in densely populated scenes.

[0107] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned face capture and recognition method in dense scenes.

[0108] Compared with the prior art, the present invention proposes a face capture and recognition method in a dense scene, which has the following advantages:

[0109] First, this scheme proposes a face region recognition method, which combines the motion information, brightness information and chromaticity components of the pixel to judge whether the pixel is a skin color pixel, and uses the skin color pixel as the seed point for regional growth. The judgement of whether to continue growing is based on the proportion of skin color pixels in the grown region, so as to avoid the arms and other regions being used as face regions, realize face region recognition, and use the convex set projection method to extract the scale information of the pixel under different motion information, brightness information and chromaticity information, to form richer reconstruction projection information, to perform high-resolution reconstruction of low-resolution face regions, and to improve the clarity of face capture.

[0110] At the same time, this scheme proposes a face posture correction and identity association recognition method, which constructs an affine matrix of a face area sub-image according to the deflection angle of both eyes in the face, and performs affine transformation on the face area sub-image, so that the face areas in different dense scene video frames are on the same horizontal line, thereby improving the accuracy of identity association recognition. In addition, the correlation coefficient between different corrected face area sub-images in adjacent dense scene video frames is calculated by combining pixel information and distance information between face area sub-images, thereby realizing recognition and retrieval of the same face in different dense scene video frames, forming a face capture image sequence of the face in dense scenes, and realizing capture and recognition of the face in dense scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 A flowchart of a face capture and recognition method in a dense scene provided by an embodiment of the present invention.

[0112] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0113] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0115] The embodiment of the present application provides a method for face capture and recognition in a dense scene. The execution subject of the face capture and recognition method in a dense scene includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the face capture and recognition method in a dense scene can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0116] Embodiment 1:

[0117] A method for face capture and recognition in a dense scene includes the following steps:

[0118] S1: Acquire a dense scene video frame in a dense scene, perform face segmentation on the dense scene video frame, and obtain a face area.

[0119] The step S1 of obtaining a dense scene video frame in a dense scene includes:

[0120] The dense scene is photographed using a video shooting device arranged in a dense scene, and the shooting result is used as a dense scene video frame, where the dense scene video frame is in the form of a pixel matrix, and the representation form of the obtained dense scene video frame is:

[0121] ;

[0122] ;

[0123] ;

[0124] ;

[0125] ;

[0126] ;

[0127] ;

[0128] ;

[0129] ;

[0130] ;

[0131] in:

[0132] represents the acquired dense scene video frame sequence, represents the acquired nth dense scene video frame, , N represents the total number of dense scene video frames acquired;

[0133] Represents the pixel in the xth row and yth column of the nth dense scene video frame Pixel information, , , X represents the number of pixel rows in the dense scene video frame, and Y represents the number of pixel columns in the dense scene video frame;

[0134] Represents dense scene video frames in sequence Medium Pixels Luminance information, blue chrominance component, and red chrominance component;

[0135] Represents dense scene video frames in sequence Medium Pixels exist Color values ​​of color channels;

[0136] Represents dense scene video frames Medium Pixels sports information; Represents dense scene video frames Medium Pixels The gray value of Represents dense scene video frames Medium Pixels The grayscale change value of the adjacent frame, Indicates the preset grayscale change threshold;

[0137] Represents the AND operator. If If both are 1, then is 1, otherwise is 0;

[0138] Perform face segmentation on dense scene video frames to obtain face regions in dense scene video frames, where dense scene video frames The face area in ,Face segmentation uses region growing detection method.

[0139] For dense scene video frames Perform face segmentation to obtain dense scene video frames The face area in ,include:

[0140] S11: Dense scene video frames The skin color pixel point detection is performed on the pixels in the dense scene video frame to obtain the skin color pixel point detection result of the pixel. Medium Pixels The skin color pixel detection result is , It represents a dense scene video frame Medium Pixels is the skin color pixel, It represents a dense scene video frame Medium Pixels Not skin color pixels:

[0141] ;

[0142] in:

[0143] represents an exponential function with a natural constant as base;

[0144] Represents the chromaticity control parameter;

[0145] S12: taking any skin color pixel as a seed point;

[0146] S13: With the seed point as the center, scan eight neighboring pixels of the seed point in a clockwise direction, and merge the eight neighboring pixels and the seed point into a rectangular pixel area;

[0147] S14: Calculate the proportion of skin color pixels in the rectangular pixel area. If the proportion is lower than a preset threshold, return to step S12 to reselect seed points and output the current rectangular pixel area as a group of face pixel areas. Otherwise, use the skin color pixels and the neighboring pixels of the skin color pixels in the rectangular pixel area as seed points, return to step S13, and expand the rectangular pixel area.

[0148] S15: Repeat steps S12 to S14 until all skin color pixels are added to the face pixel area, and the output face pixel area constitutes the face area ,in Represents dense scene video frames The Individual face pixel area, Represents dense scene video frames The total number of face pixel areas in .

[0149] S2: Perform super-resolution reconstruction on the face region to obtain a super-resolution reconstructed face region sub-image.

[0150] In the step S2, super-resolution reconstruction of the face region is performed, including:

[0151] The face area in the dense scene video frame is super-reconstructed, and the super-reconstruction adopts an improved convex set projection algorithm to obtain the face area sub-image after super-reconstruction. The super-resolution reconstruction process is:

[0152] S21: From the face area Extract the face pixel area, where the extracted The pixel area of ​​a face is :

[0153] ;

[0154] ;

[0155] in:

[0156] Represents the face pixel area Middle Row, column b pixel Pixel information, ,in Represents the face pixel area The number of pixel rows, Represents the face pixel area The number of pixel columns;

[0157] Represents the face pixel area in turn Medium Pixels Luminance information, blue chrominance component, red chrominance component and motion information;

[0158] S22: extracting pixel information of pixels in the face pixel area, and performing convex set projection on the pixel information to obtain reconstructed projection information of the pixels; wherein the pixel information The convex set projection process is:

[0159] S221: Using planar convex sets to align pixels The pixel coordinates are projected, where the projection result is :

[0160] ;

[0161] in:

[0162] Represents the normal vector of a convex set in a plane;

[0163] S222: Using convex sets to identify pixel information Projection is performed, where the projection result is :

[0164] ;

[0165] ;

[0166] in:

[0167] represents the projection formula of the j-th convex set, Indicates that As the projection formula The input value and the output projection result; ;

[0168] represents the center coordinates of the jth convex set;

[0169] represents the radius of the jth convex set;

[0170] represents the L1 norm;

[0171] S223: Reconstruction projection information of constituent pixels :

[0172] ;

[0173] in:

[0174] Showing face pixel area Medium Pixels The scale information, represents the scale factor, Set to 2;

[0175] T represents transpose, W represents the grayscale processing vector;

[0176] Indicates normalization processing, which processes the reconstructed projection information into a vector in the range of 0-1;

[0177] The reconstructed projection information of the pixel is the reconstructed projection gray value of the pixel;

[0178] S23: Super-resolution correction is performed on the face pixel area in combination with the reconstructed projection information of the pixels, and the super-resolution correction result is used as the face region sub-image after super-resolution reconstruction to form the face region A collection of face region sub-images ,in The face pixel area The corresponding face area sub-image.

[0179] In the step S23, super-resolution correction is performed on the face pixel area in combination with the reconstructed projection information of the pixels, and the super-resolution correction result is used as the face area sub-image after super-resolution reconstruction, including:

[0180] S231: Obtaining the resolution index of super-resolution correction ,in Represents the number of pixel rows of the super-resolution image. Indicates the number of pixel columns of the super-resolution corrected image;

[0181] S232: According to the resolution index The reconstructed projection information of the pixel is corrected; wherein the reconstructed projection information The correction formula is:

[0182] ;

[0183] in:

[0184] represents the correction factor;

[0185] Represents the reconstruction projection information The correction result of

[0186] S233: Using the corrected reconstructed projection information as the grayscale value of the pixel to form a grayscale face pixel area;

[0187] If the size of the grayscale face pixel area is lower than the resolution index, the grayscale face pixel area is enlarged to the size corresponding to the resolution index, and the bilinear interpolation method is used to complete the pixels, and the completion result is used as the face area sub-image after super-resolution reconstruction; otherwise, the grayscale face pixel area is directly used as the face area sub-image after super-resolution reconstruction, where The face pixel area The corresponding face area sub-image.

[0188] S3: Perform posture correction on the face region sub-image to obtain a corrected face region sub-image.

[0189] In the step S3, posture correction is performed on the face region sub-image, including:

[0190] The posture correction is performed on the face area sub-image, where the face area sub-image The posture correction process is as follows:

[0191] S31: face area sub-image Mark the skin color pixels in the image, and mark other pixels as non-skin color pixels;

[0192] The face area sub-image The center of the face is taken as the face center, and the non-skin color pixel areas in the upper left part and the upper right part of the face center are taken as the left eye area and the right eye area;

[0193] S32: Connect the center of the left eye area and the center of the right eye area to obtain a face area sub-image The eye line connecting the two eyes; calculate the angle between the eye line and the horizontal line as the sub-image of the face area Roll angle of face pose ;

[0194] S33: Using the roll angle Construct the face area sub-image The affine matrix of :

[0195] ;

[0196] S34: According to the affine matrix For face area sub-image The pixel coordinates of the pixels in the image are affine transformed to obtain the face area sub-image after posture correction .

[0197] According to the affine matrix For face area sub-image The pixel coordinates of the pixels in the image are transformed by an affine transformation, including:

[0198] Pixel coordinates The affine transformation formula is:

[0199] ;

[0200] ;

[0201] in:

[0202] is the pixel coordinate The affine transformation result of .

[0203] S4: Perform identity association recognition on the corrected face region sub-image to obtain a face capture recognition result.

[0204] In the step S4, identity association recognition is performed on the corrected face region sub-image, including:

[0205] The corrected face region sub-image is subjected to identity association recognition, wherein the identity association recognition process is as follows:

[0206] S41: Calculate the correlation coefficient between different corrected face region sub-images in adjacent dense scene video frames, where the dense scene video frames and In the figure, the rectified face region sub-image The correlation coefficient between :

[0207] ;

[0208] in:

[0209] express The pixel coordinates of the center;

[0210] express The mean value of the scale information of the pixels marked as skin color pixels in step S223;

[0211] represents the scale factor, Set to 2;

[0212] S42: taking two groups of corrected face region sub-images with correlation coefficients higher than a preset threshold as images of the same face appearing in adjacent dense scene video frames;

[0213] S43: repeating steps S41 to S42 to obtain a sequence of face region sub-images of different faces, wherein the sequence of face region sub-images represents a sequence of corrected face region sub-images of the same face in different dense scene videos;

[0214] A facial region sub-image sequence and a facial pixel region sequence of the same face are extracted as the face capture recognition result of the face in a dense scene.

[0215] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0216] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0217] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0218] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A face capture and recognition method in a dense scene, characterized in that: The method comprises: S1: Acquire a dense scene video frame in a dense scene, perform face segmentation on the dense scene video frame, and obtain a face area; For dense scene video frames Perform face segmentation to obtain dense scene video frames The face area in ,include: S11: Dense scene video frames The skin color pixel point detection is performed on the pixels in the dense scene video frame to obtain the skin color pixel point detection result of the pixel. Medium Pixels The skin color pixel detection result is , It represents a dense scene video frame Medium Pixels is the skin color pixel, It represents a dense scene video frame Medium Pixels Not skin color pixels; S12: taking any skin color pixel as a seed point; S13: With the seed point as the center, scan eight neighboring pixels of the seed point in a clockwise direction, and merge the eight neighboring pixels and the seed point into a rectangular pixel area; S14: Calculate the proportion of skin color pixels in the rectangular pixel area. If the proportion is lower than a preset threshold, return to step S12 to reselect seed points and output the current rectangular pixel area as a group of face pixel areas. Otherwise, use the skin color pixels and the neighboring pixels of the skin color pixels in the rectangular pixel area as seed points, return to step S13, and expand the rectangular pixel area. S15: Repeat steps S12 to S14 until all skin color pixels are added to the face pixel area, and the output face pixel area constitutes the face area ,in Represents dense scene video frames The Individual face pixel area, Represents dense scene video frames The total number of face pixel areas in the image; S2: Perform super-resolution reconstruction on the face region to obtain a super-resolution reconstructed face region sub-image; Face area The super-resolution reconstruction process is: S21: From the face area Extract the face pixel area; S22: extracting pixel information of pixels in the face pixel area, and performing convex set projection on the pixel information to obtain reconstructed projection information of the pixels; S23: Super-resolution correction is performed on the face pixel area in combination with the reconstructed projection information of the pixels, and the super-resolution correction result is used as the face region sub-image after super-resolution reconstruction to form the face region A collection of face region sub-images ,in The face pixel area The corresponding face area sub-image; S3: performing posture correction on the face region sub-image to obtain a corrected face region sub-image; S4: Perform identity association recognition on the corrected face region sub-image to obtain a face capture recognition result.

2. The face capture and recognition method in a dense scene as claimed in claim 1, characterized in that: The step S1 of obtaining a dense scene video frame in a dense scene includes: The dense scene is photographed using a video shooting device arranged in a dense scene, and the shooting result is used as a dense scene video frame, where the dense scene video frame is in the form of a pixel matrix, and the representation form of the obtained dense scene video frame is: in: represents the acquired dense scene video frame sequence, represents the acquired nth dense scene video frame, , N represents the total number of dense scene video frames acquired; Represents the pixel in the xth row and yth column of the nth dense scene video frame Pixel information, , , X represents the number of pixel rows in the dense scene video frame, and Y represents the number of pixel columns in the dense scene video frame; Represents dense scene video frames in sequence Medium Pixels Luminance information, blue chrominance component, and red chrominance component; Perform face segmentation on dense scene video frames to obtain face regions in dense scene video frames, where dense scene video frames The face area in .

3. The face capture and recognition method in a dense scene as claimed in claim 1, characterized in that: In the step S23, super-resolution correction is performed on the face pixel area in combination with the reconstructed projection information of the pixels, and the super-resolution correction result is used as the face area sub-image after super-resolution reconstruction, including: S231: Obtaining the resolution index of super-resolution correction ,in Represents the number of pixel rows of the super-resolution image. Indicates the number of pixel columns of the super-resolution corrected image; S232: According to the resolution index Correcting the reconstructed projection information of the pixel; S233: Using the corrected reconstructed projection information as the grayscale value of the pixel to form a grayscale face pixel area; If the size of the grayscale face pixel area is lower than the resolution index, the grayscale face pixel area is enlarged to the size corresponding to the resolution index, and the bilinear interpolation method is used to complete the pixels, and the completion result is used as the face area sub-image after super-resolution reconstruction; otherwise, the grayscale face pixel area is directly used as the face area sub-image after super-resolution reconstruction, where The face pixel area The corresponding face area sub-image.

4. The face capture and recognition method in a dense scene as claimed in claim 1, characterized in that: In the step S3, posture correction is performed on the face region sub-image, including: The posture correction is performed on the face area sub-image, where the face area sub-image The posture correction process is as follows: S31: face area sub-image Mark the skin color pixels in the image, and mark other pixels as non-skin color pixels; The face area sub-image The center of the face is taken as the face center, and the non-skin color pixel areas in the upper left part and the upper right part of the face center are taken as the left eye area and the right eye area; S32: Connect the center of the left eye area and the center of the right eye area to obtain a face area sub-image The eye line connecting the two eyes; calculate the angle between the eye line and the horizontal line as the sub-image of the face area Roll angle of face pose ; S33: Using the roll angle Construct the face area sub-image The affine matrix of : ； S34: According to the affine matrix For face area sub-image The pixel coordinates of the pixels in the image are affine transformed to obtain the face area sub-image after posture correction .

5. The method for face capture and recognition in a dense scene as claimed in claim 4, characterized in that: According to the affine matrix For face area sub-image The pixel coordinates of the pixels in the image are transformed by an affine transformation, including: Pixel coordinates The affine transformation formula is: ； ； in: is the pixel coordinate The affine transformation result of .

6. The method for face capture and recognition in a dense scene as claimed in claim 5, characterized in that: In the step S4, identity association recognition is performed on the corrected face region sub-image, including: The corrected face region sub-image is subjected to identity association recognition, wherein the identity association recognition process is as follows: S41: Calculate the correlation coefficient between different corrected face region sub-images in adjacent dense scene video frames, where the dense scene video frames and In the figure, the rectified face region sub-image The correlation coefficient between ; S42: taking two groups of corrected face region sub-images with correlation coefficients higher than a preset threshold as images of the same face appearing in adjacent dense scene video frames; S43: repeating steps S41 to S42 to obtain a sequence of face region sub-images of different faces, wherein the sequence of face region sub-images represents a sequence of corrected face region sub-images of the same face in different dense scene videos; A facial region sub-image sequence and a facial pixel region sequence of the same face are extracted as the face capture recognition result of the face in a dense scene.

Images