Face information acquisition method and device based on multiple cameras, equipment and medium

Abstract

The invention discloses a multi-camera-position-based face information acquisition method and device, equipment and a medium, and relates to the field of face recognition, and the method comprises the steps: collecting a fixed-focus picture image through a fixed-focus camera, and recognizing a head region and a face region in the fixed-focus picture image; determining the overlapping degree of the head area and the face area, and judging whether the overlapping degree meets a first preset condition or not; if the overlapping degree meets a first preset condition, determining head posture information in the head region, so as to determine a current face orientation based on the head posture information, and judging whether the current face orientation meets a second preset condition or not; and if the current face orientation meets a second preset condition, executing a zoom operation through the zoom camera so as to amplify a face picture in the head area and collect a corresponding first zoom picture image, and obtaining corresponding face information from the first zoom picture image.

Description

Technical Field

[0001] The present application relates to the field of face recognition, and in particular to a method, device, equipment and medium for acquiring face information based on multiple cameras. Background Art

[0002] With the development of science and technology and the advent of the intelligent era, face recognition technology has gradually become a core component of the modern social security protection system. This technology realizes efficient and accurate identity recognition and comparison by accurately analyzing the facial images captured by the camera, providing strong technical support for public safety, intelligent monitoring, access control management and other fields; as one of the key applications of face recognition technology, the face capture system can quickly identify the target individual through real-time monitoring and intelligent analysis, significantly improving the efficiency of security monitoring.

[0003] However, in actual applications, traditional single-camera face capture systems are difficult to fully cover scenes with dense crowds or complex dynamics, resulting in a low face capture rate, which seriously affects the monitoring effect and recognition accuracy; traditional multi-camera face capture systems generally lack an efficient coordination mechanism, each camera works independently, and lacks unified scheduling and management, resulting in resource waste and repeated capture; at the same time, for faces that are far away or have obstructions on the face, the above-mentioned face capture system cannot accurately and efficiently recognize the face. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a method, device, equipment and medium for acquiring face information based on multiple cameras, which can effectively improve the efficiency and accuracy of face capture and comprehensively improve the face acquisition rate. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a multi-camera-based facial information acquisition method, which is applied to a multi-camera camera device, wherein the multi-camera camera device is configured with a fixed-focus camera and a zoom camera, and the fixed-focus camera and the zoom camera have been calibrated; wherein the method comprises:

[0006] Capturing a fixed-focus screen image by using the fixed-focus camera, and identifying a head area and a face area in the fixed-focus screen image;

[0007] Determining the degree of overlap between the head region and the face region, and judging whether the degree of overlap satisfies a first preset condition;

[0008] If the overlap degree satisfies a first preset condition, determining head posture information in the head area, determining a current face orientation based on the head posture information, and judging whether the current face orientation satisfies a second preset condition;

[0009] If the current face orientation satisfies the second preset condition, a zoom operation is performed by the zoom camera to enlarge the face image in the head area and capture a corresponding first zoom image, and obtain corresponding face information from the first zoom image.

[0010] Optionally, the calibration process between the fixed-focus camera and the zoom camera includes:

[0011] Normalizing the pixel coordinates in the fixed-focus picture image collected by the fixed-focus camera to obtain normalized coordinates, and determining the target gimbal attitude parameters of the gimbal of the zoom camera during the calibration process; the target gimbal attitude parameters include the minimum gimbal yaw angle, the maximum gimbal yaw angle, the minimum gimbal pitch angle, and the maximum gimbal pitch angle;

[0012] The gimbal yaw angle and the gimbal pitch angle corresponding to the normalized coordinates are determined based on the target gimbal attitude parameters, so as to map the normalized coordinates into gimbal control parameters corresponding to the zoom camera, so as to complete the calibration between the fixed-focus camera and the zoom camera.

[0013] Optionally, the calibration process between the fixed-focus camera and the zoom camera includes:

[0014] Performing a first rotation operation of the pan / tilt of the zoom camera, and determining, from the zoom screen image captured by the zoom camera during the first rotation operation, a target zoom image area that matches a target fixed-focus image area in the fixed-focus screen image captured by the fixed-focus camera;

[0015] A second rotation operation of the pan / tilt control of the zoom camera is performed to move the target zoom image area to the image center position of the zoom camera, and the pan / tilt rotation parameters corresponding to the zoom camera during the second rotation operation are recorded to complete the calibration between the fixed-focus camera and the zoom camera.

[0016] Optionally, the collecting a fixed-focus screen image by the fixed-focus camera and identifying a head area and a face area in the fixed-focus screen image includes:

[0017] The fixed-focus camera captures a fixed-focus screen image, and performs a face detection operation based on the fixed-focus screen image to identify a face area in the fixed-focus screen image to obtain face information; the face information includes a face position, a face frame confidence, and a face definition; the face frame confidence is used to indicate the probability that a face is contained in the face frame area;

[0018] A head detection operation is performed on the fixed-focus screen image to identify the head area in the fixed-focus screen image to obtain head information; the head information includes the head position, the head frame confidence, and the head posture information; the head frame confidence is used to indicate the probability of containing a head in the head frame area.

[0019] Optionally, after identifying the face area in the fixed-focus image and obtaining the face information, the method further includes:

[0020] If there is face information to be detected in the face information whose face clarity is lower than a preset clarity threshold, the zoom camera is used to perform a zoom-in operation to enlarge the face picture corresponding to the face information to be detected and capture a corresponding second zoom picture image, and obtain the corresponding target face information from the second zoom picture image.

[0021] Optionally, if the overlap degree satisfies a first preset condition, determining the head posture information in the head area, determining the current face orientation based on the head posture information, and judging whether the current face orientation satisfies a second preset condition, comprises:

[0022] If the overlap degree does not exceed a preset overlap degree threshold, determining the head posture information in the head area to determine the current face orientation based on the head posture information;

[0023] It is determined whether the current face orientation is in a preset orientation camera state, if so, it is determined that the current face orientation meets a second preset condition, otherwise it is determined that the current face orientation does not meet the second preset condition.

[0024] Optionally, the preset camera facing state is a camera facing state corresponding to a preconfigured target head posture angle range, and the target head posture angle range is a head posture angle range that corresponds to and can be dynamically adjusted when the face is facing the camera frontally and / or the face is facing the camera sideways.

[0025] In a second aspect, the present application provides a multi-camera device, including a fixed-focus camera and a zoom camera; wherein, a processor in the multi-camera device is used to execute a computer program to implement the aforementioned multi-camera-based facial information acquisition method.

[0026] Optionally, the multi-camera device is further configured with a gimbal system; wherein the gimbal system includes a first gimbal and a second gimbal, the first gimbal is the gimbal of the fixed-focus camera, and the second gimbal is the gimbal of the zoom camera.

[0027] Optionally, the multi-camera device is also configured with a bottom base, a fixed-focus camera carrying a first gimbal is configured above the bottom base, and a zoom camera carrying a second gimbal is configured above the fixed-focus camera, wherein the first gimbal is used to control the lens of the fixed-focus camera to rotate horizontally, and the second gimbal is used to control the lens of the zoom camera to rotate horizontally and / or vertically.

[0028] In a third aspect, the present application provides a multi-camera-based facial information acquisition device, which is applied to a multi-camera camera device, wherein the multi-camera camera device is configured with a fixed-focus camera and a zoom camera, and the fixed-focus camera and the zoom camera have been calibrated; wherein the device comprises:

[0029] A region recognition module, used to collect a fixed-focus screen image through the fixed-focus camera, and recognize a head region and a face region in the fixed-focus screen image;

[0030] a first judging module, configured to determine the degree of overlap between the head region and the face region, and to judge whether the degree of overlap satisfies a first preset condition;

[0031] a second judgment module, configured to determine head posture information in the head area if the overlap degree satisfies a first preset condition, determine a current face orientation based on the head posture information, and judge whether the current face orientation satisfies a second preset condition;

[0032] The information acquisition module is used to perform a zoom operation through the zoom camera if the current face orientation meets the second preset condition, so as to enlarge the face picture in the head area and collect the corresponding first zoom picture image, and obtain the corresponding face information from the first zoom picture image.

[0033] In a fourth aspect, the present application provides an electronic device, including:

[0034] Memory, used to store computer programs;

[0035] The processor is used to execute the computer program to implement the aforementioned method for acquiring facial information based on multiple cameras.

[0036] In a fifth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned multi-camera-based facial information acquisition method.

[0037] In the present application, the fixed-focus screen image is captured by the fixed-focus camera, and the head area and the face area in the fixed-focus screen image are identified; the degree of overlap between the head area and the face area is determined, and it is judged whether the degree of overlap meets the first preset condition; if the degree of overlap meets the first preset condition, the head posture information in the head area is determined to determine the current face orientation based on the head posture information, and it is judged whether the current face orientation meets the second preset condition; if the current face orientation meets the second preset condition, the zoom operation is performed by the zoom camera to enlarge the face image in the head area and capture the corresponding first zoom screen image, and obtain the corresponding face information from the first zoom screen image. As can be seen from the above, the present application first identifies the head area and the face area in the fixed-focus screen image captured by the fixed-focus camera, then judges the head posture information of the head area that meets the first preset condition, and performs a zoom operation on the head area whose current face orientation meets the second preset condition to obtain the corresponding face information. In this way, after completing the detection of the face area in the fixed-focus image, the detection of the head area is added, which can more accurately locate and detect the target face that is far away or has obstructions on the face, effectively improving the efficiency and accuracy of face capture and comprehensively improving the face acquisition rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 A schematic diagram of a multi-camera device disclosed in this application;

[0040] Figure 2 A schematic diagram of a fixed-focus camera and a zoom camera calibration method disclosed in this application;

[0041] Figure 3 This is a flow chart of a method for acquiring facial information based on multiple cameras disclosed in this application;

[0042] Figure 4 A schematic diagram of face capture disclosed in this application;

[0043] Figure 5 A schematic diagram of a face detection operation disclosed in this application;

[0044] Figure 6 This is a schematic diagram of a head detection operation disclosed in this application;

[0045] Figure 7 This is a schematic diagram of the structure of a multi-camera-based facial information acquisition device disclosed in this application;

[0046] Figure 8 This is a schematic diagram of the structure of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] Nowadays, face capture systems can quickly identify target individuals through real-time monitoring and intelligent analysis, significantly improving the efficiency of security monitoring. However, in actual applications, traditional single-camera face capture systems are difficult to fully cover scenes with dense crowds or complex dynamics, resulting in a low face capture rate, which seriously affects the monitoring effect and recognition accuracy; traditional multi-camera face capture systems generally lack an efficient coordination mechanism, each camera works independently, and lacks unified scheduling and management, resulting in resource waste and repeated capture; at the same time, for faces that are far away or have obstructions on the face, the above-mentioned face capture system cannot accurately and efficiently recognize the face. To this end, the present application provides a multi-camera-based face information acquisition method that can effectively improve the efficiency and accuracy of face capture.

[0049] See also Figure 1 As shown, the embodiment of the present application discloses a multi-camera device, wherein the multi-camera device includes a fixed-focus camera and a zoom camera; the multi-camera device is also configured with a pan-tilt system, and the pan-tilt system includes a first pan-tilt and a second pan-tilt, the first pan-tilt is a pan-tilt of the fixed-focus camera, and the second pan-tilt is a pan-tilt of the zoom camera. Specifically, the multi-camera device is configured with a bottom base 03, a fixed-focus camera 02 carrying a first pan-tilt 04 is configured above the bottom base 03, and a zoom camera 01 carrying a second pan-tilt 05 is configured above the fixed-focus camera 02; wherein the first pan-tilt 04 is used to control the lens of the fixed-focus camera 02 to rotate horizontally, and the second pan-tilt 05 is used to control the lens of the zoom camera 01 to rotate horizontally and / or vertically.

[0050] It can be understood that the fixed-focus camera 02 has a large field of view and is suitable for wide-area monitoring. It can quickly discover and preliminarily detect faces within a preset effective distance range. The combination of the fixed-focus camera 02 and the first pan-tilt head 04 enables the camera device to be flexibly adjusted in the horizontal direction as needed, thereby achieving monitoring at different angles and ensuring that all possible face targets are covered in changing monitoring scenarios. The zoom camera 01 can capture high-precision details at a longer distance or at a specific target through its own adjustable focal length design. By adjusting the focal length, the zoom camera can clearly capture high-resolution images of faces within different distance ranges. However, the field of view of the zoom camera will change during the focal length adjustment process. Without the assistance of the pan-tilt head, the zoom process may result in the loss of the captured target. Therefore, the second pan-tilt head 05 corresponding to the zoom camera 01 can not only provide all-round rotation and pitch adjustment, but also ensure that the shooting direction of the zoom camera 01 is always aligned with the face target detected by the fixed-focus camera 02, thereby avoiding the loss of the target due to changes in viewing angle.

[0051] Among them, fixed-focus cameras, zoom cameras, and pan-tilt systems can be used not only one-to-one but also one-to-many or many-to-many, thereby greatly improving the system's coverage and capture accuracy. For example, installing multiple fixed-focus cameras can cover multiple viewing angles at the same time, and even 360-degree coverage; if equipped with multiple zoom cameras with pan-tilts, it is more suitable for secondary capture of crowded areas, avoiding the problem of low efficiency of a single pan-tilt zoom camera in secondary capture of faces in crowded areas.

[0052] It should be noted that the fixed-focus camera and the zoom camera in this embodiment have been calibrated.

[0053] In a specific implementation, the calibration between the fixed-focus camera and the zoom camera can be performed by manual calibration, including: first, normalizing the pixel coordinates in the fixed-focus picture image collected by the fixed-focus camera to obtain normalized coordinates, and determining the target gimbal attitude parameters of the zoom camera during the calibration process; wherein the target gimbal attitude parameters include a minimum gimbal yaw angle, a maximum gimbal yaw angle, a minimum gimbal pitch angle, and a maximum gimbal pitch angle; then, based on the target gimbal attitude parameters, determining the gimbal yaw angle and the gimbal pitch angle corresponding to the normalized coordinates, so as to map the normalized coordinates to the gimbal control parameters corresponding to the zoom camera, so as to complete the calibration between the fixed-focus camera and the zoom camera.

[0054] Specifically, the pixel coordinates of the effective image captured by the fixed-focus camera are first normalized, that is, the pixel coordinate range is scaled to the [0,1] interval. If the pixel coordinates of the fixed-focus camera are (x, y), the normalized coordinates can be calculated by the following formula:

[0055] ;

[0056] Where W and H are the width and height of the fixed-focus camera image, respectively.

[0057] During the calibration process, record the minimum and maximum parameter values ​​of the gimbal corresponding to the zoom camera, and set the upper left corner and lower right corner of the zoom camera screen to correspond to the minimum and maximum coordinates of the gimbal corresponding to the zoom camera, respectively. , ), and use the following linear transformation formula to map the coordinates of the fixed-focus camera to the control parameter space of the zoom camera gimbal:

[0058] ;

[0059] ;

[0060] Among them, P represents the yaw angle parameter of the gimbal, T represents the pitch angle parameter of the gimbal, Indicates the maximum gimbal yaw angle, Indicates the minimum gimbal yaw angle, Indicates the maximum gimbal pitch angle, Indicates the minimum gimbal pitch angle.

[0061] Through the above mapping, the normalized coordinates of the fixed-focus camera can be mapped to the control parameters of the gimbal corresponding to the zoom camera. The gimbal control parameters determine the adjustment angle of the zoom camera, thereby ensuring that the zoom camera can accurately aim at the target area captured by the fixed-focus camera.

[0062] In another specific implementation, the calibration between the fixed-focus camera and the zoom camera can be performed by an automatic calibration method, including: firstly performing a first rotation operation of the pan / tilt of the zoom camera, and determining a target zoom image area that matches the target fixed-focus image area in the fixed-focus picture image captured by the fixed-focus camera from the zoom picture image captured by the zoom camera during the first rotation operation; then performing a second rotation operation of controlling the pan / tilt of the zoom camera to move the target zoom image area to the image center position of the zoom camera, and recording the pan / tilt rotation parameters corresponding to the zoom camera during the second rotation operation to complete the calibration between the fixed-focus camera and the zoom camera.

[0063] For details, see Figure 2The single-area automatic calibration process shown in the figure first performs image preprocessing operations, that is, denoising operations are performed on the images collected by the fixed-focus camera and the zoom camera to improve the stability of feature extraction; then image feature extraction is performed, and image feature extraction such as SIFT (Scale Invariant Feature Transform) can be performed on the images of the fixed-focus camera and the zoom camera respectively to extract stable key points in the image and their local descriptors; then the Euclidean distance and NNDR (Nearest Neighbor Distance Ratio, nearest neighbor ratio) and other methods are used to match the feature points between the two images, and the accuracy of the matching results is ensured by screening the matched feature point pairs; further, the zoom camera is controlled to rotate, and the area in the zoom camera screen that matches the area related to the fixed-focus camera (such as the upper left corner area and the lower right corner area) is found according to the coordinates of the matched feature points. If the corresponding area is not matched, the pan / tilt of the zoom camera is continued to be controlled to rotate until a matching area with the area related to the fixed-focus camera is found in the zoom camera screen, the pan / tilt is controlled to move the matching area to the image center of the zoom camera, and the rotation parameters of the pan / tilt (such as the horizontal rotation angle and the vertical rotation angle) are recorded, and the single-area automatic calibration is ended; the above steps are repeated to complete the automatic calibration of each sub-area in the fixed-focus camera screen.

[0064] After completing the coordinate mapping through manual calibration or automatic calibration, the gimbal movement corresponding to the zoom camera can be accurately controlled to align with the target area corresponding to any specified coordinates in the fixed-focus camera image. At this time, if the area of ​​interest in the fixed-focus camera image is selected manually or by other means, the device can automatically drive the zoom camera to rotate and adjust the focal length to accurately capture and magnify the selected area.

[0065] As can be seen from the above, this embodiment adopts a camera device that integrates a fixed-focus camera and a zoom camera, and is supplemented by a pan-tilt control system for multi-camera layout, aiming to make full use of the unique advantages of various cameras while avoiding the inherent limitations of a single camera. In this embodiment, the pan-tilt angle of the zoom camera can be rotated in a wider range and can be applied to more scenes. At the same time, a dual pan-tilt design is adopted, so that the device can still retain the ability to adjust the direction after being installed and fixed; further, the device can be installed in movable devices such as robots, enhancing the flexibility and adaptability of the system in complex and dynamic scenes, and providing a more comprehensive and efficient face capture device.

[0066] See also Figure 3 As shown, the embodiment of the present application discloses a method for acquiring face information based on multiple cameras, which is applied to the above-mentioned multi-camera camera device, wherein the multi-camera camera device is configured with a fixed-focus camera and a zoom camera, and the fixed-focus camera and the zoom camera have been calibrated; wherein the method includes:

[0067] Step S11: collecting a fixed-focus picture image by the fixed-focus camera, and identifying a head area and a face area in the fixed-focus picture image.

[0068] In this embodiment, see Figure 4 As shown, after starting face capture, a fixed-focus screen image is first collected by a fixed-focus camera, and a face detection operation is performed on the fixed-focus screen image to identify the face area in the fixed-focus screen image to obtain face information; wherein the face information includes face position, face frame confidence, and face clarity; the face frame confidence is used to indicate the probability that a face is contained in the face frame area; then a head detection operation is performed on the fixed-focus screen image to identify the head area in the fixed-focus screen image to obtain head information; wherein the head information includes head position, head frame confidence, and head posture information; the head frame confidence is used to indicate the probability that a head is contained in the head frame area.

[0069] Among them, see Figure 5 The face detection operation diagram shown in FIG. 1 adopts a lightweight deep learning-based face detection algorithm to achieve efficient face detection and maintain good adaptability in complex and changing environments. The algorithm first preprocesses the input fixed-focus image, such as resizing the image to 224x224 and normalizing it; then the preprocessed image is input into a lightweight MobileNet (a lightweight deep neural network) feature extraction network for feature extraction; in the feature extraction stage, a depthwise separable convolution can be used to effectively reduce the computational complexity by splitting the standard convolution operation into depth convolution and point-by-point convolution, while ensuring high accuracy and inference speed; the feature extraction module outputs multiple levels of feature maps, which contain low-level visual information (such as texture, edge, color, etc.) of the image and higher-level abstract features, which provide rich expressions for subsequent face detection and regression tasks; further, the feature map is input into three different convolution branches, each of which is responsible for regressing different targets, such as:

[0070] Bounding Box Regression: Regress the exact position of the detected face area (i.e. the coordinates of the face box).

[0071] Confidence Regression: Evaluate the confidence of each face frame, indicating the probability that the area contains a face.

[0072] Clarity Regression: Regress the clarity score of each detected face region, indicating whether the face in the region is clearly visible.

[0073] After obtaining the preliminary detection results, the overlapping detection frames can be screened through post-processing such as non-maximum suppression (NMS) to remove redundant frames and retain the optimal detection frames; through the post-processing steps, the final face information is output, which may include: the precise position of the face recognition frame (face position), the face frame confidence, and the corresponding face clarity score.

[0074] This method uses lightweight design and efficient convolutional network structure to ensure fast reasoning while maintaining high detection accuracy, making it suitable for real-time face recognition tasks. In addition, this method also combines the clarity features of face images to provide a more convenient information basis for subsequent logical decision-making.

[0075] After the face area in the fixed-focus image is identified and the face information is obtained, Figure 4 As shown, it also includes: if there is no face information to be detected whose face clarity is lower than the preset clarity threshold in the face information, the corresponding face information is directly output; if there is face information to be detected whose face clarity is lower than the preset clarity threshold in the face information, a zoom camera is used to perform a zoom-in operation to enlarge the face picture corresponding to the face information to be detected and collect the corresponding second zoom picture image, so as to perform a secondary detection on the face picture to be detected, and obtain the corresponding target face information from the second zoom picture image. It should be noted that the face information corresponding to the face picture whose face size is smaller than the preset size threshold can also be regarded as the face information to be detected whose face clarity is lower than the preset clarity threshold, and a secondary detection is performed according to the above steps to ensure that all face pictures that do not meet the standards can perform a secondary zoom-in operation.

[0076] Since a small face at a distance, or an obstruction such as a mask at a certain distance will increase the missed detection rate of the face, this embodiment adds a head detection stage after completing the face detection operation. Figure 6As shown in the schematic diagram of the head detection operation, firstly, the input fixed-focus screen image is preprocessed to convert the original input image into a format acceptable to the network; then the preprocessed image is passed to the CSPDarknet53 (Cross Stage Partial Darknet53, a lightweight backbone network) network for feature extraction, and CSPNet (CSPNet with Dense Blocks; CSPNet, i.e. Cross Stage Partial Network, cross-stage local network) is combined with multi-layer convolution to efficiently extract high-level features in the image, and can handle large-scale and small-scale targets; after feature extraction, it enters the PAN-FPN network, and FPN (Feature Pyramid Network) and PANet (Path Aggregation Network) can be used to fuse the features extracted from different scales (low and high layers), thereby providing more fine-grained information for the target detection task, so as to increase the detection rate of small targets of distant faces; then head detection is performed, which consists of multiple regression branches, each branch is responsible for different tasks, for example:

[0077] Bounding Box Regression: The target position is predicted through the regression branch, and the output is the boundary coordinates of the head box.

[0078] Confidence Regression: It is used to predict the probability of a human head being contained in a box. The confidence is calculated based on the intersection over union (IoU) between the box and the target. A high confidence indicates that the box is more likely to be a human head target.

[0079] Head Pose Regression: This branch is used to regress the head posture angles, including pitch, yaw, and roll. These angles help determine the orientation of the head, thus providing a basis for more accurate head detection.

[0080] After obtaining the preliminary head detection information, the final head information is output through post-processing steps such as non-maximum suppression (NMS), including: head position, head frame confidence, and head posture information.

[0081] In the above head detection operation, the output head information includes head posture information, which is helpful for subsequent decision-making, such as subsequent judgment of facing the camera. While increasing the face detection rate, it can also reduce the waste of resources for secondary face detection of heads that are not facing the camera.

[0082] Step S12: determining the degree of overlap between the head region and the face region, and judging whether the degree of overlap satisfies a first preset condition.

[0083] In this embodiment, the head area and the face area in the fixed-focus image are determined based on the head information and the face information obtained in step S11, and it is determined whether the degree of overlap between the head area and the face area meets the first preset condition.

[0084] Step S13: If the overlap degree satisfies the first preset condition, the head posture information in the head area is determined to determine the current face orientation based on the head posture information, and to determine whether the current face orientation satisfies the second preset condition.

[0085] In this embodiment, if the degree of overlap does not exceed a preset overlap degree threshold, the head posture information in the head area is determined to determine the current face orientation based on the head posture information; then a judgment is made as to whether the current face orientation is in a preset orientation camera state, if so, the current face orientation is determined to satisfy a second preset condition, otherwise, the current face orientation is determined to not satisfy the second preset condition; wherein the preset orientation camera state is a orientation camera state corresponding to a pre-configured target head posture angle range, and the target head posture angle range is a head posture angle range that corresponds to and can be dynamically adjusted when the face is facing the camera and / or the face is facing the camera sideways.

[0086] Step S14: If the current face orientation satisfies the second preset condition, a zoom operation is performed by the zoom camera to enlarge the face image in the head area and capture a corresponding first zoom image, and obtain corresponding face information from the first zoom image.

[0087] In this embodiment, if the corresponding head posture angle range is within the pre-configured target head posture angle range when the current face is facing the camera and / or the current face is facing the camera sideways, the current face orientation satisfies the second preset condition. At this time, a zoom operation is performed by the zoom camera to enlarge the face image in the head area and capture the corresponding first zoom image, and obtain the corresponding face information from the first zoom image.

[0088] It is understandable that if Figure 4As shown, after the head detection operation of the fixed-focus screen image is performed to obtain the head information in the fixed-focus screen image, the head area is matched and analyzed with the previously detected face area to determine whether there is spatial overlap, that is, whether the head area is a face area. If the degree of overlap exceeds the preset overlap degree threshold, it means that the face information corresponding to the head area has been obtained, and a non-intervention strategy is adopted for the head area; on the contrary, for the head area that does not overlap with any detected face area or has a low degree of overlap, the head posture is further detected to obtain the head posture information, and by measuring the posture angle of the head relative to the camera, it is determined whether it is in a front facing camera state. If the head is front facing the camera, a zoom-in operation is performed through the zoom camera to enlarge the face picture in the head area and capture the corresponding first zoom screen image, and obtain the corresponding face information from the first zoom screen image.

[0089] In specific application scenarios, if side face detection is required, the target head posture angle range can be appropriately adjusted to expand the detection coverage. For head areas that meet the detection requirements (including facing the camera or within a relaxed angle range), a zoom camera can be used to perform close-range secondary face detection to ultimately obtain accurate facial feature information.

[0090] As can be seen from the above, this embodiment first identifies the head area and the face area in the fixed-focus picture image captured by the fixed-focus camera, then determines the head posture information of the head area that meets the first preset condition, and performs a zoom operation on the head area whose current face orientation meets the second preset condition to obtain corresponding face information. In this embodiment, after completing the detection of the face area in the fixed-focus picture image, the detection of the head area is added, which can more accurately locate and detect the target face that is far away or has an obstruction on the face, and start the zoom camera for secondary capture, thereby greatly improving the recognition rate and capture effect. It can not only effectively improve the detection rate of small targets at a long distance, but also to a certain extent make up for the shortcomings of face detection in complex scenes (such as obstruction or long distance), and significantly improve the real-time performance and accuracy of the system; at the same time, by combining the multi-camera layout, collaborative work and pan-tilt control system of the fixed-focus camera and the zoom camera, the face acquisition rate can be comprehensively improved, and the frame leakage phenomenon can be reduced, so as to achieve more accurate and efficient face capture and monitoring in public places, effectively improve the efficiency and accuracy of face capture, and provide a strong technical guarantee for building a safer and smarter public safety system.

[0091] Furthermore, the embodiment of the present application also discloses a multi-camera device, including a fixed-focus camera and a zoom camera; wherein the processor in the multi-camera device is used to execute a computer program to implement the multi-camera-based face information acquisition method disclosed above. For the specific steps of the method, reference can be made to the corresponding contents disclosed in the above embodiment, and no further description will be given here.

[0092] See also Figure 7 As shown, the embodiment of the present application also discloses a multi-camera-based face information acquisition device, comprising:

[0093] The region recognition module 11 is used to collect a fixed-focus screen image through the fixed-focus camera and recognize a head region and a face region in the fixed-focus screen image;

[0094] A first judging module 12, configured to determine the degree of overlap between the head region and the face region, and to judge whether the degree of overlap satisfies a first preset condition;

[0095] A second judgment module 13 is used to determine the head posture information in the head area if the overlap degree meets the first preset condition, determine the current face orientation based on the head posture information, and judge whether the current face orientation meets the second preset condition;

[0096] The information acquisition module 14 is used to perform a zoom operation through the zoom camera to enlarge the face image in the head area and capture a corresponding first zoom image if the current face orientation meets the second preset condition, and obtain corresponding face information from the first zoom image.

[0097] As can be seen from the above, the present application first identifies the head area and face area in the fixed-focus screen image captured by the fixed-focus camera, then determines the head posture information of the head area that meets the first preset condition, and performs a zoom operation on the head area whose current face orientation meets the second preset condition to obtain the corresponding face information. In this way, after completing the detection of the face area in the fixed-focus screen image, the detection of the head area is added, which can more accurately locate and detect the target face that is far away or has an obstruction on the face, effectively improving the efficiency and accuracy of face capture, and comprehensively improving the face acquisition rate.

[0098] In some specific implementations, the calibration process between the fixed-focus camera and the zoom camera includes:

[0099] A parameter determination unit, used to normalize the pixel coordinates in the fixed-focus picture image collected by the fixed-focus camera to obtain normalized coordinates, and determine the target gimbal attitude parameters of the gimbal of the zoom camera during the calibration process; the target gimbal attitude parameters include the minimum gimbal yaw angle, the maximum gimbal yaw angle, the minimum gimbal pitch angle and the maximum gimbal pitch angle;

[0100] A coordinate mapping unit is used to determine the gimbal yaw angle and gimbal pitch angle corresponding to the normalized coordinates based on the target gimbal attitude parameters, so as to map the normalized coordinates into gimbal control parameters corresponding to the zoom camera, so as to complete the calibration between the fixed-focus camera and the zoom camera.

[0101] In some specific implementations, the calibration process between the fixed-focus camera and the zoom camera includes:

[0102] a first operating unit, configured to perform a first rotation operation of the pan / tilt of the zoom camera, and to determine, from the zoom screen image captured by the zoom camera during the first rotation operation, a target zoom image area that matches a target fixed-focus image area in the fixed-focus screen image captured by the fixed-focus camera;

[0103] A second operating unit is used to execute a second rotation operation of the pan / tilt of the zoom camera to move the target zoom image area to the image center position of the zoom camera, and record the pan / tilt rotation parameters corresponding to the zoom camera during the second rotation operation to complete the calibration between the fixed-focus camera and the zoom camera.

[0104] In some specific implementations, the region identification module 11 includes:

[0105] A face information determination submodule, used to collect a fixed-focus screen image through the fixed-focus camera, and perform a face detection operation based on the fixed-focus screen image to identify the face area in the fixed-focus screen image to obtain face information; the face information includes face position, face frame confidence, and face clarity; the face frame confidence is used to indicate the probability that a face is contained in the face frame area;

[0106] A head information determination unit is used to perform a head detection operation on the fixed-focus screen image to identify the head area in the fixed-focus screen image and obtain head information; the head information includes the head position, head frame confidence, and head posture information; the head frame confidence is used to indicate the probability of containing a head in the head frame area.

[0107] In some specific implementations, the face information determination submodule further includes:

[0108] A facial information determination unit is used to use the zoom camera to perform a zoom-in operation to enlarge the face picture corresponding to the facial information to be detected and capture a corresponding second zoom picture image if there is facial information to be detected whose facial clarity is lower than a preset clarity threshold in the facial information, and obtain the corresponding target facial information from the second zoom picture image.

[0109] In some specific implementations, the second determination module 13 includes:

[0110] a face orientation determining unit, configured to determine head posture information in the head region if the overlap degree does not exceed a preset overlap degree threshold, so as to determine a current face orientation based on the head posture information;

[0111] The judging unit is used to judge whether the current face orientation is in a preset orientation camera state, and if so, judge that the current face orientation satisfies a second preset condition, otherwise, judge that the current face orientation does not satisfy the second preset condition.

[0112] In some specific embodiments, the preset camera-facing state is a camera-facing state corresponding to a preconfigured target head posture angle range, and the target head posture angle range is a head posture angle range that corresponds to and can be dynamically adjusted when the face is facing the camera frontally and / or the face is facing the camera sideways.

[0113] Furthermore, the present application also discloses an electronic device. Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0114] Figure 8 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the multi-camera-based face information acquisition method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0115] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0116] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0117] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the multi-camera-based face information acquisition method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0118] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed method for acquiring face information based on multiple cameras is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.

[0119] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0120] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0121] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0122] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0123] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for acquiring face information based on multiple cameras, characterized in that: Applied to a multi-camera device, the multi-camera device is equipped with a fixed-focus camera and a zoom camera, and the fixed-focus camera and the zoom camera have been calibrated; wherein the method comprises: Capturing a fixed-focus screen image by using the fixed-focus camera, and identifying a head area and a face area in the fixed-focus screen image; Determining the degree of overlap between the head region and the face region, and judging whether the degree of overlap satisfies a first preset condition; If the overlap degree satisfies a first preset condition, determining head posture information in the head area, determining a current face orientation based on the head posture information, and judging whether the current face orientation satisfies a second preset condition; If the current face orientation satisfies the second preset condition, a zoom operation is performed by the zoom camera to enlarge the face image in the head area and capture a corresponding first zoom image, and obtain corresponding face information from the first zoom image.

2. The method for acquiring face information based on multiple cameras according to claim 1, characterized in that: The calibration process between the fixed-focus camera and the zoom camera includes: Normalizing the pixel coordinates in the fixed-focus picture image collected by the fixed-focus camera to obtain normalized coordinates, and determining the target gimbal attitude parameters of the gimbal of the zoom camera during the calibration process; the target gimbal attitude parameters include the minimum gimbal yaw angle, the maximum gimbal yaw angle, the minimum gimbal pitch angle, and the maximum gimbal pitch angle; The gimbal yaw angle and the gimbal pitch angle corresponding to the normalized coordinates are determined based on the target gimbal attitude parameters, so as to map the normalized coordinates into gimbal control parameters corresponding to the zoom camera, so as to complete the calibration between the fixed-focus camera and the zoom camera.

3. The method for acquiring face information based on multiple cameras according to claim 1, characterized in that: The calibration process between the fixed-focus camera and the zoom camera includes: Performing a first rotation operation of the pan / tilt of the zoom camera, and determining, from the zoom screen image captured by the zoom camera during the first rotation operation, a target zoom image area that matches a target fixed-focus image area in the fixed-focus screen image captured by the fixed-focus camera; A second rotation operation of the pan / tilt control of the zoom camera is performed to move the target zoom image area to the image center position of the zoom camera, and the pan / tilt rotation parameters corresponding to the zoom camera during the second rotation operation are recorded to complete the calibration between the fixed-focus camera and the zoom camera.

4. The method for acquiring face information based on multiple cameras according to claim 1, characterized in that: The step of collecting a fixed-focus screen image by using the fixed-focus camera and identifying a head area and a face area in the fixed-focus screen image includes: The fixed-focus camera captures a fixed-focus screen image, and performs a face detection operation based on the fixed-focus screen image to identify a face area in the fixed-focus screen image to obtain face information; the face information includes a face position, a face frame confidence, and a face definition; the face frame confidence is used to indicate the probability that a face is contained in the face frame area; A head detection operation is performed on the fixed-focus screen image to identify the head area in the fixed-focus screen image to obtain head information; the head information includes the head position, the head frame confidence, and the head posture information; the head frame confidence is used to indicate the probability of containing a head in the head frame area.

5. The method for acquiring face information based on multiple cameras according to claim 4, characterized in that: After the facial area in the fixed-focus image is identified and the facial information is obtained, the method further includes: If there is face information to be detected in the face information whose face clarity is lower than a preset clarity threshold, the zoom camera is used to perform a zoom-in operation to enlarge the face picture corresponding to the face information to be detected and capture a corresponding second zoom picture image, and obtain the corresponding target face information from the second zoom picture image.

6. The method for acquiring facial information based on multiple cameras according to any one of claims 1 to 5, characterized in that: If the overlap degree satisfies the first preset condition, determining the head posture information in the head area, determining the current face orientation based on the head posture information, and judging whether the current face orientation satisfies the second preset condition, including: If the overlap degree does not exceed a preset overlap degree threshold, determining the head posture information in the head area to determine the current face orientation based on the head posture information; It is determined whether the current face orientation is in a preset orientation camera state, if so, it is determined that the current face orientation meets a second preset condition, otherwise it is determined that the current face orientation does not meet the second preset condition.

7. The method for acquiring face information based on multiple cameras according to claim 6, characterized in that: The preset camera-facing state is a camera-facing state corresponding to a preconfigured target head posture angle range, and the target head posture angle range is a head posture angle range that corresponds to and can be dynamically adjusted when the face is facing the camera frontally and / or facing the camera sideways.

8. A multi-camera device, characterized in that: It includes a fixed-focus camera and a zoom camera; wherein the processor in the multi-camera device is used to execute a computer program to implement the multi-camera-based facial information acquisition method as described in any one of claims 1 to 7.

9. The multi-camera device according to claim 8, characterized in that: The multi-camera device is also equipped with a gimbal system; wherein the gimbal system includes a first gimbal and a second gimbal, the first gimbal is the gimbal of the fixed-focus camera, and the second gimbal is the gimbal of the zoom camera.

10. The multi-camera device according to claim 9, characterized in that: The multi-camera device is also configured with a bottom base, a fixed-focus camera carrying a first pan-tilt is configured above the bottom base, and a zoom camera carrying a second pan-tilt is configured above the fixed-focus camera, wherein the first pan-tilt is used to control the lens of the fixed-focus camera to rotate horizontally, and the second pan-tilt is used to control the lens of the zoom camera to rotate horizontally and / or vertically.

11. A multi-camera-based facial information acquisition device, characterized in that: Applied to a multi-camera device, wherein the multi-camera device is equipped with a fixed-focus camera and a zoom camera, and the fixed-focus camera and the zoom camera have been calibrated; wherein the device comprises: A region recognition module, used to collect a fixed-focus screen image through the fixed-focus camera, and recognize a head region and a face region in the fixed-focus screen image; a first judging module, configured to determine the degree of overlap between the head region and the face region, and to judge whether the degree of overlap satisfies a first preset condition; a second judgment module, configured to determine head posture information in the head area if the overlap degree satisfies a first preset condition, determine a current face orientation based on the head posture information, and judge whether the current face orientation satisfies a second preset condition; The information acquisition module is used to perform a zoom operation through the zoom camera if the current face orientation meets the second preset condition, so as to enlarge the face picture in the head area and collect the corresponding first zoom picture image, and obtain the corresponding face information from the first zoom picture image.

12. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the method for acquiring facial information based on multiple cameras as described in any one of claims 1 to 7.

13. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the method for acquiring facial information based on multiple cameras as described in any one of claims 1 to 7.

Images