Face misbrush prevention method, device and equipment and storage medium

Abstract

The present application provides a face anti-misbrushing method, device and equipment and storage medium, relating to the technical field of computer. The method comprises: collecting color two-dimensional images of multiple faces and depth images of the multiple faces; mapping the depth images to the color two-dimensional images to obtain color depth images; determining the spatial relationship between a first face and a second face in the multiple faces based on the position information of the first face and the second face in the color depth images; and outputting prompt information based on the spatial relationship between the first face and the second face, the prompt information being used to prompt user operation behavior for the first face or the second face. The method can prevent unintentional face brushing users from misbrushing, not only improving the face brushing experience of users, but also avoiding causing property loss of unintentional face brushing users.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, device, and storage medium for preventing accidental facial recognition. Background Technology

[0002] With the rapid development of facial recognition technology, face scanning has been applied to all aspects of life, such as access control systems, face scanning payment, and face scanning login for users' personal information systems.

[0003] However, in actual facial recognition processes, multiple faces often appear in the image captured by the facial recognition device after the intended face clicks to scan. Because these multiple faces are too close together, false scans often occur. Summary of the Invention

[0004] This application provides a method, device, equipment, and storage medium for preventing accidental facial recognition scanning, which can prevent unintentional users from accidentally scanning their faces, thereby improving the user's facial recognition experience and avoiding financial losses for unintentional users.

[0005] Firstly, a method for preventing accidental facial recognition scanning is provided, including:

[0006] Acquire color 2D images of multiple faces and depth images of those faces;

[0007] The depth image is mapped onto the color 2D image to obtain a color depth image;

[0008] Based on the position information of the first face and the second face in the color depth image, the spatial relationship between the first face and the second face is determined.

[0009] Based on the spatial relationship between the first face and the second face, a prompt message is output, which is used to prompt the user's operation behavior for the first face or the second face.

[0010] Secondly, a facial recognition anti-mistake device is provided, comprising:

[0011] The acquisition unit is used to acquire color two-dimensional images of multiple faces and depth images of those multiple faces;

[0012] A mapping unit is used to map the depth image onto the color two-dimensional image to obtain a color depth image;

[0013] The determining unit is used to determine the spatial relationship between the first face and the second face based on the position information of the first face and the second face in the color depth image, respectively.

[0014] The output unit is used to output prompt information based on the spatial relationship between the first face and the second face. The prompt information is used to prompt user operation behavior for the first face or the second face.

[0015] Thirdly, an electronic device is provided, comprising:

[0016] A processor, adapted to execute computer programs;

[0017] A computer-readable storage medium storing a computer program that, when executed by the processor, implements the method of the first aspect described above.

[0018] Fourthly, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the method described in the first aspect.

[0019] Fifthly, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the method of the first aspect described above.

[0020] Based on the above technical solution, firstly, multiple color 2D images of faces and their corresponding depth images are acquired, and the depth images are mapped onto the color 2D images to obtain a new depth image (color depth image). This is equivalent to mapping the depth image containing depth information onto the color 2D image, so that the resulting color depth image includes the three-dimensional coordinates of multiple faces in the color 2D image. Then, based on the position information of the first and second faces in the color depth image, the spatial relationship between the first and second faces is determined. This is equivalent to considering the impact of the spatial relationship between faces on the face-scanning device's ability to acquire the intended face during the face-scanning process. Finally, based on the spatial relationship between the first and second faces, prompts are output regarding user actions related to the first or second face. This is equivalent to outputting different prompts for different user actions related to the first or second face based on different spatial relationships. These prompts help prevent unintentional face-scanning users from accidentally using the device, improving the user experience and avoiding financial losses for unintentional users. Attached Figure Description

[0021] Figure 1a This is an example of the angle range in a camera imaging scene provided in the embodiments of this application.

[0022] Figure 1b This is an example of the imaging process of a depth image provided in the embodiments of this application.

[0023] Figure 1cThis is an example of the conversion relationship between camera coordinates and pixel coordinates provided in the embodiments of this application.

[0024] Figure 1d This is another example of the conversion relationship between camera coordinates and pixel coordinates provided in the embodiments of this application.

[0025] Figure 2 This is a schematic block diagram of the system framework provided in the embodiments of this application.

[0026] Figure 3 This is a schematic flowchart of the face recognition anti-mistake method provided in the embodiments of this application.

[0027] Figure 4 This is a schematic block diagram illustrating the spatial alignment of depth images and color 2D images provided in the embodiments of this application.

[0028] Figure 5 This is an example diagram illustrating the transformation relationship between the depth camera coordinate system and the color camera coordinate system provided in the embodiments of this application.

[0029] Figure 6 This is another illustrative flowchart of the face recognition anti-mistake method provided in the embodiments of this application.

[0030] Figure 7 This is an example of a block diagram of the face anti-mistake device provided in the embodiments of this application.

[0031] Figure 8 This is a schematic structural diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0033] The solutions provided in this application may involve artificial intelligence technology.

[0034] Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0035] It should be understood that artificial intelligence (AI) technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0036] The solutions provided in this application may also involve computer vision technology.

[0037] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0038] In addition, the solution provided in this application may also involve facial recognition technology.

[0039] Facial recognition technology is a biometric identification technology that identifies individuals based on their facial features. This technology uses cameras or webcams to capture images or video streams containing faces, automatically detects and tracks faces within the images, extracts personalized features from the facial images, and then performs facial recognition on the detected faces. It should be understood that this facial recognition technology may include a series of related technologies such as image acquisition, feature localization, identity verification, and search.

[0040] To facilitate understanding of the proposed solution, the relevant terms used in this application will be explained below.

[0041] 1. Color 2D Image: An image captured by a color sensor under natural light. It should be noted that color 2D images are generally used in facial recognition payment for tasks such as face selection and comparison recognition.

[0042] 2. Depth Image: An image obtained by capturing infrared light from a speckle structure using an infrared sensor and then resolving the speckle using depth cells. In 3D computer graphics and computer vision, a depth image is an image or image channel containing information related to the distance from the surface of a scene object to the viewpoint. Each pixel in a depth image represents the vertical distance between the depth camera plane and the plane of the object being photographed, typically represented by 16 bits and measured in millimeters. It should be noted that depth images are generally used in facial recognition payment systems for liveness detection and to assist in comparative recognition.

[0043] 3. Infrared Image: An image captured by an infrared sensor using panoramic infrared light. It should be noted that infrared images are generally used for liveness detection in facial recognition payment systems.

[0044] 4. Optimization: This involves selecting a set of color 2D images, depth images, and infrared images that meet the prerequisites for liveness detection and comparison recognition algorithms. Specifically, optimization is generally performed by selecting color 2D images based on face angle, face size, face centering, and color image clarity; by selecting infrared images based on brightness; and by selecting depth images based on completeness.

[0045] 5. Preferred payment frame: A set of preferred color two-dimensional images, depth images and infrared images, which can be used for liveness detection and comparison recognition.

[0046] 6. Liveness detection: A method to determine whether the person using facial recognition is a real person, a photo, or a head model. Generally, depth images are used to determine whether it is a photo, and the brightness of infrared images is used to determine whether it is a silicone head model.

[0047] 7. Comparison and Recognition: The facial recognition system identifies which user is using facial recognition. This typically involves extracting facial features from a color 2D image, determining feature similarity, and then using depth images to further compare the similarity of facial features.

[0048] 8. Color Depth Map (RGBD): RGBD = Color (RGB) + Depth Map. RGB color mode is an industry-standard color model that generates a wide variety of colors by varying the red (R), green (G), and blue (B) color channels and superimposing them. RGB represents the colors of the red, green, and blue channels. This standard encompasses almost all colors perceptible to human vision and is one of the most widely used color systems. Furthermore, in 3D computer graphics, a Depth Map is an image or image channel containing information about the distance to the surface of objects in the scene from the viewpoint. It's important to note that a Depth Map is similar to a grayscale image; each pixel value represents the actual distance from the sensor to the object. Typically, RGB images and Depth images are registered, resulting in a one-to-one correspondence between pixels.

[0049] 9. Field of View (FOV): Used to describe the angular range of a given scene captured by the camera, such as... Figure 1a As shown, there are three main types of field of view: horizontal field of view (HFOV), vertical field of view (VFOV), and diagonal field of view (DFOV).

[0050] 10. Speckle Structured Light: Speckle structured light is a grid of light projected by an infrared speckle projector, arranged according to a specific structural pattern. A speckle structured light imaging system consists of an infrared laser projector and an infrared sensor, such as... Figure 1b As shown, these dot-matrix lights are projected onto the surface of the object. After being imaged by an infrared sensor, the 3D coordinate information of the object's surface can be reconstructed based on the principle of triangulation, thereby obtaining a depth map.

[0051] 11. Point Cloud Image: A point cloud image records the 3D coordinate information of an object in the real world. It can be calculated from a depth image and camera parameters. Although a depth image also contains the depth information of an object, the coordinates (x, y) of a point in a depth image are pixel coordinates, while a point cloud image records the physical coordinates of that point in the real world. Point cloud images can be displayed in a 3D rendering engine, reflecting the 3D positional relationships of each point.

[0052] 12. Camera Intrinsic Parameters: Camera intrinsic parameters are parameters used to describe the transformation relationship between the 3D coordinates of a real-world object when it is imaged on the camera sensor and the resulting pixel coordinates. This transformation relationship can be found in [reference needed]. Figure 1c and Figure 1d This is represented by a simplified camera model. That is, depth maps and point cloud maps can be converted to each other using the camera's intrinsic parameters; where, for example... Figure 1cAs shown, p′ is a point on the pixel plane, P is a point on the camera plane, and the camera coordinate system consists of the optical center O, the X-axis, the Y-axis, and the Z-axis; the pixel coordinate system consists of o, the X-axis, the Y-axis, and the Z-axis; the distance from o to the optical center O is the focal length f. Figure 1d As shown, the distance f between the optical center O and B is the focal length, and the distance z between the optical center O and A is... c This is the distance between the object's surface and the camera plane.

[0053] 13. Camera Coordinate System: The camera coordinate system refers to a coordinate system with the camera's optical center as the origin, the optical axis as the z-axis, and the x and y axes parallel to the pixel plane of the camera image. In essence, the 3D coordinates obtained through the transformation between the depth image and the camera's intrinsic parameters are coordinates in the camera coordinate system.

[0054] 14. Camera Extrinsic Parameters: Camera extrinsic parameters are parameters used to describe the transformation relationship between other 3D coordinate systems and the camera coordinate system. When there are multiple cameras, when the coordinates of an object point in one camera's coordinate system are transformed to another camera's coordinate system using a rotation matrix R and a translation matrix t, matrices R and t are the extrinsic parameters between the two cameras. In other words, the camera extrinsic parameters describe the transformation relationship between the two camera coordinate systems.

[0055] 15. Facial Recognition: Specifically refers to the process of sending a user's facial image, ID card, name, and other information to a verification source (usually an agency under the Ministry of Public Security) for facial recognition and identity verification when registering for facial recognition services for the first time.

[0056] 16. Face detection: refers to detecting face regions in a given image.

[0057] 17. Face comparison: Compare the similarity between face images.

[0058] 18. 1:1 face comparison: Compare the similarity between two face images.

[0059] 19. 1:1 Face Recognition: Given a face image and identity identifier, the system compares the face with the registered photo in the background to determine whether the person is the same person.

[0060] 20. Face 1:N recognition: Given a face image, identify the identity belonging to that face from a fixed set of N identities.

[0061] 21. Liveness Detection: Using facial recognition technology, it detects whether a real person is performing facial recognition.

[0062] 22. Liveness Detection: Liveness detection is performed by having users record videos of actions such as shaking their heads, blinking, opening their mouths, and reading numbers.

[0063] 23. Reflective Liveness Detection: Based on screen reflection, this facial recognition technology uses sensors and light information to determine whether a person is alive without requiring user interaction.

[0064] 24. Depth Image Liveness Detection: Liveness detection of depth images using a 3D structured light module camera.

[0065] 25. RGB Image Liveness Detection: Liveness detection using RGB images.

[0066] Figure 2 This is a schematic block diagram 200 of the system framework provided in the embodiments of this application.

[0067] like Figure 2 As shown, the block diagram 200 may include a facial recognition terminal 210, a user 220, a user 230, and a server 240.

[0068] For example, in response to a user 220's face recognition request, the face recognition terminal 210 captures the user 220's face image and sends a face recognition authentication request to the server 240. This request carries the captured face image. Upon receiving the request, the server 240 performs face recognition on the face image to obtain face recognition-related information corresponding to that image, thus achieving the face recognition purpose. It should be noted that... Figure 2 The number of facial recognition terminals 210, users 220, users 230, and servers 240 is merely illustrative.

[0069] It should be noted that the facial recognition terminal 210 and the payment server 240 can communicate wirelessly or via wired connection using the Internet Protocol. The wireless network can include, but is not limited to, Wi-Fi, Bluetooth, infrared, Zigbee, or data networks, while the wired network can be a Universal Serial Bus (USB) network.

[0070] The facial recognition terminal 210 can be a face recognition terminal, which is a terminal with face capture function; for example, the facial recognition terminal 210 can be a smartphone, tablet computer, vehicle terminal, laptop computer, or desktop computer. The payment server 240 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The server can be directly or indirectly connected through wired or wireless communication, and this application does not impose any restrictions.

[0071] Furthermore, the facial recognition terminal 210 can be a facial recognition terminal with a smart camera. This smart camera is detachably installed in the facial recognition terminal 210 and can be a 3D (3-Dimensional) camera with liveness detection capabilities. Optionally, the smart camera may include an image sensor that can be used to acquire streaming image data. The image sensor may include any combination of a color 2D image sensor, a depth image sensor, or an infrared image sensor. Optionally, the smart camera may also include a processor that can perform data processing, such as filtering the acquired streaming data to obtain the optimal facial image. Optionally, the processor may be a DSP (Digital Signal Processor), a unique microprocessor that processes data using digital signals.

[0072] Normally, the facial recognition terminal 210 only needs to collect the facial image of user 220. However, in actual facial recognition processes, it often happens that after user 220 clicks to scan their face, the facial recognition terminal 210 collects the facial image of user 220, but the image collected also includes the facial image of user 230. Moreover, the facial images of user 230 and user 220 are too close together, which may lead to the wrong facial recognition of user 230. This not only causes a bad facial recognition experience for user 230, but also causes financial loss to user 230.

[0073] Based on this, this application provides a method, device, equipment, and storage medium for preventing accidental facial recognition scanning. By acquiring a color two-dimensional image and a depth image simultaneously, the spatial relationship of multiple faces in the facial recognition scenario is detected using the depth image. Based on the spatial relationship of the multiple faces, prompt information is provided to guide user actions for the multiple faces, thereby avoiding accidental scanning during the facial recognition process. This not only improves the user's facial recognition experience but also prevents financial losses to users who do not intend to use facial recognition.

[0074] Figure 3 This is a schematic flowchart of the face recognition anti-mistake method 300 provided in this application embodiment. It should be noted that this method is applied to a face recognition terminal, which can be as follows: Figure 2 The face recognition terminal 210 shown.

[0075] like Figure 3 As shown, the method 300 may include some or all of the following:

[0076] S301, acquire color two-dimensional images of multiple faces and depth images of the multiple faces;

[0077] S302, the depth image is mapped onto the color two-dimensional image to obtain a color depth image;

[0078] S303, Based on the position information of the first face and the second face in the color depth image, determine the spatial relationship between the first face and the second face;

[0079] S304, based on the spatial relationship between the first face and the second face, output prompt information, which is used to prompt user operation behavior for the first face or the second face.

[0080] For example, a facial recognition terminal uses a color camera to capture a color two-dimensional image and a depth camera to capture a depth image.

[0081] Optionally, the depth camera can be a time-of-flight (TOF) camera, which continuously sends light pulses to the target and then uses a sensor to receive the light returning from the object. By detecting the flight (round trip) time of the light pulses, the distance from the target to the camera plane is obtained, thus obtaining a depth image.

[0082] Optionally, the depth camera can also be an infrared camera. This infrared camera projects a speckle light array arranged according to a specific structural pattern using an infrared speckle projector. This array of light is reflected from the object's surface and imaged onto the infrared camera by an infrared sensor. Based on the principle of triangulation, the infrared camera can reconstruct the 3D coordinate information of the object's surface, thereby obtaining a depth image. Of course, the depth camera can also be a structured light depth camera; this application does not specifically limit the type of camera.

[0083] It should be understood that the depth camera includes a smart camera, which may be a 3D camera with a liveness detection function; optionally, the smart camera may include an image sensor for acquiring image data, which may be any one or more of a depth image sensor or an infrared image sensor.

[0084] It should be understood that if the face in the facial recognition scenario is not stationary, the acquisition time and frequency of the depth camera and color camera should be synchronized to achieve simultaneous acquisition of color 2D images and depth images. If the face in the facial recognition scenario is stationary, the depth camera and color camera can also acquire depth images and color 2D images in a time-sharing manner, but the acquisition must be performed from the same angle.

[0085] For example, the spatial relationship may include one or more combinations of spatial positional relationship and spatial attitude relationship.

[0086] It should be noted that the first face and the second face can be any two different faces from a plurality of faces.

[0087] It should be noted that color two-dimensional images include images in luminance-chrominance (YcbCr, YUV) format, images in RGB format, etc.; where Y represents luminance (Luma), cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma), used to describe color and saturation; this application does not impose specific restrictions on the format of color two-dimensional images.

[0088] It should be noted that depth images, also known as distance images, are images that use the distance (depth) from the image acquisition device to various points in the scene as pixel values. They directly reflect the geometry of the visible surfaces of the scene.

[0089] It should be noted that the user operation behavior targeting the first face or the second face may be that the user of the first face or the second face adjusts the spatial relationship, or that the user of the first face or the second face adds identity verification information. This application does not impose specific restrictions on the specific content of the prompt information.

[0090] Based on the above technical solution, firstly, multiple color 2D images of faces and their corresponding depth images are acquired, and the depth images are mapped onto the color 2D images to obtain a new depth image (color depth image). This is equivalent to mapping the depth image containing depth information onto the color 2D image, so that the resulting color depth image includes the three-dimensional coordinates of multiple faces in the color 2D image. Then, based on the position information of the first and second faces in the color depth image, the spatial relationship between the first and second faces is determined. This is equivalent to considering the impact of the spatial relationship between faces on the face-scanning device's ability to acquire the intended face during the face-scanning process. Finally, based on the spatial relationship between the first and second faces, prompts are output regarding user actions related to the first or second face. This is equivalent to outputting different prompts for different user actions related to the first or second face based on different spatial relationships. These prompts help prevent unintentional face-scanning users from accidentally using the device, improving the user experience and avoiding financial losses for unintentional users.

[0091] In some embodiments of this application, S302 may include:

[0092] Based on the coordinate system transformation relationship between the depth image and the color 2D image, the depth image is mapped to the spatial coordinate system where the color 2D image is located to obtain the color depth image.

[0093] In other words, based on the coordinate system transformation relationship between the depth image and the color 2D image, the depth image and the color 2D image are spatially aligned to obtain the color depth image.

[0094] It should be noted that, since there is a certain positional offset between the depth camera that captures the depth image and the color camera that captures the color 2D image, it is necessary to combine the intrinsic parameters of the two cameras and the extrinsic parameters between the two cameras to determine the coordinate transformation relationship of corresponding pixels in the depth image and the color 2D image, and then perform the transformation based on the transformation relationship to make the depth image and the color 2D image spatially aligned.

[0095] Based on this, in one implementation, the depth image is converted into a first point cloud image using the intrinsic parameters of the depth camera, and the depth camera is a camera that acquires the depth image; the first point cloud image is converted into a second point cloud image using the extrinsic parameters between the depth camera and the color camera, and the color camera is a camera that acquires the color two-dimensional image; the second point cloud image is converted into the color depth image using the intrinsic parameters of the color camera.

[0096] It should be noted that the intrinsic parameters of the two cameras and the extrinsic parameters between the two cameras are obtained by calibrating the two cameras.

[0097] The following will combine Figure 4 The process of spatially aligning the depth image and the color 2D image is described in detail.

[0098] Figure 4 This is a schematic block diagram 400 illustrating spatial alignment of a depth image and a color 2D image provided in an embodiment of this application.

[0099] like Figure 4 As shown, firstly, using the intrinsic parameters of the depth camera, the pixel coordinates in the depth image are transformed to camera coordinates in the depth camera coordinate system to obtain the first point cloud image; secondly, using the extrinsic parameters between the depth camera and the color camera, the coordinates in the depth camera coordinate system corresponding to the first point cloud image are transformed to coordinates in the color camera coordinate system to obtain the second point cloud image; finally, using the intrinsic parameters of the color camera, the coordinates in the color camera coordinate system corresponding to the second point cloud image are transformed to coordinates in the pixel coordinate system to obtain the color depth image.

[0100] For example, the transformation between the camera coordinate system and the pixel coordinate system can be combined Figure 1c For example, Figure 1cAs shown, point P is a point in the camera coordinate system, and point p′ is a point in the pixel coordinate system. The coordinate transformation relationship between the two can be calculated using the camera's intrinsic parameters. Specifically, the transformation between the camera coordinate system and the pixel coordinate system can be achieved using the following formula (1). It should be noted that the coordinates in the camera coordinate system represent coordinates in 3D space, while the coordinates in the pixel coordinate system represent coordinates in 2D space.

[0101]

[0102] Among them, (f x f y Let (u0, v) be the coordinates of the focus. o Let Z be the coordinates of the optical center. c Let (u, v) be the distance between the object surface and the camera plane, and (x, v) be the pixel coordinates of point p′. c y c , z c () represents the camera coordinates of point P.

[0103] Because there is an angular and distance offset between the camera acquiring depth images and the camera acquiring color 2D images, the depth image needs to be rotated and offset to align with the color 2D image in space. Furthermore, since the transformation relationship between the depth camera coordinate system and the color camera coordinate system is similar to the transformation relationship between the world coordinate system and the camera coordinate system, the following will combine... Figure 5 An example is provided to illustrate the transformation relationship between the depth camera coordinate system and the color camera coordinate system.

[0104] Figure 5 This is an example diagram illustrating the transformation relationship between the depth camera coordinate system and the color camera coordinate system.

[0105] It should be noted that the transformation from the world coordinate system to the camera coordinate system is a rigid body transformation, meaning that the object does not undergo deformation, and only rotation and translation are required.

[0106] like Figure 5 As shown, the coordinates (x) under the depth camera w y w , z w The coordinates (x, y) are transformed to those of the color camera through rotation and translation. c y c , z c ).

[0107] Where R represents the rotation matrix, This represents the offset vector.

[0108] Specifically, the coordinates (x, y) under the depth camera can be obtained by combining formula (2). w y w, z w Convert to coordinates under a color camera (x) c y c , z c ).

[0109]

[0110] Where, r 00 to r 22 The resulting matrix is ​​the rotation matrix R,t described above. x To t z The resulting matrix is ​​the aforementioned offset vector.

[0111] Of course, in other alternative embodiments, the parameters of the depth camera and the color camera can be calibrated, and the depth image and the color 2D image can be spatially aligned using the calibrated parameters to obtain the color depth image.

[0112] In some embodiments of this application, prior to S303, the method may further include:

[0113] Face detection is performed on the color two-dimensional image to obtain the two-dimensional coordinate information of the first face and the second face in the color two-dimensional image;

[0114] The two-dimensional coordinate information of the first face and the second face in the color two-dimensional image is mapped onto the color depth image to obtain the depth information of the first face and the second face in the color depth image.

[0115] The two-dimensional coordinate information of the first face in the color two-dimensional image and the depth information of the first face in the color depth image are determined as the position information of the first face in the color depth image;

[0116] The two-dimensional coordinates of the second face in the color two-dimensional image and the depth information of the second face in the color depth image are used to determine the position information of the second face in the color depth image.

[0117] The location information may include the coordinates of the face bounding box and the coordinates of the face feature points.

[0118] It should be noted that the face bounding box can be a rectangle, and the coordinate information of the face bounding box includes the coordinate information of multiple key points used to represent the face bounding box; the coordinate information of the face feature points includes the coordinate information of multiple feature points of the face.

[0119] For example, firstly, face detection is performed on the color 2D image to obtain the 2D coordinate information of the face bounding boxes of multiple faces in the color 2D image. Then, facial feature points are extracted from the faces in each face bounding box to obtain the 2D coordinate information of the facial feature points in the color 2D image. Secondly, the 2D coordinate information of the face bounding box and the 2D coordinate information of the facial feature points are mapped onto the color depth image to obtain the 3D coordinate information of the face bounding box and the 3D coordinate information of the facial feature points.

[0120] By mapping the position information of the first face and the second face in the color two-dimensional image to the color depth image, it is equivalent to mapping the two-dimensional position information of the first face and the second face in the color two-dimensional image to the three-dimensional color depth image, thereby obtaining the three-dimensional position information of the first face and the second face, which lays the groundwork for determining the three-dimensional spatial relationship between the first face and the second face.

[0121] In some embodiments of this application, S303 may include:

[0122] Based on the position information of the first face and the second face in the color depth image, the spatial distance between the first face and the second face is determined.

[0123] When the spatial distance is less than a preset distance, the spatial positional relationship between the first face and the second face is determined based on the positional information of the first face and the second face in the color depth image, and the spatial relationship includes the spatial positional relationship.

[0124] For example, from the position information of the first face in the color depth image, the position information of n key points is filtered out, and from the position information of the second face in the color depth image, the position information of n key points corresponding to the n key points of the first face is filtered out respectively; the spatial distance D between the first face and the second face is obtained by the following formula (3):

[0125] It should be understood that the i-th key point among the n key points of the first face corresponds one-to-one with the i-th key point among the n key points of the second face. For example, the point at the left corner of the first face corresponds to the point at the left corner of the second face; and the point at the tip of the nose of the first face corresponds to the point at the tip of the nose of the second face. Here, n is a positive integer greater than 1, and i is a positive integer greater than or equal to 1 and less than or equal to n.

[0126]

[0127] Among them, (x 1i y 1i , z 1i (x) represents the location information of the i-th key point of the first face; 2i y 2i, z 2i ) represents the location information of the i-th key point of the second face.

[0128] In another example, if the position information of the center point of the first face's bounding box is (x1, y1, z1) and the position information of the center point of the second face's bounding box is (x2, y2, z2), then the distance between the center point of the first face's bounding box and the center point of the second face's bounding box is determined as the spatial distance between the first face and the second face.

[0129] As another example, the distance between the first facial feature point of the first face and the second facial feature point of the second face can also be determined as the spatial distance between the first face and the second face. It should be understood that there is a one-to-one correspondence between the first facial feature point and the second facial feature point; for example, if the first facial feature point is the point at the left corner of the first face, then the corresponding second facial feature point is the point at the left corner of the second face. It should be noted that the first facial feature point can be any feature point of the first face, and this application does not restrict the method of calculating the spatial distance between the first face and the second face.

[0130] For example, the preset distance can be obtained based on experience or based on historical erroneous data, and this application does not impose any specific restrictions on it.

[0131] It should be noted that this spatial relationship is used to characterize whether the first face and the second face are side-by-side or one in front of the other; for example, if the distance between the first face and the second face in the direction perpendicular to the acquisition device is less than a first distance, then the first face and the second face are side-by-side; otherwise, the first face and the second face are one in front of the other. It should be understood that the first distance is less than the aforementioned preset distance.

[0132] By determining the spatial distance between the first face and the second face, it is possible to determine whether one face will interfere with the other during face recognition. If the spatial distance is greater than or equal to a preset distance, it means that the spatial distance between the first face and the second face will not cause interference. Only when the spatial distance is less than the preset distance does it mean that the spatial distance between the first face and the second face will cause interference. If the spatial distance between the first face and the second face will cause interference, then the spatial positional relationship between the first face and the second face can be further determined. This is equivalent to laying the groundwork for formulating a strategy to prevent accidental face recognition and prompting the user.

[0133] In some embodiments of this application, S303 may further include:

[0134] Based on the position information of the first face and the second face in the color depth image, the relative pose of the first face and the second face is determined. The relative pose is used to characterize the spatial pose relationship between the first face and the second face, and the spatial relationship includes the spatial pose relationship.

[0135] It should be noted that the relative pose can be pose similarity.

[0136] It should be noted that the spatial pose relationship is used to characterize whether the poses of the first face and the second face are similar or different; for example, if the pose similarity between the first face and the second face is greater than or equal to a certain preset threshold, then the poses of the first face and the second face are similar, otherwise the poses of the first face and the second face are different.

[0137] In one implementation, a rotation matrix is ​​determined from the first face to the second face based on the position information of the first face and the second face in the color depth image; the rotation matrix is ​​converted into a rotation angle; and the relative pose of the first face and the second face is determined based on the rotation angle.

[0138] For example, from the position information of the first face in the color depth image, the position information of m key points is selected, and from the position information of the second face in the color depth image, the position information of m key points corresponding to the m key points of the first face is selected respectively. Based on the position information of the m key points of the first face and the m key points of the second face, the rotation matrix R from the first face to the second face can be calculated by the above formula (2). It should be noted that the j-th key point among the m key points of the first face and the j-th key point among the m key points of the second face correspond one-to-one; where m is a positive integer greater than 1; j is a positive integer greater than or equal to 1 and less than or equal to m, and the position information of each key point is the three-dimensional coordinates of each key point.

[0139] For example, the rotation matrix can be converted into Euler angles, that is, the rotation matrix can be converted into rotation angles.

[0140] It should be noted that, according to the rotation matrix, the rotation from the first face to the second face can be decomposed into rotations along three mutually perpendicular coordinate axes in space. That is, by defining a three-dimensional coordinate system, the product of the rotation matrices along the three coordinate axes represents the rotation state from the first face to the second face. These three rotation matrices can be represented as follows:

[0141]

[0142] That is, the rotation angles are β, α, and θ.

[0143] Based on this, if the rotation angle β is less than or equal to the fourth preset threshold, the rotation angle α is less than or equal to the fifth threshold, and the rotation angle is less than or equal to the sixth threshold, it is determined that the poses of the first face and the second face are similar; otherwise, it is determined that the poses of the first face and the second face are different.

[0144] Alternatively, rotation angles β, α, and θ can be used as inputs to predict the pose similarity between the first and second faces using a pre-trained first network model, thus determining whether the poses of the first and second faces are similar or different. It should be noted that this first network model is trained based on a sample set of multiple labeled rotation angles.

[0145] In another implementation, the positional information of the first face and the second face in the same color depth image are used as input. A pre-trained second network model is then used to directly predict the pose similarity between the first and second faces. It should be noted that this second network model is trained using a sample set of labeled 3D positional information of multiple sets of keypoints; each set of keypoints includes the 3D positional information of two faces sharing the same feature points.

[0146] In some embodiments of this application, S304 may include:

[0147] If the spatial relationship includes spatial position relationship and spatial posture relationship, then when the spatial position relationship is used to represent that the first face and the second face are side by side, and the spatial posture relationship is used to represent that the posture similarity between the first face and the second face is greater than or equal to a first preset threshold, a first prompt message is output. The first prompt message is used to remind the users of the first face and the second face not to scan their faces side by side.

[0148] In some embodiments of this application, S304 may include:

[0149] If the spatial relationship includes spatial position relationship and spatial pose relationship, then when the spatial position relationship is used to represent the front and back of the first face and the second face, and the spatial pose relationship is used to represent the pose similarity of the first face and the second face is greater than or equal to the second preset threshold, a second prompt message is output, which is used to prompt the input of face verification information.

[0150] It should be noted that the facial verification information can be the last four digits of the face's ID card or the last four digits of the face's mobile phone number; this application does not impose specific restrictions on this.

[0151] It should be noted that the first preset threshold and the second preset threshold can be the same.

[0152] Figure 6This is another illustrative flowchart of the face recognition anti-mistake method 600 provided in the embodiments of this application.

[0153] like Figure 6 As shown, the method 600 may include:

[0154] S601, acquire color 2D images of multiple faces and depth images of the multiple faces.

[0155] S602, Spatial alignment of the color two-dimensional image and the depth image is performed to obtain a color depth image.

[0156] For example, referring to section 4, firstly, using the intrinsic parameters of the depth camera, the pixel coordinates in the depth image are transformed to camera coordinates in the depth camera coordinate system to obtain the first point cloud image; secondly, using the extrinsic parameters between the depth camera and the color camera, the coordinates in the depth camera coordinate system corresponding to the first point cloud image are transformed to coordinates in the color camera coordinate system to obtain the second point cloud image; finally, using the intrinsic parameters of the color camera, the coordinates in the color camera coordinate system corresponding to the second point cloud image are transformed to coordinates in the pixel coordinate system to obtain the color depth image.

[0157] S603, perform face detection on the color two-dimensional image to obtain the two-dimensional coordinate information of the face bounding box of the first face and the two-dimensional coordinate information of the face feature points of the first face, as well as the two-dimensional coordinate information of the face bounding box of the second face and the two-dimensional coordinate information of the face feature points of the second face.

[0158] S604, the two-dimensional coordinate information of the face bounding box of the first face and the two-dimensional coordinate information of the face bounding box of the second face are mapped onto the color depth image respectively to obtain the three-dimensional coordinate information of the face bounding box of the first face and the three-dimensional coordinate information of the face bounding box of the second face.

[0159] For example, the two-dimensional coordinate information of the first face bounding box and the second face bounding box in the color two-dimensional image are mapped onto the color depth image to obtain the depth information of the first face bounding box and the second face bounding box in the color depth image. The two-dimensional coordinate information of the first face bounding box in the color two-dimensional image and the depth information of the first face bounding box in the color depth image are determined as the position information of the first face bounding box in the color depth image. Similarly, the two-dimensional coordinate information of the second face bounding box in the color two-dimensional image and the depth information of the second face bounding box in the color depth image are determined as the position information of the second face bounding box in the color depth image.

[0160] S605, the two-dimensional coordinate information of the facial feature points of the first face and the two-dimensional coordinate information of the facial feature points of the second face are respectively mapped into the color depth image to obtain the three-dimensional coordinate information of the facial feature points of the first face and the three-dimensional coordinate information of the facial feature points of the second face.

[0161] For example, the two-dimensional coordinate information of the facial feature points of the first face and the second face in the color two-dimensional image is mapped onto the color depth image to obtain the depth information of the facial feature points of the first face and the second face in the color depth image. The two-dimensional coordinate information of the facial feature points of the first face in the color two-dimensional image and the depth information of the facial feature points of the first face in the color depth image are determined as the position information of the facial feature points of the first face in the color depth image. Similarly, the two-dimensional coordinate information of the facial feature points of the second face in the color two-dimensional image and the depth information of the facial feature points of the second face in the color depth image are determined as the position information of the facial feature points of the second face in the color depth image.

[0162] S606, based on the three-dimensional coordinate information of the face bounding box of the first face, the three-dimensional coordinate information of the face bounding box of the second face, the three-dimensional coordinate information of the facial feature points of the first face, and the three-dimensional coordinate information of the facial feature points of the second face, calculate the spatial distance between the first face and the second face.

[0163] For example, from the position information of the first face in the color depth image, the position information of n key points is filtered out, and from the position information of the second face in the color depth image, the position information of n key points corresponding to the n key points of the first face is filtered out respectively; the spatial distance D between the first face and the second face is obtained by the following formula (3):

[0164] It should be understood that the i-th key point among the n key points of the first face corresponds one-to-one with the i-th key point among the n key points of the second face. For example, the point at the left corner of the first face corresponds to the point at the left corner of the second face; and the point at the tip of the nose of the first face corresponds to the point at the tip of the nose of the second face. Here, n is a positive integer greater than 1, and i is a positive integer greater than or equal to 1 and less than or equal to n.

[0165]

[0166] Among them, (x 1i y 1i , z 1i (x) represents the location information of the i-th key point of the first face; 2i y 2i , z 2i) represents the location information of the i-th key point of the second face.

[0167] S607, based on the three-dimensional coordinate information of the face bounding box of the first face, the three-dimensional coordinate information of the face bounding box of the second face, the three-dimensional coordinate information of the facial feature points of the first face, and the three-dimensional coordinate information of the facial feature points of the second face, calculate the spatial pose between the first face and the second face.

[0168] In one implementation, a rotation matrix is ​​determined from the first face to the second face based on the position information of the first face and the second face in the color depth image; the rotation matrix is ​​converted into a rotation angle; and the relative pose of the first face and the second face is determined based on the rotation angle.

[0169] S608 outputs prompt information based on the spatial distance between the first face and the second face, and the spatial pose between the first face and the second face, to avoid accidental scanning when multiple people are in the same frame.

[0170] For example, if the spatial relationship includes spatial position relationship and spatial pose relationship, then when the spatial position relationship is used to represent that the first face and the second face are side by side, and the spatial pose relationship is used to represent that the pose similarity of the first face and the second face is greater than or equal to a preset threshold, a first prompt message is output. The first prompt message is used to prompt the users of the first face and the second face not to scan their faces side by side.

[0171] For example, if the spatial relationship includes spatial position relationship and spatial pose relationship, then when the spatial position relationship is used to characterize the front and back of the first face and the second face, and the spatial pose relationship is used to characterize the pose similarity of the first face and the second face is less than the preset threshold, a second prompt message is output, which is used to prompt the input of face verification information.

[0172] It should be noted that in the above description, the terms "first" and "second" may be interchanged in a specific order or sequence where permitted, and should not be construed as a limitation of this application.

[0173] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application. It should also be understood that in the various method embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0174] The method provided in the embodiments of this application has been described above. The face anti-mistake device provided in the embodiments of this application will be described below.

[0175] Figure 7 This is an example of a block diagram 700 of the face recognition anti-mistake device provided in the embodiments of this application.

[0176] It should be noted that this face recognition anti-misoperation device is a face recognition terminal.

[0177] like Figure 7 As shown, the face recognition anti-mistake device 700 may include some or all of the following:

[0178] The acquisition unit 710 is used to acquire color two-dimensional images of multiple faces and depth images of the multiple faces;

[0179] The mapping unit 720 is used to map the depth image onto the color two-dimensional image to obtain a color depth image;

[0180] The determining unit 730 is used to determine the spatial relationship between the first face and the second face based on the position information of the first face and the second face in the color depth image, respectively.

[0181] The output unit 740 is used to output prompt information based on the spatial relationship between the first face and the second face. The prompt information is used to prompt user operation behavior for the first face or the second face.

[0182] In some embodiments of this application, the mapping unit 720 may be specifically used for:

[0183] Based on the coordinate system transformation relationship between the depth image and the color 2D image, the depth image is mapped to the spatial coordinate system where the color 2D image is located to obtain the color depth image.

[0184] In some embodiments of this application, the mapping unit 720 may also be specifically used for:

[0185] Using the intrinsic parameters of the depth camera, the depth image is converted into a first point cloud image. The depth camera is the camera that acquired the depth image.

[0186] Using the extrinsic parameters between the depth camera and the color camera, the first point cloud image is transformed into a second point cloud image. The color camera is the camera that acquires the color two-dimensional image.

[0187] Using the intrinsic parameters of the color camera, the second point cloud image is converted into the color depth image.

[0188] In some embodiments of this application, the determining unit 730 may specifically be used for:

[0189] Face detection is performed on the color two-dimensional image to obtain the two-dimensional coordinate information of the first face and the second face in the color two-dimensional image;

[0190] The two-dimensional coordinate information of the first face and the second face in the color two-dimensional image is mapped onto the color depth image to obtain the depth information of the first face and the second face in the color depth image.

[0191] The two-dimensional coordinate information of the first face in the color two-dimensional image and the depth information of the first face in the color depth image are determined as the position information of the first face in the color depth image;

[0192] The two-dimensional coordinates of the second face in the color two-dimensional image and the depth information of the second face in the color depth image are used to determine the position information of the second face in the color depth image.

[0193] In some embodiments of this application, the location information includes:

[0194] The coordinates of the face bounding box and the coordinates of the face feature points.

[0195] In some embodiments of this application, the determining unit 730 may also be specifically used for:

[0196] Based on the position information of the first face and the second face in the color depth image, the spatial distance between the first face and the second face is determined.

[0197] When the spatial distance is less than a preset distance, the spatial positional relationship between the first face and the second face is determined based on the positional information of the first face and the second face in the color depth image, and the spatial relationship includes the spatial positional relationship.

[0198] In some embodiments of this application, the determining unit 730 may also be specifically used for:

[0199] Based on the position information of the first face and the second face in the color depth image, the relative pose of the first face and the second face is determined. The relative pose is used to characterize the spatial pose relationship between the first face and the second face, and the spatial relationship includes the spatial pose relationship.

[0200] In some embodiments of this application, the determining unit 730 may also be specifically used for:

[0201] Based on the position information of the first face and the second face in the color depth image, a rotation matrix is ​​determined from the first face to the second face;

[0202] Convert the rotation matrix into rotation angles;

[0203] Based on the rotation angle, the relative poses of the first face and the second face are determined.

[0204] In some embodiments of this application, the output unit 740 may be specifically used for:

[0205] If the spatial relationship includes spatial position relationship and spatial posture relationship, then when the spatial position relationship is used to represent that the first face and the second face are side by side, and the spatial posture relationship is used to represent that the posture similarity between the first face and the second face is greater than or equal to a first preset threshold, a first prompt message is output. The first prompt message is used to remind the users of the first face and the second face not to scan their faces side by side.

[0206] In some embodiments of this application, the output unit 740 may also be specifically used for:

[0207] If the spatial relationship includes spatial position relationship and spatial pose relationship, then when the spatial position relationship is used to represent the front and back of the first face and the second face, and the spatial pose relationship is used to represent the pose similarity of the first face and the second face is greater than or equal to the second preset threshold, a second prompt message is output, which is used to prompt the input of face verification information.

[0208] It should be understood that the embodiments of the face recognition anti-mistake device and the method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the face recognition anti-mistake device 700 can correspond to the corresponding subject in the methods 300 and 600 of the embodiments of this application, and each unit in the face recognition anti-mistake device 700 is for implementing the corresponding process in method 300 and method 600, which will not be repeated here for the sake of brevity.

[0209] It should also be understood that the various units in the face recognition anti-mistake device 700 involved in the embodiments of this application can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This can achieve the same operation without affecting the technical effect of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the face recognition anti-mistake device 700 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by multiple units working together. According to another embodiment of this application, the face recognition anti-mistake device 700 involved in the embodiments of this application and the face payment method of the embodiments of this application can be implemented by running a computer program (including program code) capable of executing the steps involved in the corresponding method on a general-purpose computing device including processing elements and storage elements such as a central processing unit (CPU), random access storage medium (RAM), and read-only storage medium (ROM). The computer program may be recorded on, for example, a computer-readable storage medium, loaded into an electronic device through the computer-readable storage medium, and run therein to implement the corresponding methods of the embodiments of this application.

[0210] In other words, the units mentioned above can be implemented in hardware, in software instructions, or in a combination of hardware and software. Specifically, the steps of the method embodiments in this application can be completed by the integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software in the decoding processor. Optionally, the software can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads the information in the memory and completes the steps in the above method embodiments in conjunction with its hardware.

[0211] Figure 8This is a schematic structural diagram of the electronic device 800 provided in the embodiments of this application.

[0212] like Figure 8 As shown, the electronic device 800 includes at least a processor 810 and a computer-readable storage medium 820. The processor 810 and the computer-readable storage medium 820 can be connected via a bus or other means. The computer-readable storage medium 820 stores a computer program 821, which includes computer instructions. The processor 810 executes the computer instructions stored in the computer-readable storage medium 820. The processor 810 is the computing and control core of the electronic device 800, and is adapted to implement one or more computer instructions, specifically to load and execute one or more computer instructions to achieve a corresponding method flow or function.

[0213] As an example, processor 810 may also be referred to as a central processing unit (CPU). Processor 810 may include, but is not limited to: general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0214] As an example, the computer-readable storage medium 820 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device; optionally, it may also be at least one computer-readable storage medium located remotely from the aforementioned processor 810. Specifically, the computer-readable storage medium 820 includes, but is not limited to, volatile memory and / or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0215] In one implementation, the electronic device 800 can be Figure 7 The face recognition anti-mistake device 700 shown; the computer-readable storage medium 820 stores computer instructions; the processor 810 loads and executes the computer instructions stored in the computer-readable storage medium 820 to achieve... Figure 3 and Figure 6 The corresponding steps in the method embodiment shown are as follows; in specific implementation, the computer instructions in the computer-readable storage medium 820 are loaded and executed by the processor 810, and will not be described again here to avoid repetition.

[0216] According to another aspect of this application, embodiments of this application also provide a computer-readable storage medium (Memory), which is a memory device in an electronic device 800 for storing programs and data. For example, a computer-readable storage medium 820. It is understood that the computer-readable storage medium 820 here may include both the built-in storage medium in the electronic device 800 and extended storage media supported by the electronic device 800. The computer-readable storage medium provides storage space that stores the operating system of the electronic device 800. Furthermore, the storage space also stores one or more computer instructions suitable for loading and execution by a processor 810, which may be one or more computer programs 821 (including program code).

[0217] The electronic device 800 may further include a transceiver 830, which can be connected to the processor 810 or a computer-readable storage medium 820.

[0218] The computer-readable storage medium 820 can control the transceiver 830 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 830 may include a transmitter and a receiver. The transceiver 830 may further include antennas, and the number of antennas may be one or more.

[0219] According to another aspect of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. For example, computer program 821. In this case, electronic device 800 may be a computer, and processor 810 reads the computer instructions from computer-readable storage medium 820, and executes the computer instructions, causing the computer to perform the image detection method provided in the various alternative embodiments described above.

[0220] In other words, when implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes of the embodiments of this application are run or the functions of the embodiments of this application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0221] Those skilled in the art will recognize that the units and process steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0222] Finally, it should be noted that the above embodiments are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for preventing accidental facial recognition scanning, characterized in that, include: Acquire color 2D images of multiple faces and depth images of the multiple faces; The depth image is mapped onto the color two-dimensional image to obtain a color depth image; Based on the position information of the first face and the second face in the color depth image, the spatial relationship between the first face and the second face is determined. Based on the spatial relationship between the first face and the second face, a prompt message is output, which is used to prompt user operation behavior targeting the first face or the second face; The step of determining the spatial relationship between the first face and the second face based on the position information of the first face and the second face in the color depth image includes: determining the relative pose of the first face and the second face based on the position information of the first face and the second face in the color depth image, wherein the relative pose is used to characterize the spatial pose relationship between the first face and the second face, and the spatial relationship includes the spatial pose relationship. The step of determining the spatial relationship between the first face and the second face based on the position information of the first face and the second face in the color depth image further includes: determining the spatial distance between the first face and the second face based on the position information of the first face and the second face in the color depth image; and determining the spatial positional relationship between the first face and the second face based on the position information of the first face and the second face in the color depth image when the spatial distance is less than a preset distance, wherein the spatial relationship includes the spatial positional relationship. The step of outputting prompt information based on the spatial relationship between the first face and the second face includes: if the spatial relationship includes spatial position relationship and spatial posture relationship, then when the spatial position relationship is used to represent that the first face and the second face are side by side, and the spatial posture relationship is used to represent that the posture similarity of the first face and the second face is greater than or equal to a first preset threshold, a first prompt information is output. The first prompt information is used to prompt the users of the first face and the second face not to scan their faces side by side. If the distance between the first face and the second face in the direction perpendicular to the acquisition device is less than a first distance, then the first face and the second face are side by side.

2. The method according to claim 1, characterized in that, The step of mapping the depth image onto the color two-dimensional image to obtain a color depth image includes: Based on the coordinate system transformation relationship between the depth image and the color 2D image, the depth image is mapped to the spatial coordinate system where the color 2D image is located to obtain the color depth image.

3. The method according to claim 2, characterized in that, The step of mapping the depth image to the spatial coordinate system of the color 2D image based on the coordinate system transformation relationship between the depth image and the color 2D image to obtain the color depth image includes: Using the intrinsic parameters of a depth camera, the depth image is converted into a first point cloud image, where the depth camera is the camera that acquires the depth image; Using the extrinsic parameters between the depth camera and the color camera, the first point cloud image is converted into a second point cloud image, wherein the color camera is a camera that acquires the color two-dimensional image; Using the intrinsic parameters of the color camera, the second point cloud image is converted into the color depth image.

4. The method according to claim 1, characterized in that, Before determining the spatial relationship between the first face and the second face based on their respective position information in the color depth image, the method further includes: Face detection is performed on the color two-dimensional image to obtain the two-dimensional coordinate information of the first face and the second face in the color two-dimensional image respectively; The two-dimensional coordinate information of the first face and the second face in the color two-dimensional image is mapped onto the color depth image to obtain the depth information of the first face and the second face in the color depth image. The two-dimensional coordinate information of the first face in the color two-dimensional image and the depth information of the first face in the color depth image are determined as the position information of the first face in the color depth image; The two-dimensional coordinate information of the second face in the color two-dimensional image and the depth information of the second face in the color depth image are used to determine the position information of the second face in the color depth image.

5. The method according to claim 4, characterized in that, The location information includes: The coordinates of the face bounding box and the coordinates of the face feature points.

6. The method according to claim 1, characterized in that, Determining the relative pose of the first face and the second face based on their respective position information in the color depth image includes: Based on the position information of the first face and the second face in the color depth image, a rotation matrix is ​​determined to rotate from the first face to the second face; Convert the rotation matrix into rotation angles; Based on the rotation angle, the relative poses of the first face and the second face are determined.

7. The method according to claim 1, characterized in that, Based on the spatial relationship between the first face and the second face, the system outputs prompt information, including: If the spatial relationship includes spatial position relationship and spatial pose relationship, then when the spatial position relationship is used to characterize the first face and the second face before and after, and the spatial pose relationship is used to characterize the pose similarity between the first face and the second face is greater than or equal to a second preset threshold, a second prompt message is output, which is used to prompt for input of face verification information.

8. A face recognition anti-mistake device, characterized in that, include: The acquisition unit is used to acquire color two-dimensional images of multiple faces and depth images of the multiple faces; A mapping unit is used to map the depth image onto the color two-dimensional image to obtain a color depth image; The determining unit is used to determine the spatial relationship between the first face and the second face based on the position information of the first face and the second face in the color depth image, respectively, among the plurality of faces; The output unit is used to output prompt information based on the spatial relationship between the first face and the second face, the prompt information being used to prompt user operation behavior targeting the first face or the second face; The determining unit is specifically used to: determine the relative pose of the first face and the second face based on the position information of the first face and the second face in the color depth image, wherein the relative pose is used to characterize the spatial pose relationship between the first face and the second face, and the spatial relationship includes the spatial pose relationship; The determining unit is further configured to: determine the spatial distance between the first face and the second face based on the position information of the first face and the second face in the color depth image; and, if the spatial distance is less than a preset distance, determine the spatial positional relationship between the first face and the second face based on the position information of the first face and the second face in the color depth image, wherein the spatial relationship includes the spatial positional relationship. The output unit is specifically used to: if the spatial relationship includes spatial position relationship and spatial posture relationship, then when the spatial position relationship is used to characterize the first face and the second face being side by side, and the spatial posture relationship is used to characterize the posture similarity of the first face and the second face being greater than or equal to a first preset threshold, output a first prompt message. The first prompt message is used to prompt the users of the first face and the second face not to scan their faces side by side. If the distance between the first face and the second face in the direction perpendicular to the acquisition device is less than a first distance, then the first face and the second face are side by side.

9. An electronic device, characterized in that, include: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1 to 7.

Images