Human face feature recognition method, device and storage medium based on skeleton points

Abstract

The present invention discloses a facial feature recognition method, device and storage medium based on skeleton points, wherein the recognition method comprises S1: receiving facial data of a standard person, establishing a face stereo model according to the facial data, and identifying and marking each skeleton point in the face stereo model; S2: receiving a face image with depth data of a target person, identifying at least three skeleton points in the face image, marking three of the skeleton points as reference points, and constructing a first reference plane model in three-dimensional space in combination with the depth data of the three reference points; S3: selecting skeleton points corresponding to the three reference points in the face stereo model, constructing a second reference plane according to the three reference points in the face stereo model, and comparing the areas of the first reference plane and the second reference plane. If they are the same, the target person is successfully recognized. The present invention uses the relationship between skeleton points as a facial feature, which can improve the accuracy of facial feature recognition.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a face feature recognition method, device and storage medium based on skeleton points. Background Art

[0002] At present, the application field of face recognition technology is very wide, and the existing face recognition methods generally use cameras to collect the current image of the face, and then compare the image with the pre-stored standard image. If the similarity reaches the preset value, the recognition is successful, and if the similarity is low, the recognition is unsuccessful. However, this method is based on two-dimensional images and cannot collect more three-dimensional features unique to the face, making the anti-counterfeiting effect of face recognition poor. Anyone only needs to take a photo of the person concerned to complete the face recognition action, which is less secure. In addition, during the face image acquisition process, if there is occlusion by external objects or other situations that make the image features unclear, it will directly affect the recognition results and cause recognition failure. Summary of the invention

[0003] In order to overcome the deficiencies of the prior art, one of the purposes of the present invention is to provide a facial feature recognition method based on skeleton points, which uses the relationship between skeleton points as facial features to improve the accuracy of facial feature recognition.

[0004] A second object of the present invention is to provide an electronic device.

[0005] A third object of the present invention is to provide a storage medium.

[0006] One of the purposes of the present invention is achieved by the following technical solution:

[0007] The facial feature recognition method based on skeleton points includes:

[0008] Step S1: receiving facial data of a standard person, establishing a 3D facial model according to the facial data, and identifying and marking each bone point in the 3D facial model;

[0009] Step S2: receiving a face image of a target person with depth data, identifying at least three skeleton points in the face image, marking the three skeleton points as reference points, and constructing a first reference plane model in a three-dimensional space in combination with the depth data of the three reference points;

[0010] Step S3: Select the skeleton points corresponding to the three reference points in the face 3D model, construct a second reference plane based on the three reference points in the face 3D model, and compare the areas of the first reference plane and the second reference plane. If they are the same, the target person is successfully identified.

[0011] Furthermore, the facial image is directly generated by photographing the face from the front of the laser radar camera device, and the camera device detects the distance between each position of the face and the camera device to obtain the depth data of the face.

[0012] Furthermore, the facial image is generated by tilting the face by shooting the face with a mobile phone terminal having a lidar camera function and a device tilt angle detection function, the lidar camera function is used to measure the distance between the terminal and each bone point, the device tilt angle detection function is used to obtain the tilt angle between the terminal and the horizontal line, and the depth data of each bone point is calculated based on the distance and angle.

[0013] Furthermore, the reference points at least include a bone point at the nose tip.

[0014] Furthermore, when the skeleton points are marked as reference points in step S2, the method further includes: connecting any two of the three skeleton points marked as reference points, determining whether the line segments between any two skeleton points overlap, and if so, deleting one of the reference points and recombining them into a new reference plane.

[0015] Furthermore, when identifying the skeleton points in the face image in step S2, the face image is preliminarily identified in grayscale, and the area in the face image whose grayscale value reaches a preset range is marked as an exclusion area, and only the face area outside the exclusion area is identified and marked in skeleton points.

[0016] Furthermore, the step S2 also includes: dividing the face region into at least two local regions of the same area, and the three bone points marked as reference points are not in the same local region.

[0017] Furthermore, when the areas of the first reference plane and the second reference plane are different in step S3, it is determined whether the difference between the two areas is within a preset error range. If so, the recognition is successful, otherwise, the recognition fails.

[0018] The second object of the present invention is achieved by adopting the following technical solution:

[0019] An electronic device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned human face feature recognition method based on skeleton points when executing the computer program.

[0020] The third object of the present invention is achieved by adopting the following technical solution:

[0021] A storage medium stores a computer program, which implements the above-mentioned facial feature recognition method based on skeleton points when executed.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The depth of each bone point in the face is used to reflect the facial features, and the reference plane formed between the three bone points is used as the basis for face recognition. This can improve the accuracy of face recognition and avoid the problem of unsuccessful recognition due to partial occlusion of the face. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of the facial feature recognition method based on skeleton points of the present invention;

[0025] Figure 2 Schematic diagram of reference points and reference planes in the human face three-dimensional model of the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0027] Embodiment 1

[0028] This embodiment provides a facial feature recognition method based on bone points. The method recognizes facial bone features and then recognizes the facial bone features in three-dimensional space, which can greatly improve the recognition accuracy.

[0029] like Figure 1 As shown, the facial feature recognition method of this embodiment specifically includes the following steps:

[0030] Step S1: receiving facial data of a standard person, establishing a 3D facial model based on the facial data, and identifying and marking each bone point in the 3D facial model.

[0031] The facial data of a standard person can be collected through a three-dimensional scanning device. The three-dimensional scanning device scans the face of the standard person to establish a three-dimensional facial model of the standard person. In the three-dimensional facial model, each bone point of the face is identified and marked to display information such as the position of each bone point in the three-dimensional facial model.

[0032] Step S2: receiving a face image of a target person with depth data, identifying at least three skeleton points in the face image, marking three of the skeleton points as reference points, and constructing a first reference plane model in three-dimensional space in combination with the depth data of the three reference points.

[0033] The face of the target person can be photographed by a laser radar camera to generate a facial image. During the shooting process, the laser radar camera is facing the face, and the laser radar camera sends out a detection beam to scan the face. After the detection beam reaches the face, the beam is reflected. After the laser radar device receives the transmitted beam, the distance from the device to each point on the face is obtained based on the reflected information, thereby obtaining the depth data of each point on the face.

[0034] In addition, a face image can also be generated by photographing a face at an oblique angle using a mobile phone terminal that has a lidar camera function and a device tilt angle detection function. When photographing a face using the lidar camera function of the terminal, the distance between the terminal and each bone point is measured, and then the tilt angle between the terminal and the horizontal line is obtained using the device tilt angle detection function. The horizontal distance from the terminal to each bone point is calculated based on the distance between the terminal and each bone point and the tilt angle of the terminal. The horizontal distance is the depth data of each bone point.

[0035] In this embodiment, the face image and its corresponding depth data are imported into the computer. Since the local position of the face image may be unclear due to dim light during the actual shooting and scanning process, the accuracy of identifying the skeleton points is not high when the face image is unclear. Therefore, after obtaining the face image, the face image is pre-processed for grayscale recognition, and the area in the face image where the grayscale value reaches a preset range is marked as an exclusion area. The area where the grayscale value reaches the preset range may be a shadow part formed due to insufficient light during the shooting process. In order to avoid the situation where the point cannot be accurately identified due to the poor clarity of the shadow part, only the face area outside the exclusion area is identified and marked for the skeleton points to ensure that the position of the reference point is accurate.

[0036] After the skeleton points of the face area outside the exclusion area are recognized, three reference points are selected from them. Since each reference point has its own depth data, that is, each reference point has its own three-dimensional coordinates in the three-dimensional space, the three reference bands can form a surface in the three-dimensional space. The first reference surface determined by the three reference points is used to represent the characteristics of the face to complete the subsequent recognition action.

[0037] Based on the general feature of the human face, the tip of the nose is relatively prominent. When photographing the face, the position of the nose tip will be captured no matter how the shooting angle changes, that is, the position of the nose tip will be seen regardless of the side view, front view, or pitch angle. In addition, the nose tip is relatively prominent, so the height difference between it and the positions of other facial bone points is relatively obvious. Therefore, when selecting the reference point, the bone point corresponding to the nose tip is selected as the first reference point, and the remaining two reference points are arbitrarily selected outside the facial exclusion area to achieve the selection of three reference points.

[0038] The facial area can also be divided into at least two local areas of the same area, which can be the left half of the face or the right half of the face. If the three bone points marked as reference points are not in the same local area, you can select one reference point on the left half of the face and one reference point on the right half of the face. The tip of the nose can also be used as a reference point. A relatively large reference plane is formed between the three reference points to highlight the facial features, making the subsequent comparison process more accurate.

[0039] When three skeletal points are selected as reference points, the relationship between the three selected reference points also needs to be judged: among the three skeletal points marked as reference points, connect any two skeletal points. Three line segments can be generated by connecting two of them. Determine whether the line segments between any two skeletal points overlap. If they overlap, delete one of the reference points, and reselect new reference points to form a new reference plane. The deleted reference point can be any point other than the tip of the nose, that is, the reference point at the tip of the nose is always retained, so that the inclination of the reference plane composed of the three reference points is relatively large, highlighting the facial features.

[0040] The reference points can be selected manually or randomly by the system in step S2. After the three reference points are determined, the model of the first reference surface can be constructed according to the depth data of the three reference points.

[0041] Step S3: Select the skeleton points corresponding to the three reference points in the face 3D model, construct a second reference plane based on the three reference points in the face 3D model, and compare the areas of the first reference plane and the second reference plane. If they are the same, the target person is successfully identified.

[0042] After the three reference points are determined, the bone points corresponding to the three reference points are selected in the face 3D model. For example, Figure 2As shown, after selecting the three reference points of "nose tip", "left eye socket" and "right eye socket" in step S2, the three skeletal points of "nose tip", "left eye socket" and "right eye socket" are selected in the human face stereo model, and the second reference plane of the human face stereo model is formed by using the depth data of each skeletal point in the human face stereo model, and then the areas of the first reference plane and the second reference plane are compared. If they are the same, it means that the depth data of the three skeletal points of the target person and the standard person are the same, that is, the facial features of the target person and the standard person are the same, so as to determine that the target person and the standard person are the same person; if the areas of the first reference plane and the second reference plane are different, it can also be determined whether the difference between the two areas is within a preset error range. If so, it can also be considered that the target person and the standard person are the same person. If the areas of the first reference plane and the second reference plane are different, and the difference between the two areas exceeds the error range, it is considered that the distribution or depth data of the skeletal points of the target person and the standard person are different, then it can be considered that the target person and the standard person are not the same person, and the recognition fails.

[0043] In addition, after generating the first reference plane and the second reference plane, the inclinations of the first reference plane and the second reference plane can be compared, that is, the angles between the first reference plane and the second reference plane and the horizontal plane can be compared respectively. If they are the same, it is considered that the target person and the standard person are the same person; if they are different, it is considered that the target person and the standard person are not the same person.

[0044] Embodiment 2

[0045] This embodiment provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the facial feature recognition method based on skeleton points in Embodiment 1 when executing the computer program; in addition, this embodiment also provides a storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned facial feature recognition method based on skeleton points when executed.

[0046] The device and storage medium in this embodiment and the method in the aforementioned embodiment are two aspects based on the same inventive concept. The method implementation process has been described in detail above, so those skilled in the art can clearly understand the structure and implementation process of the system in this embodiment based on the aforementioned description. For the sake of brevity of the specification, they will not be repeated here.

[0047] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0048] Although the present invention has been described above with reference to various embodiments, it should be understood that many changes and modifications may be made without departing from the scope of the present invention. Therefore, it is intended that the above detailed description is considered to be illustrative rather than restrictive, and it should be understood that the following claims (including all equivalents) are intended to define the spirit and scope of the present invention. The above embodiments are only preferred embodiments of the present invention, and the scope of protection of the present invention cannot be limited thereto. Any non-substantial changes and replacements made by those skilled in the art on the basis of the present invention belong to the scope of protection claimed by the present invention.

Claims

1. A facial feature recognition method based on skeleton points, characterized in that: include: Step S1: receiving facial data of a standard person, establishing a 3D facial model according to the facial data, and identifying and marking each bone point in the 3D facial model; Step S2: receiving a face image of a target person with depth data, identifying at least three skeleton points in the face image, marking the three skeleton points as reference points, and constructing a first reference plane model in a three-dimensional space in combination with the depth data of the three reference points; When the skeleton points are marked as reference points in step S2, the step further includes: connecting any two of the three skeleton points marked as reference points, determining whether the line segments between the any two skeleton points overlap, and if so, deleting one of the reference points and recombining them into a new reference plane; When identifying the skeleton points in the face image in step S2, the face image is preliminarily identified in grayscale, and the area in the face image whose grayscale value reaches a preset range is marked as an exclusion area, and only the face area outside the exclusion area is identified and marked in skeleton points; The step S2 further comprises: dividing the face region into at least two local regions of equal area, wherein the three bone points marked as reference points are not in the same local region; Step S3: Select the skeleton points corresponding to the three reference points in the face 3D model, construct a second reference plane based on the three reference points in the face 3D model, and compare the areas of the first reference plane and the second reference plane. If they are the same, the target person is successfully identified.

2. The facial feature recognition method based on skeleton points according to claim 1 is characterized in that: The face image is directly generated by photographing the face from the front of a laser radar camera device, and the camera device detects the distance between each position of the face and the camera device to obtain depth data of the face.

3. The facial feature recognition method based on skeleton points according to claim 1, characterized in that: The facial image is generated by tilting the face by shooting the face with a mobile phone terminal having a laser radar camera function and a device tilt angle detection function. The laser radar camera function is used to measure the distance between the terminal and each bone point, and the device tilt angle detection function is used to obtain the tilt angle between the terminal and the horizontal line, and the depth data of each bone point is calculated based on the distance and angle.

4. The facial feature recognition method based on skeleton points according to claim 1, characterized in that: The reference points at least include the bone points of the nose tip.

5. The facial feature recognition method based on skeleton points according to claim 1, characterized in that: When the areas of the first reference plane and the second reference plane are different in step S3, it is determined whether the difference between the two areas is within a preset error range. If so, the recognition is successful, otherwise, the recognition fails.

6. An electronic device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for facial feature recognition based on skeleton points described in any one of claims 1 to 5 is implemented.

7. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the facial feature recognition method based on skeleton points according to any one of claims 1 to 5 is implemented.

Images