Face image recognition method, device, equipment and storage medium

Abstract

The present application relates to a facial image recognition method, device, equipment and storage medium, the method comprising: obtaining a facial image to be recognized and key points on the facial image; determining a first sub-image and a second sub-image in the facial image according to the key points on the facial image, the first sub-image being an image including eyes, and the second sub-image being an image including mouth and nose; calling a pre-trained image evaluation model, using the first sub-image and the second sub-image to obtain an expression evaluation result; when the expression evaluation result is the first evaluation result, generating a poster using the facial image. The method can avoid generating posters using images with poor expressions of people.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a facial image recognition method, device, equipment and storage medium. Background Art

[0002] Posters are important materials for film and television marketing and promotion. They interpret the theme and connotation of the film and television series, and influence and shape the audience's first impression of the film and television series. It can be said that the poster is the spokesperson for the film and television series itself, and it is inseparable from the reputation of the series.

[0003] However, for film and television drama posters, the original materials often come from the screenshots of the film and television drama. When using algorithms to automatically produce screenshots in large quantities, the screenshots will inevitably show poor expressions of the characters, such as talking, blinking, looking down, looking up, etc. Therefore, how to filter out such images with poor expressions of the characters has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides a facial image recognition method, apparatus, device and storage medium, which can avoid generating posters using images of people with poor facial expressions.

[0005] In a first aspect, the present application provides a method for facial image recognition, the method comprising:

[0006] Obtaining a face image to be recognized and key points on the face image;

[0007] Determine a first sub-image and a second sub-image in the face image according to the key points on the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including a mouth and a nose;

[0008] Calling a pre-trained image evaluation model, and using the first sub-image and the second sub-image to obtain an expression evaluation result;

[0009] When the expression evaluation result is the first evaluation result, a poster is generated using the face image.

[0010] Optionally, determining the first sub-image in the face image according to the key points on the face image includes:

[0011] Determine a first key point corresponding to an eye among the key points on the face image;

[0012] Determine a first area in the face image according to the first key point;

[0013] Dividing the facial image according to the first area to determine a third sub-image;

[0014] A first sub-image is obtained according to the third sub-image.

[0015] Optionally, determining the first sub-image according to the third sub-image includes:

[0016] Acquire the aspect ratio of the first sub-image;

[0017] When the aspect ratio satisfies a first preset condition, determining the aspect ratio as the number of repetitions of the first sub-image, and controlling the third sub-image to repeat in the vertical direction according to the number of repetitions to obtain the first sub-image;

[0018] When the aspect ratio meets the second preset condition, the reciprocal of the aspect ratio is determined as the number of repetitions of the first sub-image, and according to the number of repetitions, the third sub-image is controlled to be repeated in the horizontal direction to obtain the first sub-image.

[0019] Optionally, the pre-trained image evaluation model includes a first evaluation module, a second evaluation module and a comprehensive module, and the calling of the pre-trained image evaluation model and obtaining the expression evaluation result using the first sub-image and the second sub-image includes:

[0020] Inputting the first sub-image into the first evaluation module and outputting a first evaluation score;

[0021] inputting the second sub-image into the second evaluation module, and outputting a second evaluation score;

[0022] The comprehensive module is used to obtain an expression evaluation result using the first evaluation score and the second evaluation score.

[0023] Optionally, the using the comprehensive module to obtain an expression evaluation result using the first evaluation score and the second evaluation score includes:

[0024] determining an average evaluation score of the first evaluation score and the second evaluation score;

[0025] When the first evaluation score is greater than or equal to a first preset score, the second evaluation score is greater than or equal to a second preset score, and the average evaluation score is greater than or equal to a third preset score, determining the first evaluation result as the expression evaluation result;

[0026] When the first evaluation score is less than a first preset score, the second evaluation score is less than a second preset score, or the average evaluation score is less than a third preset score, the second evaluation result is determined as the expression evaluation result.

[0027] Optionally, the pre-trained image evaluation model further includes a third processing module, and the method further includes:

[0028] When the first evaluation score meets the third preset condition and the second evaluation score meets the fourth preset condition, the face image is input into the third processing module to obtain an expression evaluation result.

[0029] Optionally, determining a first area in the face image according to the first key point includes:

[0030] Determine a minimum circumscribed rectangle in the face image based on the first key point;

[0031] According to a preset first expansion ratio, the height of the minimum circumscribed rectangle is adjusted, and according to a preset second expansion ratio, the width of the minimum circumscribed rectangle is adjusted to obtain a circumscribed rectangle;

[0032] The area corresponding to the circumscribed rectangle is determined as the first area.

[0033] In a second aspect, the present application provides a face image recognition device, the device comprising:

[0034] An acquisition unit, used to acquire a face image to be recognized and key points on the face image;

[0035] a determining unit, configured to determine, in the face image, a first sub-image and a second sub-image according to key points on the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including a mouth and a nose;

[0036] A calling unit, configured to call a pre-trained image evaluation model and obtain an expression evaluation result using the first sub-image and the second sub-image;

[0037] A utilizing unit is used to generate a poster using the face image when the expression evaluation result is a first evaluation result.

[0038] Optionally, determine the unit for:

[0039] Determine a first key point corresponding to an eye among the key points on the face image;

[0040] Determine a first area in the face image according to the first key point;

[0041] Dividing the facial image according to the first area to determine a third sub-image;

[0042] A first sub-image is obtained according to the third sub-image.

[0043] Optionally, determine the unit for:

[0044] Acquire the aspect ratio of the first sub-image;

[0045] When the aspect ratio satisfies a first preset condition, determining the aspect ratio as the number of repetitions of the first sub-image, and controlling the third sub-image to repeat in the vertical direction according to the number of repetitions to obtain the first sub-image;

[0046] When the aspect ratio meets the second preset condition, the reciprocal of the aspect ratio is determined as the number of repetitions of the first sub-image, and according to the number of repetitions, the third sub-image is controlled to be repeated in the horizontal direction to obtain the first sub-image.

[0047] Optionally, the pre-trained image evaluation model includes a first evaluation module, a second evaluation module and a comprehensive module, and the calling unit is used to:

[0048] Inputting the first sub-image into the first evaluation module and outputting a first evaluation score;

[0049] inputting the second sub-image into the second evaluation module, and outputting a second evaluation score;

[0050] The comprehensive module is used to obtain an expression evaluation result using the first evaluation score and the second evaluation score.

[0051] Optionally, the calling unit is used to:

[0052] determining an average evaluation score of the first evaluation score and the second evaluation score;

[0053] When the first evaluation score is greater than or equal to a first preset score, the second evaluation score is greater than or equal to a second preset score, and the average evaluation score is greater than or equal to a third preset score, determining the first evaluation result as the expression evaluation result;

[0054] When the first evaluation score is less than a first preset score, the second evaluation score is less than a second preset score, or the average evaluation score is less than a third preset score, the second evaluation result is determined as the expression evaluation result.

[0055] Optionally, the pre-trained image evaluation model further includes a third processing module, and the device further includes an input unit, wherein the input unit is used to:

[0056] When the first evaluation score meets the third preset condition and the second evaluation score meets the fourth preset condition, the face image is input into the third processing module to obtain an expression evaluation result.

[0057] Optionally, determine the unit for:

[0058] Determine a minimum circumscribed rectangle in the face image based on the first key point;

[0059] According to a preset first expansion ratio, the height of the minimum circumscribed rectangle is adjusted, and according to a preset second expansion ratio, the width of the minimum circumscribed rectangle is adjusted to obtain a circumscribed rectangle;

[0060] The area corresponding to the circumscribed rectangle is determined as the first area.

[0061] In a third aspect, the present application provides a face image recognition device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to:

[0062] Obtaining a face image to be recognized and key points on the face image;

[0063] Determine a first sub-image and a second sub-image in the face image according to the key points on the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including a mouth and a nose;

[0064] Calling a pre-trained image evaluation model, and using the first sub-image and the second sub-image to obtain an expression evaluation result;

[0065] When the expression evaluation result is the first evaluation result, a poster is generated using the face image.

[0066] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the above-mentioned facial image recognition method when executed by a processor.

[0067] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: in the embodiment of the present application, a face image to be identified and key points on the face image are obtained; according to the key points on the face image, a first sub-image and a second sub-image are determined in the face image, the first sub-image is an image including eyes, and the second sub-image is an image including mouth and nose; a pre-trained image evaluation model is called, and the expression evaluation result is obtained using the first sub-image and the second sub-image; when the expression evaluation result is the first evaluation result, a poster is generated using the face image. Therefore, the embodiment of the present application evaluates the image including eyes and the image including mouth and nose through the pre-trained image evaluation model to obtain the expression evaluation result, and then when the expression evaluation result is the first evaluation result, a poster is generated using the face image, thereby avoiding the use of images with poor expressions to generate posters. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0070] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0071] Figure 1 A schematic diagram of a facial image recognition method provided in an embodiment of the present application;

[0072] Figure 2 A schematic diagram of a flow chart of a method for determining a first sub-image provided in an embodiment of the present application;

[0073] Figure 3 A schematic flow chart of a third sub-image conversion method provided in an embodiment of the present application;

[0074] Figure 4 A flowchart of a method for determining an expression evaluation result provided in an embodiment of the present application;

[0075] Figure 5 A flowchart of another method for determining an expression evaluation result provided in an embodiment of the present application;

[0076] Figure 6 A schematic diagram of a flow chart of a first region determination method provided in an embodiment of the present application;

[0077] Figure 7 A schematic flow chart of another method for determining a second sub-image provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of a process flow of a facial image recognition device provided in an embodiment of the present application;

[0079] Fig. 9 A schematic diagram of a facial image recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0081] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0082] Posters are important materials for film and television marketing and promotion. They interpret the theme and connotation of the film and television series, and influence and shape the audience's first impression of the film and television series. It can be said that the poster is the spokesperson for the film and television series itself, and it is inseparable from the reputation of the series. However, for the posters of film and television series, their original materials often come from the screenshots of the film and television series. When using algorithms to automatically produce screenshots in large quantities, the screenshots will inevitably show poor expressions of the characters, such as talking, blinking, looking down, looking up, and other ugly movements. Therefore, how to filter out such images with poor expressions of the characters has become an urgent problem to be solved.

[0083] In order to solve the above problems, the embodiment of the present application provides a method for face image recognition, which can avoid using images of people with poor facial expressions to generate posters, such as Figure 1 As shown, the specific steps include:

[0084] Step 101: Obtain a face image to be recognized and key points on the face image.

[0085] In this step, an image to be identified is obtained. If the image is a face image, the key points on the face image can be directly identified. If the image is not a face image, the face in the image can be identified first, and the image can be cropped based on the face to obtain a face image.

[0086] It should be noted that the execution subject of the embodiment of the present application can be a server or a terminal, which is not limited here. In practice, the terminal can process the image to be recognized based on the method provided in the embodiment of the present application, or upload it to the server so that the server processes the image to be recognized.

[0087] Step 102: Determine a first sub-image and a second sub-image in the face image according to key points on the face image.

[0088] The first sub-image is an image including eyes, and the second sub-image is an image including mouth and nose.

[0089] In this step, the key points corresponding to the eyes can be determined from the key points on the face image, and then the minimum bounding rectangle of these key points can be determined. According to the minimum bounding rectangle, the face image is divided to obtain the first sub-image. Alternatively, the minimum bounding rectangle can be expanded by a certain proportion to obtain a bounding rectangle, and the face image can be divided according to the bounding rectangle to obtain the first sub-image. Of course, the first sub-image can also be obtained based on other methods, which are not limited here.

[0090] In addition, the method for determining the second sub-image is similar to the method for determining the first sub-image and is not limited here.

[0091] Step 103, calling a pre-trained image evaluation model, and using the first sub-image and the second sub-image to obtain an expression evaluation result.

[0092] In this step, the first sub-image and the second sub-image can be directly input into a pre-trained image evaluation model to obtain an expression evaluation result, or the first sub-image and the second sub-image can be pre-processed first, and the pre-processed first sub-image and the second sub-image can be input into a pre-trained image evaluation model to obtain an expression evaluation result.

[0093] Among them, the preprocessing operation can be image denoising, image enhancement, image normalization and other related operations.

[0094] Step 104: when the expression evaluation result is the first evaluation result, a poster is generated using the face image.

[0095] Among them, the first evaluation result is that the facial expression is good.

[0096] In this step, when the expression evaluation result is good, it means that a poster can be generated based on the face image or a poster can be generated based on the screenshot of the face image. Therefore, a poster can be generated using the face image.

[0097] In an embodiment of the present application, a face image to be identified and key points on the face image are obtained; based on the key points on the face image, a first sub-image and a second sub-image are determined in the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including mouth and nose; a pre-trained image evaluation model is called, and the expression evaluation result is obtained using the first sub-image and the second sub-image; when the expression evaluation result is the first evaluation result, a poster is generated using the face image. Therefore, the embodiment of the present application evaluates the image including eyes and the image including mouth and nose using a pre-trained image evaluation model to obtain an expression evaluation result, and then when the expression evaluation result is the first evaluation result, a poster is generated using the face image, thereby avoiding the use of images with poor expressions to generate posters.

[0098] In the embodiment of the present application, the first sub-image can also be determined in the face image according to the key points on the face image. The embodiment of the present application provides a method for determining the first sub-image. Figure 2 As shown, the specific steps include:

[0099] Step 201, determining a first key point corresponding to the eyes among the key points on the face image.

[0100] Among them, the first key point is the key point around the eye. Generally speaking, there are four key points around each eye, namely the highest point of the eye, the lowest point of the eye, the rightmost point of the eye, and the leftmost point of the glasses.

[0101] In this step, each key point has a corresponding serial number, so the serial number corresponding to each key point around the eyes can be determined first, and the first key point can be found among the key points on the face image based on these serial numbers.

[0102] Step 202: determine a first area in the face image according to the first key point.

[0103] In this step, the minimum bounding rectangle of the first key point can be determined, and the area corresponding to the minimum bounding rectangle can be determined as the first area. The minimum bounding rectangle can also be adjusted to obtain an adjusted bounding rectangle, and the area corresponding to the bounding rectangle can be determined as the first area.

[0104] Step 203: divide the face image according to the first area to determine a third sub-image.

[0105] In this step, the face image is divided according to the first area to obtain a third sub-image corresponding to the first area.

[0106] Step 204: Obtain the first sub-image according to the third sub-image.

[0107] In this step, the third sub-image may be determined as the first sub-image, or other methods for determining the first sub-image may be used, which are not limited here.

[0108] In practice, in order to ensure accurate recognition of the machine learning model, the input image of the machine learning model is often square. However, the third sub-image is an image including the eyes, and the area corresponding to the eyes is often rectangular. Therefore, the third sub-image is often rectangular. Based on the above, it is also necessary to convert the rectangular third sub-image into the square first sub-image. Therefore, the embodiment of the present application provides a third sub-image conversion method, which is as follows: Figure 3 As shown, the specific steps include:

[0109] Step 301: Obtain the aspect ratio of the third sub-image.

[0110] In this step, the width and height of the third sub-image are obtained, and the aspect ratio is calculated according to the width and height.

[0111] Step 302: when the aspect ratio meets the first preset condition, the aspect ratio is determined as the number of repetitions of the third sub-image, and the third sub-image is controlled to be repeated in the vertical direction according to the number of repetitions to obtain the first sub-image.

[0112] The first preset condition is that the aspect ratio is greater than the first value. The first value is determined based on the experience of the technician. For example, when the aspect ratio is greater than the second preset value and less than the first preset value, the technician determines that the third sub-image does not need to be processed and directly determines the third sub-image as the first sub-image. When the aspect ratio is greater than the first value, the third sub-image needs to be processed based on the method described in this step to obtain the first sub-image. The first value and the second value may be the same or different. For example, the first value and the second value are both 1. Or the first value is 1.5 and the second value is 0.5.

[0113] In this step, when the aspect ratio meets the first preset condition, it means that the third sub-image is a rectangle with a relatively large width, and the height of the rectangle needs to be extended to be equal to the width. The method for extending the height is to determine the aspect ratio as the number of repetitions of the third sub-image, and according to the number of repetitions, control the third sub-image to be repeated vertically to achieve the extension of the height, thereby obtaining the first sub-image.

[0114] Step 303: when the aspect ratio meets the second preset condition, the inverse of the aspect ratio is determined as the number of repetitions of the third sub-image, and the third sub-image is controlled to be repeated horizontally according to the number of repetitions to obtain the first sub-image.

[0115] The second preset condition is that the aspect ratio is greater than 0 and less than a second value.

[0116] In this step, when the aspect ratio meets the second preset condition, it means that the third sub-image is a rectangle with a relatively large height, and the width of the rectangle needs to be extended to be equal to the height. The method for extending the width is to determine the aspect ratio as the number of repetitions of the third sub-image, and according to the number of repetitions, control the third sub-image to be repeated in the vertical direction to achieve the extension of the width, thereby obtaining the first sub-image.

[0117] In the embodiment of the present application, the pre-trained image evaluation model includes a first evaluation module, a second evaluation module and a comprehensive module. Therefore, these modules can also be used to process the first sub-image and the second sub-image to obtain an expression evaluation result. Therefore, the embodiment of the present application provides a method for determining an expression evaluation result. Figure 4 As shown, the specific steps include:

[0118] Step 401: input a first sub-image into a first evaluation module, and output a first evaluation score.

[0119] The first evaluation module is a machine learning model, which can be obtained by training alone or by training the entire image evaluation model. When the model is trained alone, the samples required are sample images including eyes and a benchmark evaluation score. In this step, when the expression conveyed by the eyes is better, the corresponding first evaluation score is higher, and when the expression conveyed by the eyes is worse, the corresponding first evaluation score is lower.

[0120] In this step, the first sub-image is input into a first evaluation module, and a first evaluation score is output.

[0121] Step 402: input the second sub-image into a second evaluation module, and output a second evaluation score.

[0122] The second evaluation module is a machine learning model, which can be obtained by training alone or by training the entire image evaluation model. When the model is trained alone, it requires samples including sample images of the mouth and nose and a benchmark evaluation score. In this step, when the expression conveyed by the mouth and nose is better, the corresponding second evaluation score is higher, and when the expression conveyed by the mouth and nose is worse, the corresponding second evaluation score is lower.

[0123] In this step, the second sub-image is input into the second evaluation module, and a second evaluation score is output.

[0124] Step 403, using the comprehensive module, using the first evaluation score and the second evaluation score, to obtain the expression evaluation result.

[0125] In this step, the facial expression is good only when the expression conveyed by the eyes is good, and the expressions conveyed by the mouth and nose are also good. Therefore, only when the first evaluation score is greater than or equal to the first preset score, and the second evaluation score is greater than or equal to the second preset score, the first evaluation result is determined as the expression evaluation result. When the first evaluation score is less than the first preset score, or the second evaluation score is less than the second preset score, the first evaluation result is determined as the expression evaluation result.

[0126] Among them, the first evaluation result is that the facial expression is good, and the second evaluation result is that the facial expression is not good.

[0127] Furthermore, the average evaluation score of the first evaluation score and the second evaluation score may be calculated, and then the expression evaluation result may be determined according to the average evaluation score. Of course, other expression evaluation results methods may also be used in this step, which are not limited here.

[0128] In this step, the comprehensive module is used to calculate the average evaluation score of the first evaluation score and the second evaluation score, and then the expression evaluation result is determined according to the first evaluation score, the second evaluation score and the average evaluation score. Therefore, the embodiment of the present application provides a method for determining the expression evaluation result. Figure 5 As shown, the specific steps include:

[0129] Step 501: determine the average evaluation score of the first evaluation score and the second evaluation score.

[0130] In this step, the first evaluation score and the second evaluation score are added, and the sum is divided by 2 to obtain an average evaluation score.

[0131] Step 502: when the first evaluation score is greater than or equal to the first preset score, the second evaluation score is greater than or equal to the second preset score, and the average evaluation score is greater than or equal to the third preset score, the first evaluation result is determined as the expression evaluation result.

[0132] In this step, it is detected whether the first evaluation score is less than the first preset score, whether the second evaluation score is less than the second preset score, and whether the average evaluation score is less than the third preset score. When the first evaluation score is greater than or equal to the first preset score, the second evaluation score is greater than or equal to the second preset score, and the average evaluation score is greater than or equal to the third preset score, the first evaluation result is determined as the expression evaluation result.

[0133] Step 503: when the first evaluation score is less than the first preset score, the second evaluation score is less than the second preset score, or the average evaluation score is less than the third preset score, the second evaluation result is determined as the expression evaluation result.

[0134] In this step, when the first evaluation score is less than the first preset score, the second evaluation score is less than the second preset score, or the average evaluation score is less than the third preset score, the second evaluation result is determined as the expression evaluation result.

[0135] It should be noted that, in order to reduce the amount of calculation, as long as one evaluation score is detected to be less than its corresponding preset score, the second evaluation result can be determined as the expression evaluation result. Specifically, it is sequentially detected whether the first evaluation score is less than the first preset score, whether the second evaluation score is less than the second preset score, and whether the average evaluation score is less than the third preset score. When it is detected that the first evaluation score is less than the first preset score, subsequent detection is not continued, and the second evaluation result is directly determined as the expression evaluation result.

[0136] In an embodiment of the present application, when the first evaluation score is close to the first preset score, or the second evaluation score is close to the second preset score, the detection result may be inaccurate. In order to obtain accurate evaluation results, a third processing module can also be set in the pre-trained image evaluation model, and the input of the third processing module is a face image, and the output is an expression evaluation result. The specific steps are: when the first evaluation score meets the third preset condition, or the second evaluation score meets the fourth preset condition, the face image is input into the third processing module to obtain the expression evaluation result.

[0137] Among them, the third processing model can be a machine learning model, the third processing module can be trained separately, or it can be trained on the image evaluation model. The training method is similar to the existing training method and will not be described here one by one.

[0138] In the embodiment of the present application, the minimum bounding rectangle of the eye can also be determined based on the first key point, the minimum bounding rectangle is expanded to obtain the bounding rectangle of the eye, and the area corresponding to the bounding rectangle is determined as the first area. Therefore, the embodiment of the present application provides a method for determining the first area. Figure 6 As shown, the specific steps include:

[0139] Step 601, determining a minimum circumscribed rectangle in the face image according to the first key point.

[0140] In this step, the first key points are connected to form a figure, and the minimum circumscribed rectangle corresponding to the figure is determined.

[0141] Step 602: According to a preset first expansion ratio, the height of the minimum circumscribed rectangle is adjusted, and according to a preset second expansion ratio, the width of the minimum circumscribed rectangle is adjusted to obtain a circumscribed rectangle.

[0142] The first expansion ratio is used to adjust the height, and the second expansion ratio is used to adjust the width. The first expansion ratio and the second expansion ratio can be set by technicians based on experience or by other methods, which are not limited here. The first expansion ratio and the second expansion ratio can be the same or different. For example, the first expansion ratio and the second expansion ratio can both be set to 1.1, or the first expansion ratio can be set to 1.1 and the second expansion ratio can be set to 1.2.

[0143] In this step, the height of the circumscribed rectangle is obtained by multiplying the preset first expansion ratio by the height of the minimum circumscribed rectangle, and the width of the circumscribed rectangle is obtained by multiplying the preset second expansion ratio by the width of the minimum circumscribed rectangle. The circumscribed rectangle is determined according to the height and width of the circumscribed rectangle.

[0144] Furthermore, in order to save the subsequent adjustment process, the circumscribed matrix can also be adjusted in the direction of a square. The specific steps are as follows: set two different expansion ratios without limiting the specific adjustment object. Obtain the width and height of the minimum circumscribed rectangle. When the width is greater than the height, adjust the width based on the smaller expansion ratio, and adjust the height based on the larger expansion ratio. When the height is greater than the width, adjust the height based on the smaller expansion ratio, and adjust the width based on the larger expansion ratio.

[0145] Step 603: determine the area corresponding to the circumscribed rectangle as the first area.

[0146] In this step, an area corresponding to the circumscribed rectangle is determined, and the area is determined as the first area.

[0147] In the embodiment of the present application, the second sub-image can also be determined in the face image according to the key points on the face image. The embodiment of the present application provides a method for determining the second sub-image. Figure 7 As shown, the specific steps include:

[0148] Step 701, among the key points on the face image, determine the second key points corresponding to the mouth and the nose.

[0149] Among them, the second key point is the key point around the eyes. Generally speaking, there are four key points around each eye, namely the highest point of the mouth, the lowest point of the mouth, the rightmost point of the mouth, the leftmost point of the mouth, the left side of the nose, and the right side of the nose.

[0150] In this step, each key point has a corresponding serial number, so the serial number corresponding to each key point around the eyes can be determined first, and then the second key point can be found at the key point on the face image based on these serial numbers.

[0151] Step 702: Determine a second area in the face image based on the second key point.

[0152] In this step, the minimum circumscribed square of the second key point can be determined, and the area corresponding to the minimum circumscribed square can be determined as the second area. The minimum circumscribed square can also be adjusted to obtain an adjusted circumscribed square, and the area corresponding to the circumscribed square can be determined as the second area.

[0153] Step 703: divide the face image according to the second area to determine a fourth sub-image.

[0154] In this step, the face image is divided according to the second area to obtain a fourth sub-image corresponding to the second area.

[0155] Step 704: Obtain a second sub-image according to the fourth sub-image.

[0156] In this step, the fourth sub-image may be determined as the second sub-image, and other methods for determining the second sub-image may also be used, which are not limited here.

[0157] like Figure 8 As shown, the embodiment of the present application provides a face image recognition device, which corresponds to the method embodiment and specifically includes:

[0158] An acquisition unit 801 is used to acquire a face image to be recognized and key points on the face image;

[0159] A determining unit 802 is configured to determine, in the face image, a first sub-image and a second sub-image according to key points on the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including a mouth and a nose;

[0160] A calling unit 803 is used to call a pre-trained image evaluation model and obtain an expression evaluation result using the first sub-image and the second sub-image;

[0161] The utilizing unit 804 is configured to generate a poster using the face image when the expression evaluation result is the first evaluation result.

[0162] Optionally, the determining unit 802 is configured to:

[0163] Determine a first key point corresponding to an eye among the key points on the face image;

[0164] Determine a first area in the face image according to the first key point;

[0165] Dividing the facial image according to the first area to determine a third sub-image;

[0166] A first sub-image is obtained according to the third sub-image.

[0167] Optionally, the determining unit 802 is configured to:

[0168] Acquire the aspect ratio of the first sub-image;

[0169] When the aspect ratio satisfies a first preset condition, determining the aspect ratio as the number of repetitions of the first sub-image, and controlling the third sub-image to repeat in the vertical direction according to the number of repetitions to obtain the first sub-image;

[0170] When the aspect ratio meets the second preset condition, the reciprocal of the aspect ratio is determined as the number of repetitions of the first sub-image, and according to the number of repetitions, the third sub-image is controlled to be repeated in the horizontal direction to obtain the first sub-image.

[0171] Optionally, the pre-trained image evaluation model includes a first evaluation module, a second evaluation module and a comprehensive module, and the calling unit 803 is used to:

[0172] Inputting the first sub-image into the first evaluation module and outputting a first evaluation score;

[0173] inputting the second sub-image into the second evaluation module, and outputting a second evaluation score;

[0174] The comprehensive module is used to obtain an expression evaluation result using the first evaluation score and the second evaluation score.

[0175] Optionally, the calling unit 803 is used to:

[0176] determining an average evaluation score of the first evaluation score and the second evaluation score;

[0177] When the first evaluation score is greater than or equal to a first preset score, the second evaluation score is greater than or equal to a second preset score, and the average evaluation score is greater than or equal to a third preset score, determining the first evaluation result as the expression evaluation result;

[0178] When the first evaluation score is less than a first preset score, the second evaluation score is less than a second preset score, or the average evaluation score is less than a third preset score, the second evaluation result is determined as the expression evaluation result.

[0179] Optionally, the pre-trained image evaluation model further includes a third processing module, and the device further includes an input unit 805, wherein the input unit 805 is used to:

[0180] When the first evaluation score meets the third preset condition and the second evaluation score meets the fourth preset condition, the face image is input into the third processing module to obtain an expression evaluation result.

[0181] Optionally, the determining unit 802 is configured to:

[0182] Determine a minimum circumscribed rectangle in the face image based on the first key point;

[0183] According to a preset first expansion ratio, the height of the minimum circumscribed rectangle is adjusted, and according to a preset second expansion ratio, the width of the minimum circumscribed rectangle is adjusted to obtain a circumscribed rectangle;

[0184] The area corresponding to the circumscribed rectangle is determined as the first area.

[0185] like Fig. 9 As shown, the embodiment of the present application provides a face image recognition device, including a processor 901, a communication interface 902, a memory 903 and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0186] Memory 903, used for storing computer programs;

[0187] In one embodiment of the present application, the processor 901 is used to execute the program stored in the memory 903 to implement the face image recognition method provided by any of the above method embodiments, including:

[0188] Obtaining a face image to be recognized and key points on the face image;

[0189] Determine a first sub-image and a second sub-image in the face image according to the key points on the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including a mouth and a nose;

[0190] Calling a pre-trained image evaluation model, and using the first sub-image and the second sub-image to obtain an expression evaluation result;

[0191] When the expression evaluation result is the first evaluation result, a poster is generated using the face image.

[0192] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps performed by the face image recognition method provided in any of the aforementioned method embodiments are implemented.

[0193] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0195] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0196] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A facial image recognition method, characterized in that: The method comprises: Obtaining a face image to be recognized and key points on the face image; Determine a first sub-image and a second sub-image in the face image according to the key points on the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including a mouth and a nose; Calling a pre-trained image evaluation model, and using the first sub-image and the second sub-image to obtain an expression evaluation result; When the expression evaluation result is the first evaluation result, a poster is generated using the face image.

2. The method according to claim 1, characterized in that: The step of determining a first sub-image in the face image according to key points on the face image includes: Determine a first key point corresponding to an eye among the key points on the face image; Determine a first area in the face image according to the first key point; Dividing the facial image according to the first area to determine a third sub-image; A first sub-image is obtained according to the third sub-image.

3. The method according to claim 2, characterized in that: The step of determining the first sub-image according to the third sub-image includes: Acquire the aspect ratio of the first sub-image; When the aspect ratio satisfies a first preset condition, determining the aspect ratio as the number of repetitions of the first sub-image, and controlling the third sub-image to repeat in the vertical direction according to the number of repetitions to obtain the first sub-image; When the aspect ratio meets the second preset condition, the reciprocal of the aspect ratio is determined as the number of repetitions of the first sub-image, and according to the number of repetitions, the third sub-image is controlled to be repeated in the horizontal direction to obtain the first sub-image.

4. The method according to claim 1, characterized in that: The pre-trained image evaluation model includes a first evaluation module, a second evaluation module and a comprehensive module, and the calling of the pre-trained image evaluation model and obtaining an expression evaluation result using the first sub-image and the second sub-image include: Inputting the first sub-image into the first evaluation module and outputting a first evaluation score; Inputting the second sub-image into the second evaluation module, and outputting a second evaluation score; The comprehensive module is used to obtain an expression evaluation result using the first evaluation score and the second evaluation score.

5. The method according to claim 4, characterized in that: The step of using the comprehensive module to obtain an expression evaluation result using the first evaluation score and the second evaluation score includes: determining an average evaluation score of the first evaluation score and the second evaluation score; When the first evaluation score is greater than or equal to a first preset score, the second evaluation score is greater than or equal to a second preset score, and the average evaluation score is greater than or equal to a third preset score, determining the first evaluation result as the expression evaluation result; When the first evaluation score is less than a first preset score, the second evaluation score is less than a second preset score, or the average evaluation score is less than a third preset score, the second evaluation result is determined as the expression evaluation result.

6. The method according to claim 4, characterized in that: The pre-trained image evaluation model further includes a third processing module, and the method further includes: When the first evaluation score meets the third preset condition and the second evaluation score meets the fourth preset condition, the face image is input into the third processing module to obtain an expression evaluation result.

7. The method according to claim 3, characterized in that: Determining a first area in the face image according to the first key point includes: Determine a minimum circumscribed rectangle in the face image based on the first key point; According to a preset first expansion ratio, the height of the minimum circumscribed rectangle is adjusted, and according to a preset second expansion ratio, the width of the minimum circumscribed rectangle is adjusted to obtain a circumscribed rectangle; The area corresponding to the circumscribed rectangle is determined as the first area.

8. A facial image recognition device, characterized in that: The device comprises: An acquisition unit, used to acquire a face image to be recognized and key points on the face image; a determining unit, configured to determine, in the face image, a first sub-image and a second sub-image according to key points on the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including a mouth and a nose; A calling unit, configured to call a pre-trained image evaluation model and obtain an expression evaluation result using the first sub-image and the second sub-image; A utilizing unit is used to generate a poster using the face image when the expression evaluation result is a first evaluation result.

9. A facial image recognition device, characterized in that: include: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to: Obtaining a face image to be recognized and key points on the face image; Determine a first sub-image and a second sub-image in the face image according to the key points on the face image, wherein the first sub-image is an image including eyes, and the second sub-image is an image including a mouth and a nose; Calling a pre-trained image evaluation model, and using the first sub-image and the second sub-image to obtain an expression evaluation result; When the expression evaluation result is the first evaluation result, a poster is generated using the face image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the facial image recognition method according to any one of claims 1 to 7 is implemented.

Images