Method, device, equipment and medium for detecting drooping of mouth corners

Abstract

The present invention discloses a method for detecting drooping corners of the mouth, which comprises: firstly performing a rough local threshold segmentation on a captured image of the corners of the mouth to obtain a threshold segmentation image, wherein the threshold segmentation image contains a plurality of closed areas, and after the local threshold segmentation, an area in the image with a more obvious grayscale change can be obtained, and interference from non-texture parts such as the background and cheeks can be removed. Then, based on a preset area condition and a preset angle condition, the interference area is removed to obtain a target closed area including only drooping corners of the mouth lines, so that interference from cheek contours and neck contours can be removed. Finally, the degree of drooping corners of the mouth is determined according to the length of the long side and the short side of the minimum circumscribed rectangle. It can be seen that this scheme can avoid being affected by irrelevant interferences such as the background, non-texture parts, cheek contours and neck contours, and can accurately determine the degree of drooping corners of the mouth. In addition, a drooping corners of the mouth detection device, equipment and medium are also proposed.

Description

Technical Field

[0001] The present invention relates to the technical field of face detection, and in particular to a method, device, equipment and medium for detecting drooping of mouth corners. Background Art

[0002] In contemporary society, people are paying more and more attention to their appearance, and everyone wants to stay as young as possible. The aging of the human body is usually reflected on the face, such as wrinkles, sagging cheeks, nasolabial folds, eye bags, drooping corners of the mouth, etc., which are all indicative of aging phenomena. They usually appear together. Knowing one or more of them can indirectly reflect the aging of the face.

[0003] Take the corners of the mouth as an example. The degree of drooping of the corners of the mouth varies for people in different aging states. The more severe the drooping of the corners of the mouth, the more severe the aging degree. Therefore, how to accurately obtain the degree of drooping of the corners of the mouth is particularly important in judging the degree of aging. Summary of the invention

[0004] Based on this, it is necessary to provide detection methods, devices, equipment and media for accurately judging the degree of drooping of the corners of the mouth to address the above problems.

[0005] A method for detecting drooping of mouth corners, the method comprising:

[0006] Acquire a face image to be detected, and capture a mouth corner area image in the face image;

[0007] Performing local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image includes a plurality of closed areas, and the pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than a preset degree;

[0008] Removing interference areas that do not meet a preset area condition and / or a preset angle condition from the plurality of closed areas to obtain a sagging texture image;

[0009] The degree of drooping of the corners of the mouth is determined according to the length of the long side and the length of the short side of the minimum circumscribed rectangle of the target closed area, and the target closed area is a closed area in the drooping lines image.

[0010] In one embodiment, performing local threshold segmentation on the mouth corner area image to obtain the threshold segmented image includes:

[0011] In the mouth corner area image, the grayscale mean and the standard deviation of the grayscale value of each pixel in a preset neighborhood are calculated, and the grayscale threshold at each pixel is calculated according to the grayscale mean and the standard deviation;

[0012] The pixel values ​​at the pixel points whose grayscale values ​​are greater than the grayscale threshold are set as the first pixel values, and the pixel values ​​at the pixel points whose grayscale values ​​are less than the grayscale threshold are set as the second pixel values, so as to obtain the threshold segmented image.

[0013] In one embodiment, removing interference areas that do not meet a preset area condition and / or a preset angle condition from among the plurality of closed areas includes:

[0014] Calculate the first angle between the long side of the minimum circumscribed rectangle of each closed area and the positive direction of the horizontal X-axis, where the horizontal X-axis is the axial direction for measuring the width of the face in the face image;

[0015] The closed areas whose areas are smaller than the first preset area and / or the closed areas whose first angles are smaller than the first preset angle among the plurality of closed areas are all regarded as the interference areas, and the interference areas among the plurality of closed areas are removed.

[0016] In one embodiment, removing interference areas that do not meet a preset area condition and / or a preset angle condition from among the plurality of closed areas includes:

[0017] Calculate the rectangular area of ​​the minimum enclosing rectangle of each closed area;

[0018] Calculating a second angle between the major axis of the minimum circumscribed ellipse of each closed area and the positive direction of the horizontal X-axis, where the horizontal X-axis is the axial direction for measuring the width of the face in the face image;

[0019] The closed areas whose rectangular areas are smaller than the second preset area and / or the closed areas whose second angles are smaller than the second preset angle among the multiple closed areas are taken as the interference areas, and the interference areas among the multiple closed areas are removed.

[0020] In one embodiment, determining the degree of drooping of the mouth corners according to the length of the long side and the short side of the minimum circumscribed rectangle of the target closed area includes:

[0021] Calculate the sum of the long side lengths of the minimum bounding rectangles of all target closed areas, and calculate the average short side lengths of the minimum bounding rectangles of all target closed areas;

[0022] Obtain a preset formula, substitute the sum of the long side lengths and the average value of the short side lengths into the preset formula, and obtain a result value, which represents the degree of drooping of the corners of the mouth.

[0023] In one of the embodiments, before performing local threshold segmentation on the mouth corner area image, the method further includes:

[0024] Noise detection is performed on the mouth corner area image, and when Gaussian noise is detected in the mouth corner area image, mean filtering is performed on the mouth corner area image to obtain a filtered mouth corner area image.

[0025] In one of the embodiments, before performing local threshold segmentation on the mouth corner area image, the method further includes:

[0026] Noise detection is performed on the mouth corner area image, and when salt and pepper noise is detected in the mouth corner area image, median filtering is performed on the mouth corner area image to obtain a filtered mouth corner area image.

[0027] A mouth corner droop detection device, the device comprising:

[0028] A mouth corner area image acquisition module is used to acquire a face image to be detected and capture a mouth corner area image in the face image;

[0029] A threshold segmentation image acquisition module is used to perform local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image includes a plurality of closed areas, and the pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than a preset degree;

[0030] A drooping pattern image acquisition module, used to remove interference areas that do not meet a preset area condition and / or a preset angle condition in the plurality of closed areas, to obtain a drooping pattern image;

[0031] The module for determining the degree of drooping of the corners of the mouth is used to determine the degree of drooping of the corners of the mouth according to the length of the long side and the length of the short side of the minimum circumscribed rectangle of the target closed area, and the target closed area is the closed area in the drooping line image.

[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0033] Acquire a face image to be detected, and capture a mouth corner area image in the face image;

[0034] Performing local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image includes a plurality of closed areas, and the pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than a preset degree;

[0035] Removing interference areas that do not meet a preset area condition and / or a preset angle condition from the plurality of closed areas to obtain a sagging texture image;

[0036] The degree of drooping of the corners of the mouth is determined according to the length of the long side and the length of the short side of the minimum circumscribed rectangle of the target closed area, and the target closed area is a closed area in the drooping lines image.

[0037] A mouth corner droop detection device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0038] Acquire a face image to be detected, and capture a mouth corner area image in the face image;

[0039] Performing local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image includes a plurality of closed areas, and the pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than a preset degree;

[0040] Removing interference areas that do not meet a preset area condition and / or a preset angle condition from the plurality of closed areas to obtain a sagging texture image;

[0041] The degree of drooping of the corners of the mouth is determined according to the length of the long side and the length of the short side of the minimum circumscribed rectangle of the target closed area, and the target closed area is a closed area in the drooping lines image.

[0042] The present invention provides a method, device, equipment and medium for detecting drooping corners of the mouth. First, a rough local threshold segmentation is performed on the captured image of the corner of the mouth area to obtain a threshold segmentation image. The threshold segmentation image contains several closed areas. After the local threshold segmentation, the area with more obvious grayscale changes in the image can be obtained, and the interference of non-texture parts such as the background and cheeks can be removed. Then, based on the preset area condition and the preset angle condition, the interference area is removed to obtain a target closed area that only includes the drooping corner of the mouth lines, so that the interference of the cheek contour and the neck contour can be removed. Finally, the degree of drooping corner of the mouth is determined according to the length of the long side and the short side of the minimum circumscribed rectangle. It can be seen that this scheme can avoid the influence of irrelevant interference such as the background, non-texture parts, cheek contours and neck contours, and can accurately determine the degree of drooping corner of the mouth. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] in:

[0045] Figure 1A schematic diagram of a flow chart of a method for detecting drooping of mouth corners in one embodiment;

[0046] Figure 2 is a schematic diagram of marking key points in a face image in one embodiment;

[0047] Figure 3 is a schematic diagram of a mouth corner area image in one embodiment;

[0048] Figure 4 is a schematic diagram of a threshold value segmentation image in one embodiment;

[0049] Figure 5 A schematic diagram of calculating a first angle in one embodiment;

[0050] Figure 6 is a schematic diagram of calculating the second angle in one embodiment;

[0051] Figure 7 is a schematic diagram of a sagging texture image in one embodiment;

[0052] Figure 8 is a schematic structural diagram of a mouth corner droop detection device in one embodiment;

[0053] Fig. 9 1 is a structural block diagram of a mouth corner droop detection device in one embodiment. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] like Figure 1 As shown, Figure 1 : is a flow chart of a method for detecting drooping of the mouth corners in one embodiment. The method for detecting drooping of the mouth corners in this embodiment includes the following steps:

[0056] Step 102, obtaining a face image to be detected, and capturing an image of the mouth corner area in the face image.

[0057] Among them, the mouth corner area image is an image that only covers the mouth corner area in the face image. Because the features of the face background and other areas, such as the eye area and the nose area, are not closely related to the drooping mouth corners, in order to avoid interference, the mouth corner area is cut out separately.

[0058] For details, see Figure 2, use the Dlib library's 127-point landmark model to extract 68 key points in the face image. These key points are used to describe the positions of the facial features of the human body. Then determine some of these key points as target key points, and obtain the coordinate values ​​of the target key points. For example, obtain the x-value and y-value of the coordinate number 90, the y-value of the coordinate number 18, and the x-value of the coordinate number 23. Then intercept the face image based on these coordinate values, and you can get the following: Figure 3 Of course, the above is a capture of the right half of the mouth corner area image, and the preset numbers can also be expanded to capture the left half of the mouth corner area image, for example, obtaining the coordinate x value and coordinate y value of number 84, the coordinate y value of number 14, and the coordinate x value of number 9 for capture.

[0059] Step 104, performing local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image.

[0060] The local threshold segmentation is to set a threshold for each pixel or local area and segment the pixel or local area based on the threshold. The threshold segmentation image includes several closed areas with obvious grayscale changes in the mouth corner area image. The pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than the preset degree. After the local threshold segmentation, the interference of non-texture parts such as the background and cheeks can be removed.

[0061] Further, in order to avoid the noise and skin noise generated in the acquisition and transmission process of the picture from interfering with the detection of the drooping of the corners of the mouth, denoising should be performed before performing local threshold segmentation. In a specific embodiment, firstly, based on the opencv library of python or matlab, the image of the corner of the mouth is subjected to noise detection, and the corresponding denoising method is selected for denoising according to the characteristics of the noise. According to the different characteristics of the noise, different denoising methods are selected for denoising, which can achieve more accurate denoising, thereby helping to achieve more accurate detection of the drooping of the corner of the mouth. For example, when it is detected that the probability density function of the noise in the image of the corner of the mouth obeys the Gaussian distribution, it is determined that the noise is Gaussian noise, and mean filtering is performed at this time, that is, for the current pixel point (x, y) to be processed, a neighboring area is selected, and the neighboring area is composed of several pixels of its neighbors (generally 3*3), and the mean of all pixels in the neighboring area is calculated, and then the mean is assigned to the current pixel point (x, y) as the grayscale value of the processed image at this point. After all pixels are subjected to such processing, the image of the corner of the mouth after mean filtering is obtained. After mean filtering, the impact on image quality caused by the image sensor not being bright enough, the brightness not being uniform enough, and the mutual influence of circuit components can be reduced.

[0062] For example, when randomly appearing white or black dots are detected in the image of the corner of the mouth, it is determined that the noise is salt and pepper noise. At this time, median filtering is performed, that is, for the current pixel point (x, y) to be processed, a neighboring area is selected, which is composed of N pixels of its neighbors (usually 3*3), and the pixels in the neighboring area are sorted according to the size of the pixel value to generate a monotonically rising (or falling) two-dimensional data sequence, and the median value is taken as the grayscale value assigned to the current pixel point (x, y) as the processed image at this point. After all pixels are processed in this way, the image of the corner of the mouth after median filtering is obtained. After median filtering, the black and white light and dark point noise generated by the image sensor, transmission channel, decoding processing and other links in the image can be eliminated.

[0063] In a feasible implementation, the mouth corner area image can be segmented by local threshold in the following manner: in the mouth corner area image, the grayscale mean and the standard deviation of the grayscale value of each pixel in a preset neighborhood centered on the pixel are calculated as follows:

[0064]

[0065]

[0066] Among them, m(x, y) is the grayscale mean of the current pixel in the preset neighborhood, (x, y) is the coordinate, the size of the preset neighborhood is r*r, g(i, j) represents the grayscale value at (i, j) in the neighborhood, and s(x, y) is the standard deviation.

[0067] Then calculate the grayscale threshold at each pixel based on the grayscale mean and standard deviation, as follows:

[0068]

[0069] Wherein, R is the dynamic range of the standard deviation. If the current input image is an 8-bit grayscale image, then R=128; k is a custom correction parameter, 0<k<1.

[0070] Finally, each pixel is compared with its respective grayscale threshold T(x, y), and the pixel value at the pixel point with a grayscale value greater than the grayscale threshold T(x, y) is set as the first pixel value, and the pixel value at the pixel point with a grayscale value less than the grayscale threshold is set as the second pixel value, for example, the first pixel value is 0 and the second pixel value is 1; or the first pixel value is 1 and the second pixel value is 0. Finally, we get Figure 4The threshold segmented image shown (numbers 1-8 represent closed areas). The grayscale threshold obtained by the above local threshold segmentation is not fixed, but is determined by the grayscale mean and standard deviation of the grayscale values ​​of the surrounding neighborhood pixels. After calculation, the grayscale threshold in the image area with higher brightness will usually be higher, while the grayscale threshold in the image area with lower brightness will be correspondingly smaller, thus eliminating the influence of brightness on threshold segmentation. In addition, a coefficient k is also set in this scheme to perform detail correction in some areas.

[0071] In addition, in another embodiment, a local threshold segmentation scheme can also be used: first artificially set two preset values ​​S=15, TT=128. Then calculate the maximum value M and the minimum value N of all pixels in the k×k window centered on any pixel P in the image, and calculate the average value of M and N as T. If MN is greater than S, it means that the grayscale difference in the area where the window is located is large, and the grayscale threshold of the current pixel P is set to T; if MN is less than S, it means that the grayscale difference in the area where the window is located is small, then the window is in a closed area or in the background area, and then judge the relationship between T and TT. If T>TT, the grayscale value of pixel P is the first pixel value, otherwise the grayscale value of the current point is the second pixel value.

[0072] Step 106 , removing interference areas that do not meet a preset area condition and / or a preset angle condition from among the plurality of closed areas, to obtain a sagging texture image.

[0073] The preset area condition is used to remove interference areas such as moles and wrinkles, and the preset angle condition is used to remove interference areas such as cheek contours and neck contours. In this way, the drooping lines image can only include the target closed area of ​​the drooping lines at the corners of the mouth.

[0074] In a specific embodiment, each closed area in the threshold segmentation image is traversed, and the area S1 of the closed area is first determined by calculating the number of pixels in the closed area, and then the closed area with an area smaller than the first preset area S2 is regarded as the interference area, so that the closed area can be removed. Figure 4 The closed areas with too small area are as follows, such as the closed areas with numbers 3, 4, 6, and 7. In addition, the minimum circumscribed rectangle of each closed area is drawn, such as Figure 5 As shown, the first angle α1 between the long side of the minimum circumscribed rectangle and the positive direction of the horizontal line X axis is calculated. The horizontal line X axis is the axial direction for measuring the width of the face in the face image. The first angle α1 is less than the first preset angle α2 as the interference area. For example, α2 is set to 90°. This can remove Figure 4The area in which the direction of the drooping lines of the actual corners of the mouth are inconsistent, such as the closed areas numbered 5 and 8. Of course, if the image of the left half of the corner of the mouth is intercepted in step 102, the preset angle condition here can be set accordingly to use the first angle α1 greater than the first preset angle α2 as the interference area. In this specific embodiment, the screening is directly based on the area of ​​the closed area, and the closed area with a small area can be accurately screened out. And the angle between the closed area and the positive direction of the X-axis is determined based on the minimum circumscribed rectangle, which can overall reflect the angle of the closed area, and the screening of the preset angle conditions can be completed relatively quickly.

[0075] In another specific embodiment, each closed area in the figure is traversed, and the minimum circumscribed rectangle of each closed area is first made, and the rectangular area S3 of the minimum circumscribed rectangle is determined by calculating the number of pixels in the minimum circumscribed rectangle, and the rectangular area S3 smaller than the second preset area S4 is regarded as the interference area. In addition, the minimum circumscribed ellipse of each closed area is made, such as Figure 6 As shown, the second angle α3 between the major axis of the minimum circumscribed ellipse and the positive direction of the horizontal X-axis is calculated, and the second angle α3 less than the second preset angle α4 is used as an interference area. Similarly, if the image of the left half of the mouth corner area is intercepted in step 102, the preset angle condition here can be set accordingly to use the second angle α3 greater than the second preset angle α4 as an interference area. In this specific embodiment, the screening is based on the area of ​​the minimum circumscribed rectangle. Since the minimum circumscribed rectangle is a regular shape, it can quickly screen out closed areas with too small an area. And the angle between the closed area and the positive direction of the X-axis is determined based on the minimum circumscribed ellipse, which can also reflect the angle of the closed area as a whole, and the screening of the preset angle conditions can be completed relatively quickly.

[0076] Finally, after removing the interference areas 3, 4, 5, 6, 7, and 8, we get Figure 7 Image of the drooping pattern shown.

[0077] Step 108, determining the degree of drooping of the corners of the mouth according to the length of the long side and the length of the short side of the minimum circumscribed rectangle of the target closed area.

[0078] In a specific embodiment, the calculation process of the degree of drooping of the mouth corners is as follows: first, the sum of the lengths of the long sides of the minimum circumscribed rectangles of all target closed areas is calculated, and then the average length of the short sides of the minimum circumscribed rectangles of all target closed areas, m2, is calculated. Finally, m1 and m2 are substituted into the following formula for calculation:

[0079] L=a*m1+b*m2

[0080] Among them, a and b can be flexibly assigned according to the specific situation, and L represents the degree of drooping of the corners of the mouth, which is linked to the degree of facial aging. The calculation process of this method is simple, and it can be flexibly adjusted for different users based on the assigned values ​​of a and b. The larger L is, the greater the degree of drooping of the corners of the mouth, which reflects the more serious the degree of facial aging.

[0081] Or, since the sum of the long side lengths m1 roughly reflects the drooping length of the corners of the mouth, and the average length m2 roughly reflects the width of the corners of the mouth. Generally speaking, the longer m1 is, the more obvious the aging is. Therefore, m1 is more closely linked to the degree of drooping of the corners of the mouth, and m1 and m2 can also be substituted into the following formula for calculation:

[0082] L=m1+ln(m2)

[0083] In the above formula, m1 has a relatively greater impact on L, while m2 has a relatively smaller impact on L.

[0084] Furthermore, the right half of the mouth corner area and the left half of the mouth corner area can be combined to detect the degree of drooping of the mouth corner. For example, after combining the above embodiments, the drooping degree L1 of the left half of the mouth corner and the drooping degree L2 of the right half of the mouth corner are first obtained, and then the drooping degree L0 of the mouth corner reflecting the whole face can be calculated based on L1 and L2:

[0085]

[0086] or,

[0087] L0=K1L1+K2L2

[0088] The K1 and K2 can be set by the user. The calculated L0 is more representative than L1 or L2.

[0089] The above-mentioned method for detecting droopy mouth corners first performs a rough local threshold segmentation on the captured image of the mouth corner area to obtain a threshold segmentation image, which contains several closed areas. After local threshold segmentation, the area with more obvious grayscale changes in the image can be obtained, and the interference of non-texture parts such as the background and cheeks can be removed. Then, based on the preset area condition and the preset angle condition, the interference area is removed to obtain a target closed area that only includes the droopy mouth corner lines, so that the interference of the cheek contour and the neck contour can be removed. Finally, the degree of droopy mouth corners is determined according to the length of the long side and the short side of the minimum circumscribed rectangle. It can be seen that this scheme can avoid the influence of irrelevant interference such as the background, non-texture parts, cheek contours and neck contours, and can accurately determine the degree of droopy mouth corners.

[0090] In one embodiment, Figure 8 As shown, a mouth corner droop detection device is proposed, the device comprising:

[0091] The mouth corner area image acquisition module 802 is used to acquire the face image to be detected and capture the mouth corner area image in the face image;

[0092] The threshold segmentation image acquisition module 804 is used to perform local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image includes a plurality of closed areas, and the pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than a preset degree;

[0093] A drooping pattern image acquisition module 806 is used to remove interference areas that do not meet a preset area condition and / or a preset angle condition among a plurality of closed areas to obtain a drooping pattern image;

[0094] The mouth corner droop degree determination module 808 is used to determine the mouth corner droop degree according to the long side length and the short side length of the minimum circumscribed rectangle of the target closed area, and the target closed area is the closed area in the drooping line image.

[0095] The above-mentioned mouth corner droop detection device first performs a rough local threshold segmentation on the intercepted mouth corner area image to obtain a threshold segmentation image, which contains several closed areas. After local threshold segmentation, the area with more obvious grayscale changes in the image can be obtained, and the interference of non-texture parts such as the background and cheeks can be removed. Then, based on the preset area condition and the preset angle condition, the interference area is removed to obtain a target closed area that only includes the drooping lines of the mouth corners, so that the interference of the cheek contour and the neck contour can be removed. Finally, the degree of drooping of the mouth corners is determined according to the length of the long side and the short side of the minimum circumscribed rectangle. It can be seen that this scheme can avoid the influence of irrelevant interference such as the background, non-texture parts, cheek contours and neck contours, and can accurately determine the degree of drooping of the mouth corners.

[0096] In one embodiment, the threshold segmentation image acquisition module 804 is specifically used to: calculate the grayscale mean and standard deviation of the grayscale value of each pixel point in a preset neighborhood in the mouth corner area image, and calculate the grayscale threshold at each pixel point based on the grayscale mean and standard deviation; set the pixel value at the pixel point whose grayscale value is greater than the grayscale threshold to the first pixel value, and set the pixel value at the pixel point whose grayscale value is less than the grayscale threshold to the second pixel value, to obtain the threshold segmentation image.

[0097] In one embodiment, the drooping lines image acquisition module 806 is specifically used to: calculate the first angle between the long side of the minimum circumscribed rectangle of each closed area and the positive direction of the horizontal X-axis, the horizontal X-axis being the axial direction for measuring the width of the face in the face image; treating a number of closed areas whose area is smaller than a first preset area, and / or a closed area whose first angle is smaller than a first preset angle as interference areas, and removing the interference areas from the number of closed areas.

[0098] In one embodiment, the drooping lines image acquisition module 806 is specifically used to: calculate the rectangular area of ​​the minimum circumscribed rectangle of each closed area; calculate the second angle between the major axis of the minimum circumscribed ellipse of each closed area and the positive direction of the horizontal X-axis, the horizontal X-axis being the axial direction for measuring the width of the face in the face image; take closed areas whose rectangular areas are smaller than the second preset area and / or closed areas whose second angles are smaller than the second preset angle among several closed areas as interference areas, and remove the interference areas among several closed areas.

[0099] In one embodiment, the module 808 for determining the degree of drooping of the corners of the mouth is specifically used to: calculate the sum of the long side lengths of the minimum circumscribed rectangles of all target closed areas, and calculate the average short side lengths of the minimum circumscribed rectangles of all target closed areas; obtain a preset formula, substitute the sum of the long side lengths and the average short side lengths into the preset formula to obtain a result value, which represents the degree of drooping of the corners of the mouth.

[0100] In one embodiment, the mouth corner droop detection device also includes: a filtering module, which is used to perform noise detection on the mouth corner area image. When Gaussian noise is detected in the mouth corner area image, the mouth corner area image is mean filtered to obtain a filtered mouth corner area image.

[0101] In one embodiment, the filtering module is further used to perform noise detection on the mouth corner area image. When salt and pepper noise is detected in the mouth corner area image, median filtering is performed on the mouth corner area image to obtain a filtered mouth corner area image.

[0102] Fig. 9 FIG. 2 shows an internal structure diagram of a mouth corner droop detection device in one embodiment. Fig. 9 As shown, the mouth corner droop detection device includes a processor, a memory and a network interface connected via a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the mouth corner droop detection device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement a method for detecting mouth corner droop. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute a method for detecting mouth corner droop. Those skilled in the art can understand that Fig. 9 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the mouth corner droop detection device to which the scheme of the present application is applied. The specific mouth corner droop detection device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0103] A mouth corner droop detection device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: acquiring a face image to be detected, and intercepting a mouth corner area image in the face image; performing local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image comprises a plurality of closed areas, and the pixel difference between pixels in the closed area and pixels outside the closed area is greater than a preset degree; removing interference areas in the plurality of closed areas that do not meet preset area conditions and / or preset angle conditions to obtain a drooping pattern image; and determining the degree of mouth corner droop according to the long side length and the short side length of the minimum circumscribed rectangle of the target closed area, wherein the target closed area is a closed area in the drooping pattern image.

[0104] In one embodiment, local threshold segmentation is performed on the mouth corner area image to obtain a threshold segmented image, including: in the mouth corner area image, calculating the grayscale mean and the standard deviation of the grayscale value of each pixel point in a preset neighborhood, and calculating the grayscale threshold at each pixel point according to the grayscale mean and the standard deviation; setting the pixel value at the pixel point whose grayscale value is greater than the grayscale threshold to the first pixel value, and setting the pixel value at the pixel point whose grayscale value is less than the grayscale threshold to the second pixel value, to obtain a threshold segmented image.

[0105] In one embodiment, interference areas that do not meet preset area conditions and / or preset angle conditions are removed from a number of closed areas, including: calculating a first angle between the long side of the minimum circumscribed rectangle of each closed area and the positive direction of the horizontal X-axis, where the horizontal X-axis is the axial direction for measuring the width of the face in the face image; taking closed areas whose areas are smaller than the first preset area and / or closed areas whose first angles are smaller than the first preset angle from a number of closed areas as interference areas, and removing interference areas from the number of closed areas.

[0106] In one embodiment, removing interference areas that do not meet preset area conditions and / or preset angle conditions from a number of closed areas includes: calculating the rectangular area of ​​the minimum circumscribed rectangle of each closed area; calculating the second angle between the major axis of the minimum circumscribed ellipse of each closed area and the positive direction of the horizontal X-axis, where the horizontal X-axis is the axial direction for measuring the width of the face in the face image; taking closed areas whose rectangular areas are smaller than the second preset area and / or closed areas whose second angles are smaller than the second preset angle from a number of closed areas as interference areas, and removing interference areas from the number of closed areas.

[0107] In one embodiment, the degree of drooping of the corners of the mouth is determined based on the long side length and the short side length of the minimum circumscribed rectangle of the target closed area, including: calculating the sum of the long side lengths of the minimum circumscribed rectangles of all target closed areas, and calculating the average short side lengths of the minimum circumscribed rectangles of all target closed areas; obtaining a preset formula, substituting the sum of the long side lengths and the average short side lengths into the preset formula to obtain a result value, which characterizes the degree of drooping of the corners of the mouth.

[0108] In one embodiment, before performing local threshold segmentation on the mouth corner area image, it also includes: performing noise detection on the mouth corner area image, and when Gaussian noise is detected in the mouth corner area image, performing mean filtering on the mouth corner area image to obtain a filtered mouth corner area image.

[0109] In one embodiment, before performing local threshold segmentation on the mouth corner area image, it also includes: performing noise detection on the mouth corner area image, and when salt and pepper noise is detected in the mouth corner area image, performing median filtering on the mouth corner area image to obtain a filtered mouth corner area image.

[0110] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0111] A face image to be detected is obtained, and a mouth corner area image in the face image is captured; local threshold segmentation is performed on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image includes a plurality of closed areas, and the pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than a preset degree; interference areas that do not meet a preset area condition and / or a preset angle condition in the plurality of closed areas are removed to obtain a drooping line image; the degree of drooping of the mouth corner is determined according to the long side length and the short side length of the minimum circumscribed rectangle of the target closed area, and the target closed area is a closed area in the drooping line image.

[0112] In one embodiment, local threshold segmentation is performed on the mouth corner area image to obtain a threshold segmented image, including: in the mouth corner area image, calculating the grayscale mean and the standard deviation of the grayscale value of each pixel point in a preset neighborhood, and calculating the grayscale threshold at each pixel point according to the grayscale mean and the standard deviation; setting the pixel value at the pixel point whose grayscale value is greater than the grayscale threshold to the first pixel value, and setting the pixel value at the pixel point whose grayscale value is less than the grayscale threshold to the second pixel value, to obtain a threshold segmented image.

[0113] In one embodiment, interference areas that do not meet preset area conditions and / or preset angle conditions are removed from a number of closed areas, including: calculating a first angle between the long side of the minimum circumscribed rectangle of each closed area and the positive direction of the horizontal X-axis, where the horizontal X-axis is the axial direction for measuring the width of the face in the face image; taking closed areas whose areas are smaller than the first preset area and / or closed areas whose first angles are smaller than the first preset angle from a number of closed areas as interference areas, and removing interference areas from the number of closed areas.

[0114] In one embodiment, removing interference areas that do not meet preset area conditions and / or preset angle conditions from a number of closed areas includes: calculating the rectangular area of ​​the minimum circumscribed rectangle of each closed area; calculating the second angle between the major axis of the minimum circumscribed ellipse of each closed area and the positive direction of the horizontal X-axis, where the horizontal X-axis is the axial direction for measuring the width of the face in the face image; taking closed areas whose rectangular areas are smaller than the second preset area and / or closed areas whose second angles are smaller than the second preset angle from a number of closed areas as interference areas, and removing interference areas from the number of closed areas.

[0115] In one embodiment, the degree of drooping of the corners of the mouth is determined based on the long side length and the short side length of the minimum circumscribed rectangle of the target closed area, including: calculating the sum of the long side lengths of the minimum circumscribed rectangles of all target closed areas, and calculating the average short side lengths of the minimum circumscribed rectangles of all target closed areas; obtaining a preset formula, substituting the sum of the long side lengths and the average short side lengths into the preset formula to obtain a result value, which characterizes the degree of drooping of the corners of the mouth.

[0116] In one embodiment, before performing local threshold segmentation on the mouth corner area image, it also includes: performing noise detection on the mouth corner area image, and when Gaussian noise is detected in the mouth corner area image, performing mean filtering on the mouth corner area image to obtain a filtered mouth corner area image.

[0117] In one embodiment, before performing local threshold segmentation on the mouth corner area image, it also includes: performing noise detection on the mouth corner area image, and when salt and pepper noise is detected in the mouth corner area image, performing median filtering on the mouth corner area image to obtain a filtered mouth corner area image.

[0118] It should be noted that the above-mentioned method, device, equipment and computer-readable storage medium for detecting drooping of mouth corners belong to a general inventive concept, and the contents in the embodiments of the method, device, equipment and computer-readable storage medium for detecting drooping of mouth corners are applicable to each other.

[0119] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting drooping of mouth corners, characterized in that: The detection method comprises: Acquire a face image to be detected, and capture a mouth corner area image in the face image; Performing local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image includes a plurality of closed areas, and the pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than a preset degree; removing interference areas that do not meet preset area conditions and preset angle conditions from the plurality of closed areas to obtain a sagging texture image; Determine the degree of drooping of the corners of the mouth according to the length of the long side and the length of the short side of the minimum circumscribed rectangle of the target closed area, wherein the target closed area is the closed area in the drooping lines image; The removing of interference areas that do not meet the preset area condition and the preset angle condition from the plurality of closed areas comprises: calculating a first angle between the long side of the minimum circumscribed rectangle of each closed area and the positive direction of the horizontal line X-axis, wherein the horizontal line X-axis is the axial direction for measuring the width of the face in the face image; taking the closed areas whose areas are smaller than the first preset area and the closed areas whose first angles are smaller than the first preset angle from the plurality of closed areas as the interference areas, and removing the interference areas from the plurality of closed areas; Or, calculating the rectangular area of ​​the minimum circumscribed rectangle of each closed area; calculating the second angle between the major axis of the minimum circumscribed ellipse of each closed area and the positive direction of the horizontal line X axis, wherein the horizontal line X axis is the axial direction for measuring the width of the face in the face image; taking the closed areas whose rectangular areas are smaller than the second preset area and the closed areas whose second angles are smaller than the second preset angle among the several closed areas as the interference areas, and removing the interference areas among the several closed areas; The method of determining the degree of drooping of the corners of the mouth according to the long side length and the short side length of the minimum circumscribed rectangle of the target closed area comprises: calculating the sum of the long side lengths of the minimum circumscribed rectangles of all target closed areas, and calculating the average short side lengths of the minimum circumscribed rectangles of all target closed areas; obtaining a preset formula, substituting the sum of the long side lengths and the average short side lengths into the preset formula to obtain a result value, which represents the degree of drooping of the corners of the mouth; The preset formula is: or: In the above formula, is the degree of drooping of the mouth corners, a and b are preset values, is the sum of the long side lengths of the minimum circumscribed rectangles of all target closed areas, It is the average length of the short side of the minimum bounding rectangle of all target closed areas.

2. The detection method according to claim 1, characterized in that: The performing local threshold segmentation on the mouth corner area image to obtain the threshold segmentation image includes: In the mouth corner area image, the grayscale mean and the standard deviation of the grayscale value of each pixel in a preset neighborhood are calculated, and the grayscale threshold at each pixel is calculated according to the grayscale mean and the standard deviation; The pixel values ​​at the pixel points whose grayscale values ​​are greater than the grayscale threshold are set as the first pixel values, and the pixel values ​​at the pixel points whose grayscale values ​​are less than the grayscale threshold are set as the second pixel values, so as to obtain the threshold segmented image.

3. The detection method according to claim 1, characterized in that: Before performing local threshold segmentation on the mouth corner area image, the method further includes: Noise detection is performed on the mouth corner area image, and when Gaussian noise is detected in the mouth corner area image, mean filtering is performed on the mouth corner area image to obtain a filtered mouth corner area image.

4. The detection method according to claim 1, characterized in that: Before performing local threshold segmentation on the mouth corner area image, the method further includes: Noise detection is performed on the mouth corner area image, and when salt and pepper noise is detected in the mouth corner area image, median filtering is performed on the mouth corner area image to obtain a filtered mouth corner area image.

5. A mouth corner droop detection device, characterized in that: The device comprises: A mouth corner area image acquisition module is used to acquire a face image to be detected and capture a mouth corner area image in the face image; A threshold segmentation image acquisition module is used to perform local threshold segmentation on the mouth corner area image to obtain a threshold segmentation image, wherein the threshold segmentation image includes a plurality of closed areas, and the pixel difference between the pixels in the closed area and the pixels outside the closed area is greater than a preset degree; A drooping pattern image acquisition module, used to remove interference areas that do not meet preset area conditions and preset angle conditions in the plurality of closed areas, to obtain a drooping pattern image; A drooping degree determination module for the corners of the mouth, used to determine the drooping degree of the corners of the mouth according to the length of the long side and the length of the short side of the minimum circumscribed rectangle of the target closed area, wherein the target closed area is a closed area in the drooping lines image; The drooping lines image acquisition module is specifically used to calculate the first angle between the long side of the minimum circumscribed rectangle of each closed area and the positive direction of the horizontal line X axis, where the horizontal line X axis is the axial direction for measuring the width of the face in the face image; the closed areas whose areas are smaller than the first preset area and the closed areas whose first angles are smaller than the first preset angle among the plurality of closed areas are taken as the interference areas, and the interference areas among the plurality of closed areas are removed; Or, calculating the rectangular area of ​​the minimum circumscribed rectangle of each closed area; calculating the second angle between the major axis of the minimum circumscribed ellipse of each closed area and the positive direction of the horizontal line X axis, wherein the horizontal line X axis is the axial direction for measuring the width of the face in the face image; taking the closed areas whose rectangular areas are smaller than the second preset area and the closed areas whose second angles are smaller than the second preset angle among the several closed areas as the interference areas, and removing the interference areas among the several closed areas; The module for determining the degree of drooping of the corners of the mouth is specifically used to calculate the sum of the lengths of the long sides of the minimum circumscribed rectangles of all target closed areas, and to calculate the average lengths of the short sides of the minimum circumscribed rectangles of all target closed areas; obtain a preset formula, substitute the sum of the long side lengths and the average short side lengths into the preset formula, and obtain a result value, which represents the degree of drooping of the corners of the mouth; The preset formula is: or: In the above formula, is the degree of drooping of the mouth corners, a and b are preset values, is the sum of the long side lengths of the minimum circumscribed rectangles of all target closed areas, It is the average length of the short side of the minimum bounding rectangle of all target closed areas.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 4.

7. A mouth corner droop detection device, comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 4.

Images