A face image segmentation method and system based on image processing

Abstract

This application relates to the field of image processing technology, and in particular to a face image segmentation method and system based on image processing. The method includes: constructing a high-frequency structure matching index reflecting the degree of matching between pixels and the detailed structure of a face image, wherein orientation uniformity and gradient level are positively correlated with the high-frequency structure matching index; calculating the volatility of the high-frequency structure matching index within an observation window and the overall matching level of the high-frequency structure matching index within the observation window; using the product of volatility and matching level as the face structure saliency; selecting a segmentation receptive field based on the face structure saliency of each pixel; and using a U-net segmentation network to segment the image based on the segmentation receptive field. This application improves image segmentation efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a face image segmentation method and system based on image processing. Background Technology

[0002] With the continuous development of artificial intelligence and computer vision technologies, facial image processing has been applied in fields such as security monitoring, biometrics, and smart healthcare. Facial image segmentation technology needs to accurately separate the facial subject from the background to provide a structured data foundation for subsequent tasks such as feature extraction and identity recognition. Traditional CNN structures perform regular sampling on image feature layers through fixed receptive fields, exhibiting good local perception capabilities. However, when processing image regions with geometric deformations, pose changes, or complex boundary structures, the perception position of fixed receptive fields cannot flexibly adapt to the non-rigid changes of the target, easily leading to problems such as inaccurate boundary recognition and loss of structural details. Therefore, based on the needs of social production, segmentation models based on deep learning convolutional neural networks (such as U-Net, DeepLab, etc.) are applied to production activities. These methods, through multi-level feature fusion and context-aware mechanisms, exhibit high robustness in overall face localization and large-area region segmentation, and can effectively cope with interference factors such as illumination changes and pose diversity.

[0003] In the U-net segmentation network, each segmentation layer corresponds to a receptive field scale. Different receptive field sizes can extract different types of features; for example, smaller receptive fields can extract details, while larger receptive fields can extract features from larger regions. To improve the accuracy of image feature extraction and segmentation, a large number of receptive fields of different sizes are usually set for image extraction. However, in the U-net segmentation network, the computational cost increases exponentially with the number of feature layers, resulting in low computational efficiency. Summary of the Invention

[0004] To improve the efficiency of U-net segmentation network in image segmentation, this application provides a face image segmentation method and system based on image processing.

[0005] Firstly, this application provides a face image segmentation method based on image processing, employing the following technical solution:

[0006] A face image segmentation method based on image processing is proposed. First, a preprocessed grayscale face image is obtained. For any pixel in the grayscale face image, an observation window is constructed. The directional uniformity of the gradient direction of each pixel in the observation window and the gradient level of each pixel in the observation window are obtained. A high-frequency structure matching index reflecting the degree of matching between the pixel and the detailed structure of the face image is constructed, where the directional uniformity and gradient level are positively correlated with the high-frequency structure matching index. The volatility of the high-frequency structure matching index in the observation window and the overall matching level of the high-frequency structure matching index in the observation window are calculated. The product of the volatility and the matching level is used as the face structure saliency. Based on the face structure saliency of each pixel, a segmentation receptive field is selected. Based on the feature layer corresponding to the segmentation receptive field, the U-net segmentation network is used to segment the image.

[0007] In this method, for each pixel, an observation window is constructed, and the gradient and gradient direction of each pixel within the observation window are analyzed. The high-frequency structure matching index, reflecting the presence of numerous facial details around the pixel, is used to determine the facial structure saliency. Subsequently, the volatility and matching level of the high-frequency structure index of each pixel within the observation window are analyzed to construct facial structure saliency, reflecting whether the pixel possesses significant facial features. Based on the facial structure saliency of each pixel, the receptive field is filtered; only a small number of feature layers (segmentation feature layers) corresponding to receptive fields with significant contributions are extracted for image processing. This reduces the number of feature layers used for image processing in the U-net network, thereby improving computational efficiency while preserving the image segmentation effect.

[0008] Optionally, the calculation steps for the directional uniformity include: in the observation window corresponding to any pixel, any two pixels form a comparison point pair; calculate the gradient direction difference index between the two pixels in the comparison point pair, and take the reciprocal of the sum of the gradient direction difference indices as the gradient direction uniformity.

[0009] By taking any two pixels in the observation window as a comparison pair, the difference in gradient directions between the two pixels in the comparison pair is used to reflect whether the gradient directions of the two pixels are consistent. This process is repeated for all comparison pairs, and the gradient direction difference exponents are accumulated to reflect the uniformity of gradient directions among all pixels in the observation window.

[0010] Optionally, the formula for calculating the gradient direction difference exponent between two pixels in a comparison point pair is: In the formula, This represents the gradient direction difference index between two pixels. It represents the absolute value of the difference between the gradient directions of two pixels; It is a sine function.

[0011] This method is primarily used to measure the difference in gradient directions between two pixels. However, while the 2° and 358° gradient directions show a small directional difference, their numerical differences are significant. To improve the accuracy of calculating the gradient direction difference index between two gradient directions, the sine function is stretched to reduce the occurrence of the aforementioned issue.

[0012] Optionally, the mean gradient of the pixels in the observation window can be used as the gradient level of that observation window.

[0013] The average gradient of each pixel in the observation window is calculated to reflect the overall gradient magnitude within the observation window, and this average value is used as the gradient level of the observation window.

[0014] Optionally, the product of gradient direction uniformity and gradient level can be used as the high-frequency structure matching index.

[0015] A high-frequency structure matching index is constructed by coordinating gradient direction uniformity and gradient level, thereby improving the accuracy of the high-frequency structure matching index calculation. Furthermore, multiplying the two factors ensures a positive correlation between gradient direction uniformity, gradient level, and the high-frequency structure matching index.

[0016] Optionally, the standard deviation of the high-frequency structure matching index of each pixel in the observation window can be used as the volatility of the high-frequency structure matching index in the observation window.

[0017] Standard deviation reflects the degree of variation in data; therefore, the standard deviation of the high-frequency structure matching index within the observation window is used as the volatility.

[0018] Optionally, the steps for obtaining the overall matching level of the high-frequency structure matching index in the observation window include: processing the high-frequency structure matching index of each pixel in the observation window using a logarithmic function to obtain the matching degree, and using the sum of the matching degrees of each pixel as the matching level.

[0019] Optionally, the step of selecting the segmentation receptive field based on the facial structure saliency of each pixel includes: adjusting the size of the receptive field using facial saliency within a preset receptive field size range to obtain the optimal receptive field for each pixel, wherein facial saliency is negatively correlated with the size of the optimal receptive field; and selecting the receptive field based on the number of pixels with the same optimal receptive field to determine the segmentation receptive field.

[0020] Optionally, the steps for filtering receptive fields based on the number of pixels with the same optimal receptive field and determining the segmented receptive field include: counting the number of pixels corresponding to each optimal receptive field size, sorting them according to the number to establish priority, accumulating the number of pixels according to priority, and selecting the current receptive field as the segmented receptive field when the accumulation ratio exceeds a set threshold.

[0021] Different receptive field sizes are suitable for feature extraction from different pixels. For some receptive fields, the contribution to feature extraction is relatively small, but they still consume significant computational resources. Therefore, we calculate the receptive field size for each pixel. For example, the optimal receptive field size for 90% of the data points in the image is... This means that the optimal receptive field can extract 90% of the features in the image, which is sufficient to ensure the segmentation quality of the image. Therefore, other receptive fields are no longer involved in the calculation, reducing the waste of computing resources and improving computing efficiency.

[0022] Secondly, this application provides a face image segmentation system based on image processing, which adopts the following technical solution:

[0023] A face image segmentation system based on image processing includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the face image segmentation method based on image processing described above is implemented.

[0024] The beneficial effect is that the above-mentioned face image segmentation method based on image processing generates a computer program, which is stored in the memory so that it can be loaded and executed by the processor. Thus, the system can be made based on the memory and the processor, making it convenient to use.

[0025] This application has the following technical effects:

[0026] By calculating the facial structure saliency of each pixel, the optimal receptive field corresponding to each pixel is determined. Priorities are set for each receptive field based on the number of pixels with the same receptive field. Some receptive fields are selected as segmentation receptive fields according to the priority, thereby reducing the amount of computation and improving computational efficiency, while ensuring the segmentation effect. Attached Figure Description

[0027] Figure 1 This is a flowchart of a face image segmentation method based on image processing according to this application.

[0028] Figure 2 This is a flowchart of step S4 of the face image segmentation method based on image processing in this application. Detailed Implementation

[0029] This application discloses a face image segmentation method based on image processing. For any pixel in the image, a high-frequency structure matching index is constructed to measure its degree of similarity to facial details. Then, facial structure saliency is constructed based on the high-frequency structure matching index. The receptive field for segmentation is selected based on the magnitude of the facial structure saliency at different pixels, reducing the number of feature layers with low contribution in the U-net network, thus reducing the computational load in the U-net network and improving segmentation efficiency.

[0030] Reference Figure 1 A face image segmentation method based on image processing includes steps S1-S4.

[0031] S1: Obtain the preprocessed grayscale image of the face.

[0032] The original face image is captured by a camera or other video recording equipment, then converted to grayscale, and finally denoised to obtain a grayscale face image.

[0033] In the process of face image segmentation, the key lies in extracting structural and texture features, rather than illumination changes or background interference. Therefore, to improve the efficiency of subsequent face image analysis, this application performs grayscale processing on the original face image in the RGB color space. Simultaneously, during the acquisition of face images, environmental interference and other factors may cause noise in the acquired face images. To avoid noise affecting the quality of the face image, this application uses a bilateral filtering denoising algorithm to denoise the face image, obtaining a grayscale face image.

[0034] The bilateral filtering algorithm is a conventional technique in this field and will not be elaborated here.

[0035] S2: For any pixel in a grayscale face image, construct an observation window, obtain the directional uniformity of the gradient direction of each pixel in the observation window and the gradient level of the pixel in the observation window, and construct a high-frequency structure matching index that reflects the degree of matching between the pixel and the detailed structure of the face image. The directional uniformity and gradient level are positively correlated with the high-frequency structure matching index.

[0036] In order to identify the detailed information of different parts of a face in a grayscale image, an observation window is constructed for each pixel. The gradient direction and gradient magnitude in the observation window are used to analyze whether the pixel conforms to the features of the face contour. A high-frequency structure matching index is constructed to facilitate the identification of the contour information of different parts of the face.

[0037] The steps for constructing the observation window include: using the pixels in the grayscale image of the face as the center, constructing a window with a side length of... A square region is defined, and this square region is used as the observation window for each pixel. The observation window includes multiple pixels surrounding the first pixel. The dimensions are calculated using the side length... for For example, the observation window contains 49 pixels.

[0038] After the observation window is constructed, the directional uniformity and gradient level are calculated based on the gradient direction and gradient features of the pixels in the observation window.

[0039] In this embodiment, the calculation steps of the direction uniformity include: in the observation window corresponding to any pixel, any two pixels form a comparison point pair; calculate the gradient direction difference index between the two pixels in the comparison point pair, and take the reciprocal of the sum of the gradient direction difference indices as the gradient direction uniformity.

[0040] To comprehensively analyze the differences in gradient directions among all pixels in the observation window, we combine any two pixels in the observation window to form comparison pairs, and then iterate through the differences in gradient directions among all comparison pairs to reflect the differences between each pixel in the observation window.

[0041] The formula for calculating the gradient direction difference exponent between two pixels in a comparison point pair is as follows: In the formula, This represents the gradient direction difference index between two pixels. It represents the absolute value of the difference between the gradient directions of two pixels; It is a sine function.

[0042] In the image, the gradient direction ranges from [0, 360]. In actual processing, the 2° direction and the 358° direction are quite close, but their numerical values ​​differ significantly. To reduce the occurrence of such cases, this method utilizes the following formula. This describes the actual difference in the gradient direction between two pixels.

[0043] After calculating the gradient direction difference index, the gradient direction difference indices of all corresponding comparison point pairs in the observation window are summed. The reciprocal of the sum of the gradient direction difference indices is taken as the gradient direction consistency. The gradient direction difference index reflects the difference in direction between two pixels. The larger the difference, the more inconsistent the gradient directions of the two pixels, and thus the smaller the uniformity of the gradient directions of the pixels in the observation window.

[0044] In this embodiment, the average gradient of each pixel in the observation window is used as the gradient level of the observation window.

[0045] There are no special requirements for the order of calculating gradient level and gradient uniformity; in other embodiments, gradient level can be calculated first.

[0046] After calculating the uniformity of gradient level and gradient direction, a high-frequency structure matching index is constructed based on these two factors. In an image, if the pixels within the observation window belong to the edge region of a facial structure (such as the eyelids or nose edges), the gradients of each pixel within the observation window should be large, and the gradient directions should be relatively uniform. Therefore, the calculated average gradient value is large, and the gradient directions of each pixel are relatively uniform, resulting in a large high-frequency structure matching index. If the pixels within the observation window do not belong to the edge region of a facial structure and have tiny pores or noise textures, the gradient direction changes significantly, resulting in a smaller high-frequency structure matching index. Therefore, in this embodiment, the product of the uniformity of gradient direction and the gradient level is used as the high-frequency structure matching index.

[0047] Specifically, the formula for calculating the high-frequency structure matching index can be expressed as:

[0048] In the formula, Represents pixels The uniformity of gradient direction in the corresponding observation window; This indicates the total number of pixels in the observation window; Represents pixels With pixels The gradient direction difference index between them; This represents the mean gradient of each pixel in the observation window, used to reflect the gradient level of each pixel in the observation window.

[0049] This part represents the uniformity of gradient direction of pixels in the observation window. The 1 in the denominator of this part is mainly used to prevent the denominator from being 0, which is a common technical means in this field. For gradient levels.

[0050] S3: Calculate the volatility of the high-frequency structure matching index within the observation window and the overall matching level of the high-frequency structure matching index within the observation window; use the product of volatility and matching level as the saliency of face structure.

[0051] In grayscale images of faces, facial structure also exhibits spatial abruptness. This means that a face is not composed of a single structure or texture, but rather is formed by the smooth or abrupt splicing of multiple regions with different functions. For example, there is a sharp transition region between the smooth cheek area and the clearly defined jawline, or a relatively complex transition region at the junction of the skin area and the lips with their fine textures. Specifically, pixels in low-complexity structural regions have a lower high-frequency structure matching index, while pixels in high-complexity structural regions have a higher high-frequency structure matching index. Furthermore, the change in the high-frequency structure matching index is more significant in the transition regions from low-complexity to high-complexity structural regions, exhibiting greater volatility.

[0052] Therefore, facial structure saliency is constructed based on the volatility of the high-frequency structure matching index within the observation window and the overall matching level.

[0053] In this embodiment, the standard deviation of the high-frequency structure matching index of each pixel in the observation window is used as the volatility.

[0054] The steps for obtaining the overall matching level of the high-frequency structure matching index in the observation window include: processing the high-frequency structure matching index of each pixel in the observation window using a logarithmic function to obtain the matching degree, and using the sum of the matching degrees of each pixel as the matching level.

[0055] The formula for calculating the saliency of facial structure can be expressed as:

[0056] In the formula, where Represents the pixels in a grayscale image of a human face. The facial structure significance index; This indicates the total number of pixels in the observation window; represents the logarithmic function with the natural constant as its base, used for nonlinear smoothing scaling of the high-frequency structure matching index to improve the robustness of the facial structure saliency index; 1 represents the hyperparameter used to ensure that the logarithmic result is non-negative. Represents pixels The first observation window High-frequency structure matching index of each pixel; It represents the standard deviation of the high-frequency structure matching index within the observation window.

[0057] This represents the result of the high-frequency structure matching index after applying a logarithmic function, i.e., the matching degree. The 1 in the formula is mainly used to prevent the calculation result in parentheses from being 0.

[0058] Within the observation window, if the high-frequency structure matching index of a pixel changes significantly (i.e., it fluctuates greatly), and there are pixels with a high high-frequency structure matching index, the pixels with a high structure matching index will raise the overall matching level of the pixels in the observation window, indicating a higher matching level. This suggests that the region corresponding to the observation window conforms to the characteristics of significant facial structure changes, i.e., the intensity of the facial structure is more significant; for example, the jawline. Conversely, if the high-frequency structure matching index of a pixel changes little, and there are no pixels with a high high-frequency structure matching index, it indicates that the region corresponding to the observation window does not conform to the characteristics of facial structure changes, i.e., the intensity of the facial structure is less significant, such as the cheek area. Therefore, the calculated facial structure significance index will be lower.

[0059] S4: The receptive field for segmentation is selected based on the saliency of the facial structure at each pixel, and the U-net segmentation network is used to segment the image based on the receptive field.

[0060] Reference Figure 2 Step S4 includes steps S41-S42.

[0061] S41: Adjust the size of the receptive field using face saliency within a preset receptive field size range to obtain the optimal receptive field, where face saliency is negatively correlated with the size of the optimal receptive field.

[0062] When using the U-net segmentation network to segment face images, a larger receptive field is needed for large, structurally smooth regions, such as the cheek area, to ensure smoothness and continuity of the segmentation results and avoid misinterpreting minute textures as boundaries, which would reduce segmentation accuracy. Conversely, for regions with significant structures and dense boundaries, such as the perilips and jawline, a smaller receptive field is required to achieve fine-grained local detail depiction and prevent boundary blurring due to an excessively large receptive field. Therefore, the size of the receptive field is inversely proportional to the salient features of the face.

[0063] Specifically, the formula for calculating the receptive field size can be expressed as:

[0064] ;in Represents the first grayscale image of a face. The receptive field size of a pixel This represents the maximum value of the receptive field size. This represents the minimum size of the receptive field; in this application The value is 9. The value is 3. This represents a function that takes odd integers, such as , . Represents the first grayscale image of a face. The facial structure saliency index of each pixel, This represents the maximum value of the facial structure saliency index of all pixels in a grayscale image of a face, and is used to normalize the facial structure saliency index.

[0065] S42: Filter the receptive fields based on the number of pixels with the same optimal receptive field to determine the segmented receptive field.

[0066] The number of pixels corresponding to each optimal receptive field size is counted, and priority is established by sorting them according to the number of pixels. The number of pixels is accumulated according to the priority. When the accumulation ratio exceeds a set threshold, the receptive field currently participating in the accumulation is selected as the segmentation receptive field.

[0067] For example, based on the above steps, the optimal receptive field corresponding to each pixel is obtained; the optimal receptive field corresponding to each pixel is acquired, and the number of pixels with the same optimal receptive field size is counted. For example, in the image, there are 20 pixels with an optimal receptive field size of... This means that the optimal receptive field corresponds to 20 pixels. The priority of the receptive field is determined by the number of pixels; the more pixels, the higher the priority of the receptive field. For example, if there are three receptive fields... , , The corresponding number of pixels are 30, 50, and 20 respectively; then The receptive field priority is higher than The feeling of the wild and The receptive field is determined. During subsequent accumulation, the number of pixels corresponding to higher-priority receptive fields is accumulated first.

[0068] The number of pixels corresponding to each optimal receptive field is accumulated according to priority. The ratio of the accumulated number of pixels to the total number of pixels in the grayscale face image is used as the accumulation ratio. Accumulation stops when the accumulation ratio exceeds a preset threshold. The size of the accumulated receptive field is used as the segmentation receptive field. Receptive fields of other sizes are no longer used in the U-net segmentation network for grayscale face image segmentation, thus improving the efficiency of the U-net segmentation network for grayscale face image segmentation.

[0069] In the process of segmenting grayscale face images using the U-net segmentation network, the segmentation feature layer is selected from the downsampled feature layer based on the above method. In the upsampling process, the output results of the segmentation feature layer in the downsampling process and the segmentation feature layer in the upsampling process (i.e., the feature layer with the same receptive field size as the downsampling) are skipped to improve the U-net segmentation network.

[0070] In the process of segmenting face images using the U-net segmentation network, the grayscale image of the face is used as the input to the U-net segmentation network. Following the steps described above, the segmentation layer corresponding to the receptive field is selected, and the output is the segmented grayscale image of the face. The processing procedure of the U-net segmentation network is a well-known technique and will not be elaborated here.

[0071] This application also discloses an image processing-based face image segmentation system, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement an image processing-based face image segmentation method according to this application.

[0072] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0073] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A face image segmentation method based on image processing, characterized in that, The method comprises the following steps: Obtaining a preprocessed face grayscale image; For any pixel point in the face grayscale image, a viewing window is constructed, the direction uniformity degree of the gradient direction of each pixel point in the viewing window is obtained, and the gradient level of the pixel points in the viewing window is obtained, and a high-frequency structure matching index reflecting the degree of coincidence of the pixel points with the detailed structure of the face image is constructed, wherein the direction uniformity degree and the gradient level are positively correlated with the high-frequency structure matching index; The volatility of the high-frequency structure matching index in the viewing window and the matching level of the high-frequency structure matching index as a whole in the viewing window are calculated, and the product of the volatility and the matching level is taken as the face structure saliency; based on the face structure saliency of each pixel point, a segmentation receptive field is screened out, and the image is segmented based on the U-net segmentation network of the feature layer corresponding to the segmentation receptive field.

2. The face image segmentation method based on image processing according to claim 1, characterized in that, The calculation step of the direction uniformity degree comprises: in the viewing window corresponding to any pixel point, any two pixel points form a comparison point pair; the gradient direction difference index between the two pixel points in the comparison point pair is calculated, and the reciprocal of the sum of the gradient direction difference index is taken as the gradient direction uniformity degree.

3. The face image segmentation method based on image processing according to claim 2, characterized in that, The gradient direction difference index between two pixel points in a comparison point pair is calculated according to the following formula: ; in the formula, the gradient direction difference index between two pixel points is represented by; the absolute value of the difference between the gradient directions of two pixel points is represented by; is a sine function.

4. The face image segmentation method based on image processing according to claim 1, characterized in that, The mean value of the gradient of the pixel points in the viewing window is taken as the gradient level of the viewing window.

5. The face image segmentation method based on image processing according to claim 1, characterized in that, The product of the gradient direction uniformity degree and the gradient level is taken as the high-frequency structure matching index.

6. The face image segmentation method based on image processing according to claim 1, characterized in that, The standard deviation of the high-frequency structure matching index of each pixel point in the viewing window is taken as the volatility of the high-frequency structure matching index in the viewing window.

7. The face image segmentation method based on image processing according to claim 1, characterized in that, The acquisition step of the matching level of the high-frequency structure matching index as a whole in the viewing window comprises: using a logarithmic function to process the high-frequency structure matching index of each pixel point in the viewing window, obtaining the matching degree, and taking the sum of the matching degrees of each pixel point as the matching level.

8. The face image segmentation method based on image processing according to claim 1, characterized in that, The step of screening out the segmentation receptive field based on the face structure saliency of each pixel point comprises: obtaining the optimal receptive field of each pixel point, wherein the face saliency is negatively correlated with the size of the optimal receptive field; the receptive field is screened based on the number of pixel points with the same optimal receptive field, and the segmentation receptive field is determined.

9. The face image segmentation method based on image processing according to claim 8, characterized in that, The step of screening the receptive field based on the number of pixel points with the same optimal receptive field to determine the segmentation receptive field comprises: counting the number of pixel points corresponding to each optimal receptive field size, establishing a priority according to the number, and adding the number of pixel points according to the priority, and when the cumulative proportion exceeds a set threshold, the current receptive field is selected as the segmentation receptive field.

10. An image processing based face image segmentation system, characterized by, The method comprises the following steps: A processor and a memory, the memory stores computer program instructions, when the computer program instructions are executed by the processor, a face image segmentation method based on image processing according to any one of claims 1-9 is realized.

Images