A facial image analysis system for Williams syndrome

Abstract

The present invention discloses a facial image analysis system for Williams syndrome, comprising: a facial detection module for identifying and cropping facial regions in facial images to obtain training images and test images; a facial feature extraction module for extracting local facial regions from the training images using a MobileFaceNet network and remapping their coordinate values; the local facial regions include the eye region, the nose region, and the mouth region; a pixel-by-pixel weighting module for weighting the local facial regions of the training images to obtain a weighted image; a network training module for classifying and predicting the weighted image using a VMamba network to obtain a trained VMamba network; and an image analysis module for inputting a test image into the trained VMamba network to obtain facial image analysis results for Williams syndrome. The present invention can accurately analyze facial images of people with Williams syndrome.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image recognition, and in particular relates to a facial image analysis system for Williams syndrome. Background Art

[0002] Williams syndrome (WS) is a genetic disorder often associated with complications such as hypercalcemia, cardiovascular disease, and developmental delay. Early identification of WS is crucial for preventing and treating its associated complications. Current testing for WS relies primarily on comprehensive evaluation using genetic testing, clinical observation, cardiac examinations, and blood tests. These procedures are time-consuming, expensive, and inconvenient.

[0003] With the rapid development of deep learning, facial images are becoming increasingly important in disease identification. Researchers have designed numerous intelligent disease identification methods, including a multi-feature-based generative multi-view disease classification model and DeepGastalt, a facial image analysis framework based on computer vision and deep learning algorithms. These methods identify disease by identifying specific features that differ between healthy and diseased faces.

[0004] Although facial image analysis technology has shown potential in disease identification, existing methods are not specifically designed for Williams syndrome identification and lack specificity. For example, automatic facial recognition methods based on deep convolutional neural networks rely entirely on model-based feature extraction and fail to actively utilize the characteristic facial features of patients with Williams syndrome. While methods based on region navigation and feature fusion combine global and local features, the fusion effect is less than ideal and the processing methods are relatively fragmented. Consequently, research on facial recognition technology for Williams syndrome is limited, primarily relying on traditional and inconvenient detection methods. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a facial image analysis system for Williams syndrome to solve the problems existing in the above prior art.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a facial image analysis system for Williams syndrome, comprising:

[0007] The face detection module is used to identify and crop the face area in the face image to obtain the training image and the test image;

[0008] A facial feature extraction module is used to extract local facial regions and remap coordinate values ​​based on the training image using the MobileFaceNet network; the local facial regions include: eye regions, nose regions, and mouth regions;

[0009] a pixel-by-pixel weighting module, configured to perform weighted processing on the local facial region of the training image to obtain a weighted image;

[0010] A network training module is used to perform classification prediction on the weighted image through a VMamba network to obtain a trained VMamba network;

[0011] The image analysis module is used to input the test image into the trained VMamba network to obtain the facial image analysis results for Williams syndrome.

[0012] Preferably, the system further comprises a data enhancement module for performing an enhancement operation on the training data to obtain enhanced training data; wherein the enhancement operation comprises: random cropping, flipping, color perturbation and rotation.

[0013] Preferably, the facial feature extraction module includes:

[0014] An eye region positioning unit, used to calculate the eye region using the coordinate values ​​of six points;

[0015] A nose area positioning unit is used to calculate the nose area using the coordinate values ​​of four points;

[0016] The mouth area positioning unit is used to calculate the mouth area through the coordinate values ​​of three points.

[0017] Preferably, in the pixel-by-pixel weighting module, first weights are applied to the eye area, nose area and mouth area respectively, and second weights are applied to the remaining areas.

[0018] Preferably, in the network training module, a visual state space model is selected as the backbone network to perform feature extraction from facial images and implement an end-to-end Williams syndrome face recognition task.

[0019] Preferably, the system further includes a loss calculation module for calculating the loss of the VMamba network and optimizing model parameters through a loss function.

[0020] Preferably, the loss function is:

[0021]

[0022] Where N is the number of samples, y i is the label of the i-th sample, is the model’s predicted label for the i-th sample, Represents the sigmoid function.

[0023] In a second aspect, the present invention further discloses a computer-readable storage medium having a computer program stored thereon, which implements the steps of the system described in the first aspect when executed by a processor.

[0024] In a third aspect, the present invention further discloses a computer program product, comprising a computer program, which implements the steps of the system described in the first aspect when executed by a processor.

[0025] Compared with the prior art, the present invention has the following advantages and technical effects:

[0026] The present invention provides a facial image analysis system for Williams syndrome, comprising: a facial detection module for identifying and cropping facial regions in facial images to obtain training images and test images; a facial feature extraction module for extracting local facial regions and remapping coordinate values ​​based on the training images through a MobileFaceNet network, wherein the local facial regions include eye regions, nose regions, and mouth regions; a pixel-by-pixel weighting module for performing weighted processing on the local facial regions of the training images to obtain weighted images; a network training module for performing classification prediction on the weighted images through a VMamba network to obtain a trained VMamba network; and an image analysis module for inputting a test image into the trained VMamba network to obtain facial image analysis results for Williams syndrome.

[0027] The present invention applies a facial image to a MobileFaceNet-based facial feature extractor to obtain the coordinates of the left and right eyes, nose, and mouth. A pixel-by-pixel weighting module is then used to fuse the global facial image with the detected local regions, focusing more attention on these regions. Important regions are assigned higher weights, while less important regions are assigned lower weights. This not only emphasizes task-relevant local features but also retains the entire facial image as a global feature. Finally, the present invention applies the fused image to a VMamba model based on a state-space model to predict whether the facial image posses Williams syndrome. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0029] Figure 1 A diagram showing the system network structure of an embodiment of the present invention;

[0030] Figure 2 Schematic diagram of MobileFaceNet extraction of coordinate points according to an embodiment of the present invention, where (a) is the original coordinate point and (b) is the filtered coordinate point;

[0031] Figure 3 An image processed by a pixel-by-pixel weighting unit according to an embodiment of the present invention;

[0032] Figure 4 This is a flow chart of network training according to an embodiment of the present invention;

[0033] Figure 5 This is a flow chart of a network test according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] First, the technical terms involved in the following embodiments are explained.

[0037] (1) MobileFaceNet is an efficient and lightweight face recognition network designed specifically for mobile devices. By optimizing the network structure and reducing the amount of computation, it enables real-time face recognition on resource-limited mobile devices. The MobileFaceNet model is only 4MB in size, but its recognition accuracy is comparable to that of larger models, and in some cases even better.

[0038] MobileFaceNet's primary contribution lies in its network architecture. It uses global depthwise convolution (GDConv) instead of traditional global average pooling layers. This design enables the network to capture richer information, thereby improving recognition accuracy. Furthermore, MobileFaceNet uses the bottlenecks of MobileNetV2 as its basic building block and further optimizes computational complexity and parameter requirements through techniques such as depthwise separable convolution.

[0039] Performance: MobileFaceNet achieved 99.55% recognition accuracy on the LFW dataset, demonstrating its excellent performance. After training on the MS-Celeb-1M dataset, its performance is even comparable to some larger models. Furthermore, MobileFaceNet runs very fast on mobile devices, with the fastest version achieving inference time of only 18 milliseconds on mobile phones, significantly improving face recognition efficiency.

[0040] MobileFaceNet is suitable for various scenarios that require real-time face recognition, such as identity authentication, payment verification, security monitoring, and social applications.

[0041] (2) Pixel-by-pixel weighting refers to the process of weighting each pixel in an image to achieve a specific image effect or enhance certain image features. Pixel-by-pixel weighting typically involves setting a weight for each pixel and then performing a weighted average calculation of the pixel values ​​based on these weights.

[0042] Pixel-by-pixel weighting has many application scenarios in image processing, such as:

[0043] Image fusion: In image fusion, pixel-by-pixel weighting can be used to merge multiple images. By setting the weights of different images, a new image can be generated while retaining the features of multiple images.

[0044] Image sharpening: In image sharpening, pixel-by-pixel weighting can be used to enhance image edges and details. For example, the Laplace sharpening algorithm performs a weighted calculation on each pixel by finding a circle of nearby points to achieve a sharpening effect.

[0045] The implementation of pixel-by-pixel weighting usually involves the following steps:

[0046] Read image: First read the image to be processed.

[0047] Set weights: Set the appropriate weights for each pixel based on the specific effect you want to process.

[0048] Weighted average calculation: Perform weighted average calculation on the grayscale value of each pixel to generate a new image.

[0049] The advantages of pixel-by-pixel weighting include:

[0050] High flexibility: The weight of each pixel can be adjusted as needed to achieve a variety of image processing effects.

[0051] Wide range of applications: Suitable for a variety of image processing tasks, such as fusion, sharpening, etc.

[0052] Pixel-by-pixel weighting has wide applications and important technical value in image processing.

[0053] (3) VMamba network is a linear time series modeling method based on selective state space, designed to process long sequence data.

[0054] The core advantages of the VMamba network lie in its global receptive field and dynamic weighting properties, which enable it to excel in long sequence modeling tasks, alleviate the modeling constraints of convolutional neural networks, and provide advanced modeling capabilities similar to Transformer without incurring the quadratic computational complexity associated with Transformer.

[0055] The VMamba model, proposed by researchers from the Chinese Academy of Sciences, Huawei, and Pengcheng Laboratory, aims to address complex problems in visual tasks. The model has a global receptive field and linear complexity, making it suitable for a variety of computer vision tasks.

[0056] VMamba network has demonstrated its strong application potential in many areas, including:

[0057] Biomedical image segmentation: The U-Mamba network architecture performs well in the field of biomedical image segmentation. It can capture local fine-grained features of images and establish long-range dependencies. Compared with the traditional Transformer architecture, its feature expansion capability has a more linear advantage.

[0058] Medical image segmentation: The Weak-Mamba-UNet framework reduces data annotation costs through weakly supervised learning and combines the feature learning capabilities of CNN, ViT, and VMamba to improve the accuracy of medical image segmentation.

[0059] 2D medical image segmentation: The Swin-UMamba network integrates a pre-trained model based on ImageNet, verifying its superior performance on different medical image segmentation datasets.

[0060] like Figure 1 As shown, this embodiment provides a facial image analysis system for Williams syndrome, including:

[0061] The face detection module is used to identify and crop the face area in the face image to obtain the training image and the test image;

[0062] Specifically, naturally captured facial images often contain many areas unrelated to the face, which can affect the training of the Williams syndrome recognition network. Therefore, for naturally captured facial images, it is necessary to first use a face detector to detect and crop the facial region of the image as a preprocessing step. In this example, the popular face detector YOLO-FaceV2 is used to detect faces, crop non-facial regions, and retain only the facial region in the image.

[0063] This embodiment provides an overall architecture, WLGFFVMN, consisting of three components: a facial feature extractor module, a pixel-by-pixel weighting module, and a VMamba network. WLGFFVMN first receives preprocessed facial images of children with Williams syndrome and healthy children. These images are then innovatively fed into a facial feature extractor based on MobileFaceNet to obtain the coordinates of the left and right eyes, nose, and mouth. To ensure that the network focuses more on these locally distinguished regions, this embodiment innovatively designs a pixel-by-pixel weighting unit to fuse the global facial image with these detected local regions. This approach assigns higher weights to important regions such as the eyes, nose, and mouth, while lower weights are assigned to less important regions. This not only emphasizes task-relevant local features but also preserves the entire facial image as a global feature. Finally, the fused images are fed into a VMamba model based on a state-space model to determine whether the faces belong to children with Williams syndrome.

[0064] First, a facial feature extraction module is used to extract local facial regions and remap coordinate values ​​based on the training image using the MobileFaceNet network; the local facial regions include: eye region, nose region, and mouth region;

[0065] As an innovative implementation, a lightweight MobileFaceNet network is used to detect the coordinates of facial landmarks. Figure 2 As shown in (a), the network initially detects 68 sets of coordinates scattered around the face. This embodiment designs the following technology to filter out coordinates that can concisely and effectively locate facial features from these 68 sets of coordinates.

[0066] To locate the eye area, the original MobileFaceNet model uses six points to depict the position of each eye, while this embodiment calculates the average height and width values ​​of these six points to obtain the most likely position of the eye center.

[0067] For the nose, the original MobileFaceNet model uses nine points for positioning. This embodiment selects four typical points among these nine points to locate it: the topmost, bottommost, leftmost, and rightmost points.

[0068] For the mouth, the original MobileFaceNet model uses 20 points to locate the mouth. This embodiment first selects the leftmost and rightmost points of the 20 points as the horizontal boundaries. The original model uses three points to mark the top and bottom of the mouth respectively. This embodiment obtains the vertical boundary of the mouth by calculating the average height and width of these three points. The result of this process is shown in Figure 2. Figure 2 (b) shown.

[0069] Second, a pixel-by-pixel weighting module is used to perform weighted processing on the local facial area of ​​the training image to obtain a weighted image;

[0070] Specifically, after obtaining the positions of the left and right eyes, nose, and mouth, this embodiment applies weights to the original image based on the extracted facial features. For each eye, a square region with a fixed height and width is captured. This square region is then added to the corresponding eye region of the global facial image, with the center of the eye as its center of symmetry. For the nose and mouth, a rectangular region is captured using the aforementioned four points.

[0071] As an innovative implementation, after experiments to optimize relevant parameters, a weight of 1.053 is applied pixel by pixel to these regions to enhance their influence on subsequent models, while other less important regions are given a weight of 0.950, e.g. Figure 3 As shown in Figure 2. After this process, the local facial regions that are crucial for WS recognition are highlighted, while other regions are weakened accordingly. In addition, through this operation, the connection between local features and global features is preserved.

[0072] Third, a network training module is used to perform classification prediction on the weighted image through a VMamba network to obtain a trained VMamba network;

[0073] Specifically, extracting discriminative features from facial images is crucial for achieving better Williams syndrome face recognition performance. Convolutional neural networks (CNNs) and Transformers are two of the most popular tools for feature extraction in recent years. Although Transformers outperform convolutional neural networks in visual modeling, they are computationally more expensive. Compared with convolutional neural networks and Transformers, the newly proposed VMamba network takes both performance and efficiency into consideration, achieving excellent performance on several visual perception tasks. Therefore, this embodiment selects the visual state space model (VMamba) as the backbone network to perform feature extraction from facial images and implement end-to-end Williams syndrome face recognition tasks.

[0074] Considering that Williams syndrome face recognition is a typical binary classification problem, this embodiment uses the sigmoid-based binary cross entropy loss (BCEWithLogitsLoss) as the loss function. BCEWithLogitsLoss receives the model's raw output values ​​and applies a sigmoid function to them, converting them into probabilities. Because this loss function effectively prevents computational overflow, it can better ensure numerical stability. The calculation formula for BCEWithLogitsLoss is:

[0075]

[0076] In the above formula, N is the number of samples, y i is the label of the i-th sample, is the model’s predicted label for the i-th sample, Represents the sigmoid function.

[0077] like Figure 4 As shown, the specific training steps of this embodiment are described as follows:

[0078] Step 1: Cropping Preprocessing. The collected image data is subjected to cropping preprocessing. In this example, YOLO-FaceV2 is used to identify the face region in the original image and crop the face image from irrelevant content such as the background, forming an image with the face as the core content.

[0079] Step 2: Data division: 80% of the pre-processed images are selected as the training set data for the model.

[0080] Step 3: Data augmentation. Due to the difficulty in collecting facial images of children with Williams syndrome and the scarcity of facial images of normal Asian children, the dataset used for training is very small. When the training set is very small, overfitting is prone to occur, resulting in weak generalization of the model and low test accuracy. Therefore, this embodiment uses data augmentation to expand the dataset capacity before training begins. The specific method is as follows: first, randomly crop the image to a size of 224×224×3, then randomly flip it horizontally, randomly perturb its color, and finally randomly rotate the image [0°, 30°].

[0081] Step 4: Extract the local facial region. First, resize the image to 112×112×3 and then feed it into the facial feature extractor MobileFaceNet. Filter and calculate the local facial region coordinates using the aforementioned method, then remap the coordinates to a 224×224×3 space.

[0082] Step 5: Pixel-by-Pixel Weighting. Based on the coordinate information obtained in Step 4, perform pixel-by-pixel weighting on the 224×224×3 input image. The following areas require additional weighting: the square area with a side length of 25° centered on the left eye center, the square area with a side length of 25° centered on the right eye center, the area of ​​the nose defined by the aforementioned horizontal and vertical boundaries, and the area of ​​the mouth defined by the aforementioned horizontal and vertical boundaries. A weight of 1.053 is applied to these areas. A weight of 0.950 is applied to the remaining areas.

[0083] Step 6: VMamba network classification: Input the image processed in step 5 into the VMamba network to obtain the network predicted label.

[0084] Step 7: Network loss calculation. Using the true label of the image and the predicted label generated by the network in step 6, the network loss is calculated using Formula 1.

[0085] Step 8: Termination condition judgment. If the number of iteration steps is equal to the maximum number of iteration steps (set to 300), end the training, otherwise proceed to step 9.

[0086] Step 9: Gradient Feedback Optimization. Based on the network loss value obtained in the previous step, use the gradient descent optimization algorithm to optimize all network model parameters. Jump to step 3 to continue.

[0087] The image analysis module is used to input the test image into the trained VMamba network to obtain the facial image analysis results for Williams syndrome.

[0088] like Figure 5 As shown, the specific test steps of this embodiment are described as follows:

[0089] Step 1: Cropping Preprocessing. The collected image data is subjected to cropping preprocessing. In this example, YOLO-FaceV2 is used to identify the face region in the original image and crop the face image from irrelevant content such as the background, forming an image with the face as the core content.

[0090] Step 2: Data division: 20% of the preprocessed images are selected as the test set data for the model.

[0091] Step 3: Extract the local facial region. First, resize the image to 112×112×3 and then feed it into the facial feature extractor MobileFaceNet. Filter and calculate the local facial region coordinates using the aforementioned method, then remap the coordinates to a 224×224×3 space.

[0092] Step 4: Pixel-by-Pixel Weighting. Based on the coordinate information obtained in Step 4, perform pixel-by-pixel weighting on the 224×224×3 input image. The following areas require additional weighting: the square area with a side length of 25° centered on the left eye center, the square area with a side length of 25° centered on the right eye center, the area of ​​the nose defined by the aforementioned horizontal and vertical boundaries, and the area of ​​the mouth defined by the aforementioned horizontal and vertical boundaries. A weight of 1.053 is applied to these areas. A weight of 0.950 is applied to the remaining areas.

[0093] Step 5: VMamba network classification: Input the image processed in step 5 into the VMamba network to obtain the network predicted label.

[0094] Step 6: Recognition result output. A sigmoid activation function is applied to the original predicted labels generated by the network in step 6, projecting them to the interval [0, 1]. Based on the binary classification label definition rules, label values ​​greater than or equal to 0.5 are set to 1, and label values ​​less than 0.5 are set to 0, resulting in the final 0-1 binary label for the input image.

[0095] The experimental comparison is as follows:

[0096] Experimental results of this embodiment on a self-built WS image dataset show that this embodiment achieves an accuracy of 96.610% in face-based WS recognition tasks, surpassing other traditional deep convolutional neural networks. At the same time, the recall and precision of this embodiment in the recognition task reached 84.615% and 100%, respectively. When identifying children suspected of having Williams syndrome, it can accurately and nearly completely screen out patients who actually have Williams syndrome. This embodiment can be applied in the field of Williams syndrome identification as an important auxiliary recognition technology for doctors, enabling early identification of Williams syndrome patients and providing them with better treatment, thereby ensuring their quality of life. The performance of this embodiment and traditional deep convolutional neural networks in the Williams syndrome face recognition task is shown in Table 1.

[0097] Table 1

[0098] method Accuracy Recall Accuracy F1 VGG16 88.136 76.923 71.429 74.074 VGG19 91.525 69.231 90.000 78.261 AlexNet 89.831 61.539 88.889 72.727 ResNet18 91.525 69.231 90.000 78.261 ResNet34 93.220 69.231 100 81.818 ResNet50 91.525 69.231 90.000 78.261 ResNet101 88.136 76.923 71.429 74.074 This embodiment 96.610 84.615 100 91.667

[0099] Example 2

[0100] This embodiment further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the system described in the first embodiment are implemented.

[0101] Example 3

[0102] This embodiment also discloses a computer program product, including a computer program, which implements the steps of the system described in the first embodiment when executed by a processor.

[0103] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A facial image analysis system for Williams syndrome, characterized in that: include: The face detection module is used to identify and crop the face area in the face image to obtain the training image and the test image; A facial feature extraction module is used to extract local facial regions and remap coordinate values ​​using a MobileFaceNet network based on the training image; The local facial areas include: eye area, nose area, mouth area; a pixel-by-pixel weighting module, configured to perform weighted processing on the local facial region of the training image to obtain a weighted image; in the pixel-by-pixel weighting module, a first weight is applied to the eye region, the nose region, and the mouth region, respectively, and a second weight is applied to the remaining regions; A network training module is used to perform classification prediction on the weighted image through a VMamba network to obtain a trained VMamba network; The image analysis module is used to input the test image into the trained VMamba network to obtain the facial image analysis results for Williams syndrome.

2. The system according to claim 1, wherein: It also includes a data enhancement module for performing enhancement operations on the training data to obtain enhanced training data; wherein the enhancement operations include: random cropping, flipping, color perturbation and rotation.

3. The system according to claim 1, wherein: The facial feature extraction module includes: An eye region positioning unit, used to calculate the eye region using the coordinate values ​​of six points; A nose area positioning unit is used to calculate the nose area using the coordinate values ​​of four points; The mouth area positioning unit is used to calculate the mouth area through the coordinate values ​​of three points.

4. The system according to claim 1, wherein: In the network training module, a visual state space model is selected as the backbone network to perform feature extraction from facial images and implement end-to-end Williams syndrome face recognition tasks.

5. The system according to claim 1, wherein: It also includes a loss calculation module, which is used to calculate the loss of the VMamba network and optimize the model parameters through the loss function.

6. The system according to claim 5, characterized in that The loss function is: Where N is the number of samples, y i is the label of the i-th sample, is the model’s predicted label for the i-th sample, Represents the sigmoid function.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the system according to any one of claims 1 to 6 are implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the system according to any one of claims 1 to 6 are implemented.

Images