Facial image analysis system for Williams syndrome

Abstract

The invention discloses a face image analysis system for Williams syndrome, and the system comprises a face detection module which is used for recognizing and cutting a face region in a face image, and obtaining a training image and a test image; the facial feature extraction module is used for extracting a local facial region through a MobileFaceNet network according to the training image and remapping coordinate values; wherein the local face area comprises an eye area, a nose area and a mouth area; the pixel-by-pixel weighting module is used for performing weighting processing on the local face region of the training image to obtain a weighted image; the network training module is used for carrying out classification prediction on the weighted images through the VMama network to obtain a trained VMama network; and the image analysis module is used for inputting the test image into the trained VMamba network to obtain a facial image analysis result oriented to the Williams syndrome. According to the method, the face image with the Williams comprehensive certificate can be accurately analyzed.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and image recognition, and in particular relates to a facial image analysis system for Williams syndrome. Background Art

[0002] Williams syndrome (WS) is a genetic disease that is often accompanied by complications such as hypercalcemia, cardiovascular disease, and developmental delay. Early identification of WS is crucial for the prevention and treatment of its related complications. Current detection of WS mainly relies on comprehensive evaluation through genetic testing, clinical observation, cardiac examination, and blood tests. These testing processes are not only time-consuming, expensive, but also inconvenient.

[0003] With the rapid development of deep learning, the importance of facial images in disease recognition has become increasingly prominent. Researchers have designed many intelligent recognition methods for related diseases, including a multi-feature-based generative multi-view disease classification model and DeepGastalt, a facial image analysis framework based on computer vision and deep learning algorithms. These methods identify diseases by identifying the special feature differences between healthy faces and faces of diseased patients.

[0004] Although facial image analysis technology has shown potential in disease identification, existing methods are not specifically used for Williams syndrome identification and lack specificity. For example, the automatic facial recognition method based on deep convolutional neural network relies entirely on the model to extract features and does not actively utilize the characteristic facial features of patients with Williams syndrome. Although the method based on regional navigation and feature fusion combines global and local features, the fusion effect is not ideal and the processing method is relatively scattered. Therefore, there is little research on facial recognition technology for Williams syndrome, which mainly relies on traditional and inconvenient detection methods. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a facial image analysis system for Williams syndrome to solve the above problems in the prior art.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a facial image analysis system for Williams syndrome, comprising:

[0007] A face detection module is used to identify and crop the face area in the face image to obtain a training image and a test image;

[0008] A facial feature extraction module, used to extract local facial areas and remap coordinate values ​​through the MobileFaceNet network according to the training image; wherein the local facial areas include: eye areas, nose areas, and mouth areas;

[0009] A pixel-by-pixel weighting module, used for performing weighted processing on the local facial area of ​​the training image to obtain a weighted image;

[0010] A network training module, used for classifying and predicting the weighted image through a VMamba network to obtain a trained VMamba network;

[0011] The image analysis module is used to input the test image into the trained VMamba network to obtain the facial image analysis results for Williams syndrome.

[0012] Preferably, the system further comprises a data enhancement module for performing enhancement operations on the training data to obtain enhanced training data; wherein the enhancement operations comprise: random cropping, flipping, color perturbation and rotation.

[0013] Preferably, the facial feature extraction module comprises:

[0014] An eye region positioning unit, used to calculate the eye region through the coordinate values ​​of six points;

[0015] A nose area positioning unit, used to calculate the nose area through the coordinate values ​​of four points;

[0016] The mouth area positioning unit is used to calculate the mouth area through the coordinate values ​​of three points.

[0017] Preferably, in the pixel-by-pixel weighting module, first weights are applied to the eye area, nose area and mouth area respectively, and second weights are applied to the remaining areas.

[0018] Preferably, in the network training module, a visual state space model is selected as a backbone network to perform feature extraction from facial images and implement an end-to-end Williams disease face recognition task.

[0019] Preferably, the system further comprises a loss calculation module for calculating the loss of the VMamba network and optimizing the model parameters through a loss function.

[0020] Preferably, the loss function is:

[0021]

[0022] Where N is the number of samples, y i is the label of the i-th sample, is the model’s predicted label for the i-th sample, Represents the sigmoid function.

[0023] In a second aspect, the present invention further discloses a computer-readable storage medium having a computer program stored thereon, which implements the steps of the system described in the first aspect when executed by a processor.

[0024] In a third aspect, the present invention further discloses a computer program product, including a computer program, which implements the steps of the system described in the first aspect when executed by a processor.

[0025] Compared with the prior art, the present invention has the following advantages and technical effects:

[0026] The invention provides a facial image analysis system for Williams syndrome, comprising: a facial detection module, used for identifying and cutting out a human face area in a facial image, so as to obtain a training image and a test image; a facial feature extraction module, used for extracting a local facial area and remapping a coordinate value through a MobileFaceNet network according to the training image; wherein the local facial area includes an eye area, a nose area, and a mouth area; a pixel-by-pixel weighting module, used for performing weighted processing on the local facial area of ​​the training image, so as to obtain a weighted image; a network training module, used for performing classification prediction on the weighted image through a VMamba network, so as to obtain a trained VMamba network; and an image analysis module, used for inputting a test image into the trained VMamba network, so as to obtain a facial image analysis result for Williams syndrome.

[0027] The present invention puts the facial image into a facial feature extractor based on MobileFaceNet to obtain the coordinates of the left eye, right eye, nose and mouth; through a pixel-by-pixel weighting module, the global facial image and the detected local area are fused, so that more attention can be paid to the local area. The present invention assigns higher weights to important areas and lower weights to less important areas, which not only emphasizes the local features related to the task, but also retains the entire facial image as a global feature. Finally, the present invention puts the fused image into a VMamba model based on a state-space model to predict whether the facial image belongs to Williams syndrome. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 A system network structure diagram of an embodiment of the present invention;

[0030] Figure 2 Schematic diagram of MobileFaceNet extracting coordinate points according to an embodiment of the present invention, where (a) is the original coordinate point and (b) is the filtered coordinate point;

[0031] Figure 3 An image processed by a pixel-by-pixel weighting unit according to an embodiment of the present invention;

[0032] Figure 4 is a flow chart of network training according to an embodiment of the present invention;

[0033] Figure 5 The network test flow chart of the embodiment of the present invention. DETAILED DESCRIPTION

[0034] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] First, the technical terms involved in the following embodiments are explained.

[0037] (1) MobileFaceNet is an efficient and lightweight face recognition network designed for mobile devices. It optimizes the network structure and reduces the amount of computation to achieve real-time face recognition on mobile devices with limited resources. The model size of MobileFaceNet is only 4MB, but its recognition accuracy is comparable to that of large models, and even better in some cases.

[0038] The main contribution of MobileFaceNet lies in the design of its network structure. It uses global deep convolution (GDConv) to replace the traditional global average pooling layer. This design enables the network to obtain richer information, thereby improving recognition accuracy. In addition, MobileFaceNet uses the bottlenecks of MobileNetV2 as the basic module, and further optimizes the amount of calculation and parameters through technologies such as deep separable convolution.

[0039] Performance: On the LFW dataset, MobileFaceNet achieved 99.55% recognition accuracy, showing its excellent performance. After training on the MS-Celeb-1M dataset, its performance is even comparable to some large models. In addition, MobileFaceNet runs very fast on mobile devices, with the fastest version taking only 18 milliseconds to infer on a mobile phone, greatly improving the efficiency of face recognition.

[0040] MobileFaceNet is suitable for various scenarios that require real-time face recognition, such as identity authentication, payment verification, security monitoring, and social applications.

[0041] (2) Pixel-by-pixel weighting refers to the process of weighting each pixel of an image in image processing to achieve a specific image effect or enhance certain features of the image. Pixel-by-pixel weighting usually involves setting a weight for each pixel and then calculating the weighted average of the pixel values ​​based on these weights.

[0042] Pixel-by-pixel weighting has many application scenarios in image processing, such as:

[0043] Image fusion: In image fusion, pixel-by-pixel weighting can be used to merge multiple images. By setting the weights of different images, a new image can be generated while retaining the features of multiple images.

[0044] Image sharpening: In image sharpening, pixel-by-pixel weighting can be used to enhance the edges and details of an image. For example, the Laplace sharpening algorithm performs a weighted operation on a circle of nearby points for each pixel to achieve a sharpening effect.

[0045] The implementation of pixel-by-pixel weighting usually involves the following steps:

[0046] Read the image: First read the image to be processed.

[0047] Set weights: Set the appropriate weights for each pixel according to the specific effect you need to process.

[0048] Weighted average calculation: Perform weighted average calculation on the gray value of each pixel to generate a new image.

[0049] The advantages of pixel-by-pixel weighting include:

[0050] High flexibility: The weight of each pixel can be adjusted as needed to achieve a variety of image processing effects.

[0051] Wide range of applications: Suitable for a variety of image processing tasks, such as fusion, sharpening, etc.

[0052] Pixel-by-pixel weighting has wide applications and important technical value in image processing.

[0053] (3) VMamba network is a linear time series modeling method based on selective state space, designed to process long sequence data.

[0054] The core advantages of the VMamba network lie in its global receptive field and dynamic weighting properties, which enable it to excel in long sequence modeling tasks, alleviate the modeling constraints of convolutional neural networks, and provide advanced modeling capabilities similar to the Transformer without incurring the secondary computational complexity associated with the Transformer.

[0055] The VMamba model was proposed by researchers from the Chinese Academy of Sciences, Huawei, and Pengcheng Laboratory to solve complex problems in visual tasks. The model has a global receptive field and linear complexity, and is suitable for a variety of computer vision tasks.

[0056] VMamba network has demonstrated its strong application potential in many fields, including:

[0057] Biomedical image segmentation: The U-Mamba network architecture performs well in the field of biomedical image segmentation. It can capture local fine-grained features of images and establish long-range dependencies. Compared with the traditional Transformer architecture, its feature expansion capability is more linear.

[0058] Medical image segmentation: The Weak-Mamba-UNet framework reduces the data annotation cost through weakly supervised learning, combines the feature learning capabilities of CNN, ViT and VMamba, and improves the accuracy of medical image segmentation.

[0059] 2D Medical Image Segmentation: The Swin-UMamba network integrates a pre-trained model based on ImageNet, verifying its superior performance on different medical image segmentation datasets.

[0060] like Figure 1 As shown, in this embodiment, a facial image analysis system for Williams syndrome is provided, including:

[0061] A face detection module is used to identify and crop the face area in the face image to obtain a training image and a test image;

[0062] Specifically, considering that naturally captured facial images usually contain many areas irrelevant to the face, these irrelevant areas will affect the training of the Williams syndrome recognition network. Therefore, for naturally captured facial images, it is necessary to first use a face detector to detect and crop the facial area image as a preprocessing. In this embodiment, the popular face detector YOLO-FaceV2 is used to detect the face, crop the non-face area, and only retain the face area in the image.

[0063] The present embodiment provides an overall architecture WLGFFVMN, which consists of three parts: a facial feature extractor module, a pixel-by-pixel weighting module, and a VMamba network. WLGFFVMN first receives pre-processed facial images of children with Williams syndrome and normal children. Then, these images are innovatively placed in a facial feature extractor based on MobileFaceNet to obtain the coordinates of the left eye, right eye, nose, and mouth. To ensure that the network can pay more attention to these local distinguished areas, the present embodiment innovatively designs a pixel-by-pixel weighting unit to fuse the global facial image and these detected local areas. In this way, important areas such as eyes, nose, and mouth will be given higher weights, while less important areas will be given lower weights, which not only emphasizes the local features related to the task, but also retains the entire facial image as a global feature. Finally, the fused image is placed in a VMamba model based on a state-space model to determine whether these faces belong to children with Williams syndrome.

[0064] First, a facial feature extraction module is used to extract local facial areas and remap coordinate values ​​through the MobileFaceNet network according to the training image; wherein the local facial areas include: eye areas, nose areas, and mouth areas;

[0065] As an innovative implementation, a lightweight MobileFaceNet network is used to detect the coordinates of facial landmarks. Figure 2 As shown in (a), the network initially detects 68 sets of coordinates scattered on the face. This embodiment designs the following technology to filter out coordinates that can concisely and effectively locate facial features from these 68 sets of coordinates.

[0066] To locate the eye region, the original MobileFaceNet model uses six points to depict the position of each eye, and this embodiment calculates the average height and width values ​​of the six points to obtain the most likely position of the eye center.

[0067] For the nose, the original MobileFaceNet model uses nine points for positioning, and this embodiment selects four typical points among the nine points to locate it: the topmost, bottommost, leftmost, and rightmost points.

[0068] For the mouth, the original MobileFaceNet model uses 20 points to locate. This embodiment first selects the leftmost and rightmost points of the 20 points as the horizontal boundaries. The original model uses three points to mark the top and bottom of the mouth respectively. This embodiment obtains the vertical boundary of the mouth by calculating the average height and width of these three points. The result of this process is shown in Figure 2 (b) as shown.

[0069] Second, a pixel-by-pixel weighting module is used to perform weighted processing on the local facial area of ​​the training image to obtain a weighted image;

[0070] Specifically, after obtaining the positions of the left eye, right eye, nose and mouth, this embodiment applies weights to the original image according to the extracted facial features. For each eye, a square area with a fixed height and width is intercepted. Then, this square area is added to the corresponding eye area of ​​the global facial image, with the center of the eye as its symmetry center. For the nose and mouth, a rectangular area is intercepted using the above four points.

[0071] As an innovative implementation, after experiments to optimize the relevant parameters, a weight of 1.053 is applied pixel by pixel to these regions to enhance their influence on the subsequent model, while other less important regions are given a weight of 0.950, such as Figure 3 As shown in Figure 2. After this process, the local facial area that is crucial for WS recognition is highlighted, while other areas will be weakened accordingly. In addition, through this operation, the connection between local features and global features is preserved.

[0072] Third, a network training module is used to classify and predict the weighted image through a VMamba network to obtain a trained VMamba network;

[0073] Specifically, extracting discriminative features from facial images is crucial to achieving better Williams' face recognition performance. Convolutional neural networks (CNNs) and Transformers are two of the most popular tools for feature extraction in recent years. Although Transformers outperform convolutional neural networks in visual modeling, they are more computationally expensive. Compared with convolutional neural networks and Transformers, the newly proposed VMamba network takes both performance and efficiency into consideration and achieves excellent performance on several visual perception tasks. Therefore, this embodiment selects the visual state space model (VMamba) as the backbone network to perform feature extraction from facial images and implement an end-to-end Williams' face recognition task.

[0074] Considering that Williams syndrome face recognition is a typical binary classification problem, this embodiment selects the binary cross entropy loss (BCEWithLogitsLoss) based on the sigmoid function as the loss function. BCEWithLogitsLoss receives the original output values ​​of the model and applies the sigmoid function to them to convert them into probabilities. Since this loss function effectively prevents calculation overflow, it can better ensure numerical stability. The calculation formula of BCEWithLogitsLoss is:

[0075]

[0076] In the above formula, N is the number of samples, y i is the label of the i-th sample, is the model’s predicted label for the i-th sample, Represents the sigmoid function.

[0077] like Figure 4 As shown, the specific training steps of this embodiment are introduced as follows:

[0078] Step 1: Cropping preprocessing. The collected image data is subjected to cropping preprocessing. In this embodiment, YOLO-FaceV2 is used to identify the area where the face is located in the original image, and the face image is cropped from irrelevant content such as the background to form an image with the face as the core content.

[0079] Step 2: Data division: 80% of the preprocessed images are selected as the training set data for the model.

[0080] Step 3: Data enhancement. Since facial images of children with Williams syndrome are difficult to collect and there are few facial images of normal Asian children, the capacity of the data set used for training is very small. When the training set is very small, overfitting is prone to occur, resulting in weak generalization ability of the model and low test accuracy. Therefore, this embodiment uses data enhancement to expand the capacity of the data set before training begins. The specific method is: first, randomly crop the image to a size of 224×224×3, then randomly flip it horizontally, randomly perturb its color, and finally randomly rotate the image to [0°, 30°].

[0081] Step 4: Extract local facial regions. First, resize the image to 112×112×3, and then input it into the facial feature extractor MobileFaceNet. Filter and calculate the coordinates of the local facial region according to the above method, and then remap the coordinate values ​​to a 224×224×3 space.

[0082] Step 5: Pixel-by-pixel weighting. According to the coordinate information obtained in step 4, pixel-by-pixel weighting is performed in the input image of size 224×224×3. The areas that need to be weighted are: the square area with the left eye center as the geometric center and a side length of 25, the square area with the right eye center as the geometric center and a side length of 25, the area of ​​the nose determined by the aforementioned horizontal and vertical boundaries, and the area of ​​the mouth determined by the aforementioned horizontal and vertical boundaries. Apply a weight of 1.053 to the above areas. Apply a weight of 0.950 to the remaining areas.

[0083] Step 6: VMamba network classification. Input the image processed in step 5 into the VMamba network to obtain the network prediction label.

[0084] Step 7: Network loss calculation. Using the true label of the image and the predicted label generated by the network in step 6, the network loss is calculated using Formula 1.

[0085] Step 8: Determine the termination condition. If the number of iterations is equal to the maximum number of iterations (set to 300), end the training, otherwise proceed to step 9.

[0086] Step 9: Gradient feedback optimization. Based on the network loss value obtained in the previous step, use the gradient descent optimization algorithm to optimize the parameters of all network models once. Jump to step 3 and continue to execute.

[0087] The image analysis module is used to input the test image into the trained VMamba network to obtain the facial image analysis results for Williams syndrome.

[0088] like Figure 5 As shown, the specific test steps of this embodiment are described as follows:

[0089] Step 1: Cropping preprocessing. The collected image data is subjected to cropping preprocessing. In this embodiment, YOLO-FaceV2 is used to identify the area where the face is located in the original image, and the face image is cropped from irrelevant content such as the background to form an image with the face as the core content.

[0090] Step 2: Data division: 20% of the preprocessed images are selected as the test set data for the model.

[0091] Step 3: Extract the local facial region. First, resize the image copy to 112×112×3, and then input it into the facial feature extractor MobileFaceNet. Filter and calculate the coordinates of the local facial region according to the above method, and then remap the coordinate values ​​to a 224×224×3 space.

[0092] Step 4: Pixel-by-pixel weighting. According to the coordinate information obtained in step 4, pixel-by-pixel weighting is performed in the input image of size 224×224×3. The areas that need to be weighted are: the square area with the left eye center as the geometric center and a side length of 25, the square area with the right eye center as the geometric center and a side length of 25, the area of ​​the nose determined by the aforementioned horizontal and vertical boundaries, and the area of ​​the mouth determined by the aforementioned horizontal and vertical boundaries. Apply a weight of 1.053 to the above areas. Apply a weight of 0.950 to the remaining areas.

[0093] Step 5: VMamba network classification. The image processed in step 5 is input into the VMamba network to obtain the network prediction label.

[0094] Step 6: Recognition result output. The original predicted label generated by the network in step 6 is projected to the interval [0,1] using the sigmoid activation function. According to the binary label definition rule, label values ​​greater than or equal to 0.5 are set to 1, and label values ​​less than 0.5 are set to 0, thus obtaining the final 0-1 binary label of the input image.

[0095] The experimental comparison is as follows:

[0096] The experimental results of this embodiment on the self-built WS image dataset show that the accuracy of this embodiment in the face-based WS recognition task reaches 96.610%, surpassing other traditional deep convolutional neural networks. At the same time, the recall rate and precision of this embodiment in the recognition task reach 84.615% and 100% respectively, and when identifying suspected Williams children, it can accurately and almost without omission screen out patients who do have Williams. This embodiment can be applied in the field of Williams recognition as an important auxiliary recognition technology for doctors to identify Williams patients early and provide them with better treatment, thereby ensuring their quality of life. The effects of this embodiment and the traditional deep convolutional neural network on the Williams face recognition task are shown in Table 1.

[0097] Table 1

[0098] method Accuracy Recall Accuracy F1 VGG16 88.136 76.923 71.429 74.074 VGG19 91.525 69.231 90.000 78.261 AlexNet 89.831 61.539 88.889 72.727 ResNet18 91.525 69.231 90.000 78.261 ResNet34 93.220 69.231 100 81.818 ResNet50 91.525 69.231 90.000 78.261 ResNet101 88.136 76.923 71.429 74.074 This embodiment 96.610 84.615 100 91.667

[0099] Embodiment 2

[0100] This embodiment also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the system described in the first embodiment are implemented.

[0101] Embodiment 3

[0102] This embodiment also discloses a computer program product, including a computer program, which implements the steps of the system described in the first embodiment when executed by a processor.

[0103] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A facial image analysis system for Williams syndrome, characterized in that: include: A face detection module is used to identify and crop the face area in the face image to obtain a training image and a test image; A facial feature extraction module, used to extract local facial areas and remap coordinate values ​​through a MobileFaceNet network according to the training image; The local facial areas include: eye area, nose area, mouth area; A pixel-by-pixel weighting module, used for performing weighted processing on the local facial area of ​​the training image to obtain a weighted image; A network training module, used for classifying and predicting the weighted image through a VMamba network to obtain a trained VMamba network; The image analysis module is used to input the test image into the trained VMamba network to obtain the facial image analysis results for Williams syndrome.

2. The system according to claim 1, characterized in that It also includes a data enhancement module, which is used to perform enhancement operations on the training data to obtain enhanced training data; wherein the enhancement operations include: random cropping, flipping, color perturbation and rotation.

3. The system according to claim 1, characterized in that The facial feature extraction module comprises: An eye region positioning unit, used to calculate the eye region through the coordinate values ​​of six points; A nose area positioning unit, used to calculate the nose area through the coordinate values ​​of four points; The mouth area positioning unit is used to calculate the mouth area through the coordinate values ​​of three points.

4. The system according to claim 1, characterized in that In the pixel-by-pixel weighting module, first weights are applied to the eye area, nose area and mouth area respectively, and second weights are applied to the remaining areas.

5. The system according to claim 1, characterized in that In the network training module, a visual state space model is selected as the backbone network to perform feature extraction from facial images and implement an end-to-end Williams disease face recognition task.

6. The system according to claim 1, characterized in that It also includes a loss calculation module, which is used to calculate the loss of the VMamba network and optimize the model parameters through the loss function.

7. The system according to claim 6, characterized in that The loss function is: Where N is the number of samples, y i is the label of the i-th sample, is the model’s predicted label for the i-th sample, Represents the sigmoid function.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the system according to claims 1 to 7 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the system according to claims 1 to 7 are implemented.

Images