Newborn facial feature rare disease auxiliary diagnosis system based on AI image recognition

Abstract

The application discloses a rare disease auxiliary diagnosis system for neonatal facial features based on AI image recognition, relates to the rare disease auxiliary diagnosis technical field, and aims to solve the technical problem of insufficient dynamic feature analysis technology in the existing auxiliary diagnosis of neonatal rare diseases, and comprises the following steps: a skin analysis module is used to detect, segment, classify features and analyze dynamic changes of the skin area of the face of a newborn, and to identify abnormal texture, pigment distribution or blood vessel features; a face key point analysis module is used to mark key anatomical points of the face, calculate three-dimensional space coordinates, analyze the relative position, geometric relationship of the key points and the evolution trend thereof over time; a dynamic modeling module is used to construct a dynamic evolution model of the facial features of the newborn changing with the age, and to associate the time dependence of the growth and development stage with the rare disease features; and a feature fusion unit is used to integrate the output results of the skin analysis module, the face key point analysis module and the dynamic modeling module. The application has the advantage of improving the diagnosis accuracy.

Description

TECHNICAL FIELD

[0001] The application relates to the field of rare disease auxiliary diagnosis, in particular to a neonatal facial feature rare disease auxiliary diagnosis system based on AI image recognition. BACKGROUND

[0002] In the field of neonatal rare disease auxiliary diagnosis, the prior art mainly relies on static facial feature analysis or single modal data, and lacks systematic modeling of the dynamic changes of neonatal facial features with growth and development. Specifically, the traditional auxiliary diagnosis system cannot effectively integrate multi-dimensional features such as skin texture and facial geometric structure, and cannot accurately capture the time-dependent evolution rule of rare disease features with age, resulting in low recognition rate of early atypical symptoms of rare diseases, and easy occurrence of missed diagnosis or misdiagnosis. In view of this, the application provides a neonatal facial feature rare disease auxiliary diagnosis system based on AI image recognition. SUMMARY

[0003] The application aims to provide a neonatal facial feature rare disease auxiliary diagnosis system based on AI image recognition, to solve the problem of insufficient dynamic feature analysis in the prior art of neonatal rare disease auxiliary diagnosis.

[0004] To solve the above technical problems, the application provides the following technical scheme: a neonatal facial feature rare disease auxiliary diagnosis system based on AI image recognition, comprising:

[0005] A skin analysis module, which detects, segments, classifies features and analyzes dynamic changes of the skin area of the neonatal face, and identifies abnormal texture, pigment distribution or blood vessel features;

[0006] A facial key point analysis module, which labels facial key anatomical points and calculates three-dimensional space coordinates, analyzes the relative position, geometric relationship of the key points and the evolution trend thereof with time;

[0007] A dynamic modeling module, which constructs a dynamic evolution model of neonatal facial features with age, and correlates the time dependence of the growth and development stage and the rare disease features;

[0008] A feature fusion unit, which integrates the output results of the skin analysis module, the facial key point analysis module and the dynamic modeling module, and generates a comprehensive feature vector containing static features and dynamic evolution features;

[0009] A comprehensive diagnosis unit, which combines clinical information, multi-modal feature models and growth and development standards, cross- validates the fused feature vector, and generates a diagnosis report containing disease suggestions, feature matching basis and follow-up plans.

[0010] Preferably, it further comprises:

[0011] An image acquisition module continuously acquires clinical video streams at different time points after the birth of the newborn, and the clinical video streams contain a plurality of continuous images to record dynamic changes of facial features with growth and development.

[0012] An image preprocessing module performs facial region positioning, geometric correction and brightness optimization on each frame of image in the video stream to eliminate the influence of shooting angle deviation and uneven light on image quality.

[0013] An image recognition model is constructed based on a deep learning framework, and multi-dimensional features of texture, contour and color of the newborn's face are automatically extracted by a convolutional neural network to form a standardized feature expression.

[0014] A feature matching module performs similarity calculation on the real-time extracted facial features and the pre-constructed rare disease facial feature library, and outputs the disease type with the highest matching degree and the confidence score.

[0015] Preferably, the skin analysis module comprises:

[0016] A skin detection unit establishes a skin color model in a YCbCr color space, fits the color distribution of normal baby skin through a Gaussian mixture clustering algorithm, automatically determines a skin color segmentation threshold by combining an Otsu threshold method, locates a facial skin region and excludes non-skin pixels;

[0017] A skin segmentation unit refines the boundary of the detected skin region by using a GrabCut interactive segmentation algorithm, optimizes the probability distribution of foreground / background pixels through iteration, and introduces a conditional random field model to optimize the continuity of the segmentation boundary, thereby ensuring complete extraction of the skin lesion region.

[0018] A skin classification unit constructs a skin feature classification network based on a DenseNet architecture, inputs the segmented skin region image, and outputs the classification results of skin phenotypes such as pigmentation, vascular malformation and texture abnormality, while analyzing the change rate of each phenotype at different time points.

[0019] A region analysis unit quantifies the spatial and temporal features of skin lesions, and judges whether the skin phenotype evolution rule related to rare diseases is met by comparing the growth curve of normal skin features.

[0020] A positioning auxiliary unit delimits the anatomical sub-regions of the eye, nose and mouth based on the coordinates of the eye corner, nose wing and mouth corner output by the facial key point analysis module, analyzes the skin features of each anatomical sub-region, and improves the detection accuracy of local abnormalities.

[0021] Preferably, the facial key point analysis module comprises:

[0022] The key point labeling unit automatically detects and labels several key points of the face based on a cascaded convolutional neural network to form a two-dimensional coordinate set reflecting the face shape.

[0023] The noise filtering unit processes the key point coordinates of different frames of the same newborn using a DBSCAN density clustering algorithm, identifies and eliminates abnormal displacement points caused by crying expressions, performs mean filtering on the coordinate sequence through a time-space sliding window, and improves the stability of the key point trajectory.

[0024] The three-dimensional reconstruction unit reconstructs the three-dimensional space coordinates of the key points by triangulation method using multi-view videos at the same time point , constructs a three-dimensional geometric model of the face, and is used for analyzing the three-dimensional structural features of the nasal ridge height and facial symmetry;

[0025] The trend analysis unit calculates the Euclidean distance change, angle change, and relative position offset of the key points at adjacent time points, generates a dynamic evolution trajectory of the facial key points, and identifies the evolution mode of rare disease-related facial feature proportion abnormalities or contour deformities.

[0026] The time series modeling unit models the three-dimensional coordinate sequence of the key points using a long short-term memory network, learns the time dependence of the key point trajectory in the normal growth process, and establishes an abnormality detection model of the pathological evolution trajectory.

[0027] Preferably, the dynamic modeling module comprises:

[0028] The growth stage division unit divides the evolution process after birth into three stages: early, middle, and late, according to the development speed of the facial features of the newborn, and each stage corresponds to different feature change sensitivity and pathological feature manifestation probability.

[0029] The dynamic feature extraction unit calculates the change amplitude, fluctuation frequency, and abnormal feature occurrence frequency of the skin features and key point features for each growth stage, and generates a quantitative index reflecting the dynamic characteristics of the features.

[0030] The standard model library stores the facial feature evolution standards of known rare diseases at each growth stage, including the normal fluctuation range of feature parameters, the typical appearance time of pathological features, and the evolution curve, which is used for real-time detection and stage comparison of data.

[0031] Preferably, the feature fusion unit comprises:

[0032] The static feature fusion module serially combines the skin texture features and key point three-dimensional structure features in a single frame image, or performs weighted fusion through an attention mechanism to form a multi-dimensional static feature vector containing surface features and geometric features.

[0033] The dynamic feature fusion module integrates feature change data at different time points to generate dynamic evolution features in combination with growth stage weights. The growth stages are divided into early, middle and late stages according to the age of the newborn, and the growth stage weights are automatically adjusted according to the age of the newborn to highlight the key diagnostic features of each growth stage.

[0034] The normalization processing module normalizes the fused static features and dynamic features, uses the Z-score normalization method to unify the feature dimensions and scales, and eliminates the scale differences of different feature types to provide input data with consistent formats for subsequent diagnostic models.

[0035] Preferably, the comprehensive diagnostic unit comprises:

[0036] The weight distribution module dynamically adjusts the weight coefficients of skin features, key point features and their dynamic evolution features based on feature types and growth stages through a gradient descent algorithm, and gives priority to features with high correlation with the current age;

[0037] The multi-modal comparison module uses support vector machine (SVM) and random forest algorithm to classify static feature vectors and identify the structural / texture matching degree with known rare diseases;

[0038] Meanwhile, the dynamic time warping technology is used to match the real-time feature evolution trajectory with the pathological curve in the standard model library to calculate the similarity distance of the time series, realizing double verification of static structure and dynamic evolution;

[0039] In the dynamic time warping technology, the distance between two time series and is calculated First, a local distance matrix is defined, and then a cumulative distance matrix is solved by dynamic programming:

[0040] ;

[0041] The final similarity distance ;

[0042] Wherein, represents the real-time dynamic evolution time series of the facial features of the newborn, represents the th feature data point in the real-time time series , represents the index of the data point in the sequence, represents the pre-constructed standard evolution time series of rare disease facial features, represents the th feature data point in the standard time series , represents the index of the data point in the standard sequence, represents the real-time time sequence with the standard time sequence similarity distance calculated by the dynamic time warping technique;

[0043] represents the real-time time sequence th data point in the th data point in the th data point in the th data point in the th data point in the local distance between the represents the cumulative distance matrix element in the dynamic time warping process, which is used to represent the cumulative distance of the optimal matching path from the starting point of the real-time sequence th data point, from the starting point of the standard sequence th data point; th data point; th data point;

[0044] represents the cumulative distance of the previous real-time data point and the current standard data point, represents the cumulative distance of the current real-time data point and the previous standard data point, represents the cumulative distance of the previous real-time data point and the previous standard data point, represents the final element of the cumulative distance matrix, that is, the total cumulative distance of the real-time time sequence with the standard time sequence after complete matching;

[0045] The report generation module generates a visual diagnostic report according to the comparison result, and the content includes: a feature matching heat map, a three-dimensional face model, a dynamic evolution curve, a clinical suggestion, and a contribution degree of each feature to the diagnostic result.

[0046] Compared with the prior art, the beneficial effects of the present application are:

[0047] 1、The dynamic modeling module of the present application constructs a dynamic evolution model of the facial features of the newborn as the age changes, and the feature fusion unit integrates the multi-module outputs such as skin analysis and key point analysis to form a comprehensive vector containing static and dynamic features, realizes the accurate capture of the time dependence of rare disease features, significantly improves the diagnostic accuracy, and effectively solves the problem of insufficient dynamic feature analysis in the prior art.

[0048] 2、The skin analysis module of the application delimits the anatomical sub-region by combining the face key point coordinates with the positioning auxiliary unit, analyzes the local skin features such as the eye and mouth, and the face key point analysis module improves the stability of the key point trajectory through three-dimensional reconstruction and noise filtering technology, and the two work together to accurately identify subtle lesions such as wide eye distance and local pigment abnormalities, and further strengthen the capture ability of rare disease characteristics.

[0049] 3、The application also generates a diagnosis report containing a follow-up plan by combining clinical information and growth and development standards through a comprehensive diagnosis unit, matches real-time features with standard pathological curves through dynamic time warping technology, not only provides disease suggestions, but also automatically adjusts the follow-up frequency and examination items according to the age, realizes early intervention and disease course monitoring of rare diseases. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 It is a schematic diagram of the system framework of the application. DETAILED DESCRIPTION

[0051] In order to facilitate those skilled in the art to understand the technical scheme of the application, the technical scheme of the application will be further described in conjunction with the drawings of the specification.

[0052] Example 1, as shown, the application provides a neonatal facial feature rare disease auxiliary diagnosis system based on AI image recognition, comprising: Figure 1 An image acquisition module is configured with a high-definition video acquisition device to continuously acquire clinical video streams at different time points after the birth of a neonate, and synchronously acquires clinical information including basic physiological indicators, family history of genetic diseases, and risk factors during pregnancy. The clinical video stream contains multiple consecutive images to record the dynamic changes of facial features with growth and development;

[0053] An image preprocessing module performs face region positioning, geometric correction and brightness optimization on each frame of image in the video stream, eliminating the influence of shooting angle deviation and uneven light on image quality;

[0054] An image recognition model is built based on a deep learning framework, which automatically extracts multi-dimensional features such as texture, contour and color of the neonatal face through a convolutional neural network, forming a standardized feature expression;

[0055] A feature matching module performs similarity calculation on the real-time extracted facial features and the pre-constructed rare disease facial feature library, and outputs the disease type with the highest matching degree and the confidence score;

[0056] A skin analysis module for the neonatal facial skin area, completes detection, segmentation, feature classification and dynamic change analysis, and identifies abnormal texture, pigment distribution or vascular features;

[0057] ​

[0058] a face key point analysis module for labeling key anatomical points of the face and calculating three-dimensional spatial coordinates, analyzing the relative positions, geometric relationships of the key points, and their evolution trends over time;

[0059] a dynamic modeling module for constructing a dynamic evolution model of the facial features of the newborn as a function of the age, correlating the time dependence of the growth and development stage with the rare disease characteristics;

[0060] a feature fusion unit for integrating the output results of the skin analysis module, the face key point analysis module, and the dynamic modeling module to generate a comprehensive feature vector containing static features and dynamic evolution features;

[0061] a comprehensive diagnosis unit for cross-validating the fused feature vector in combination with clinical information, multi-modal feature models, and growth and development standards to generate a diagnosis report containing disease suggestions, feature matching bases, and follow-up plans.

[0062] In an embodiment of the present application, the image preprocessing module comprises:

[0063] a camera calibration unit for obtaining camera internal parameters (focal length, principal point coordinates, distortion coefficients) and external parameters (rotation matrix, translation vector) by shooting a checkerboard calibration object, establishing a conversion relationship between the camera coordinate system and the image coordinate system, and correcting image distortion caused by differences in camera position and angle;

[0064] a face correction unit for calculating a homography transformation matrix based on the camera calibration results , converting the face images from side, top, or bottom views to a standard front view through perspective transformation, and eliminating the influence of geometric distortion on feature analysis;

[0065] the homography transformation matrix is calculated by the following formula:

[0066] ;

[0067] wherein, is the pixel coordinate in the image coordinate system, is the coordinate in the world coordinate system;

[0068] a brightness enhancement unit for performing local contrast adjustment on the corrected image using adaptive histogram equalization technology, enhancing the texture details and color differences of the baby's skin by dividing the image sub-blocks and performing equalization processing respectively, and improving the sensitivity of subsequent feature detection;

[0069] for the pixel value in the image sub-block, the pixel value after adaptive histogram equalization is calculated by the following formula:

[0070] ;

[0071] wherein, is a histogram statistic of pixel values is a total number of gray levels of the image.

[0072] In an embodiment of the present application, the training method of the image recognition model comprises:

[0073] In the data labeling stage, the clinical video dataset containing different day-old newborns is labeled, and the labeled content includes the key point coordinates of the facial region contour, the eye corner, the nose tip, the mouth corner, the position and range of the skin lesion area (such as pigmented spots, hemangioma), and the like;

[0074] In the face segmentation stage, a U-Net semantic segmentation network architecture is used to train a face segmentation model, and a binary mask (0 for the face region and 1 for the background) is output after inputting the original image, so as to separate the face region from the non-face region (such as hair, clothes, medical equipment), and generate a pure face image dataset;

[0075] In the U-Net network training process, a cross-entropy loss function is used for optimization, and the formula is as follows:

[0076] ;

[0077] wherein, is the number of samples, is the number of categories, is the true label of the sample belongs to the category , and is the probability that the sample predicted by the model belongs to the category ;

[0078] In the feature extraction stage, ResNet-50 is used as the backbone network to extract the deep semantic features of the face image, and a feature pyramid network is used to fuse feature maps of different levels to capture multi-scale features and generate a feature vector containing geometric features (proportion of five organs), texture features (skin roughness), and color features (skin color uniformity);

[0079] In the ResNet-50 network, the output feature map of a certain layer is calculated by the following formula:

[0080] ;

[0081] wherein, is a weight matrix, is the feature map of the previous layer,​ is a bias vector, is an activation function;

[0082] Model training phase, cross-entropy loss function and triplet loss function Jointly optimize network parameters, through end-to-end training to enable the model to accurately distinguish normal facial features from abnormal feature patterns related to rare diseases;

[0083] Triplet loss function The formula is as follows:

[0084] ;

[0085] Wherein, is an anchor sample, is a positive sample, is a negative sample, is a feature extraction function, is a margin parameter, The formula represents the maximum value of and 0, which is used in the triplet loss function to ensure that the loss value is non-negative. Specifically, when calculating the triplet loss, a difference value is obtained after adding the margin parameter The result If is negative, it means that the distance between the anchor sample and the positive sample is already small enough, and is less than the distance between the anchor sample and the negative sample, at which time the loss is 0.

[0086] Only when is positive, it will be involved in the optimization of network parameters as part of the loss, so as to make the model learn more discriminative feature expressions, so that the feature vectors of the same category samples are closer, and the feature vectors of different category samples are farther away.

[0087] In an embodiment of the present application, the skin analysis module comprises:

[0088] The skin detection unit establishes a skin color model in the YCbCr color space, fits the color distribution of normal baby skin through Gaussian mixture clustering algorithm, automatically determines the skin color segmentation threshold combined with Otsu threshold method, locates the facial skin area and excludes non-skin pixels (such as lips, hair);

[0089] In Gaussian mixture clustering, the probability of a pixel point belonging to the th Gaussian distribution is calculated by the following formula:

[0090] ;

[0091] wherein, is the weight of the th Gaussian distribution, is the Gaussian probability density function with mean and covariance , is the number of Gaussian distributions;

[0092] The skin segmentation unit adopts the GrabCut interactive segmentation algorithm to refine the boundary of the detected skin region, optimizes the probability distribution of the foreground / background pixels through iteration, and introduces a conditional random field model to optimize the continuity of the segmentation boundary, thereby ensuring the complete extraction of the skin lesion region.

[0093] In the CRF model, the energy function is calculated by the following formula:

[0094] ;

[0095] wherein, is a unary potential function describing the likelihood of a pixel belonging to a certain class, is a binary potential function describing the relationship between pixels and ;

[0096] The skin classification unit constructs a skin feature classification network based on the DenseNet architecture, inputs the segmented skin region image, and outputs the classification results of pigmentation, vascular malformation, and texture abnormality skin phenotypes, while analyzing the change rate of each phenotype at different time points.

[0097] In the classification layer of the DenseNet network, the output class probability is calculated by the following formula:

[0098] ;

[0099] wherein, and are the weights and biases of the fully connected layer, is the input feature of the fully connected layer;

[0100] The region analysis unit quantifies the spatial features (position coordinates, area size, shape complexity) and temporal features (first appearance time, duration, change amplitude) of the skin lesions, and judges whether it conforms to the skin phenotype evolution rule related to rare diseases by comparing the growth curve of normal skin features.

[0101] For the area of the skin lesion region, the change rate at time ​ It can be calculated using the following formula:

[0102] ;

[0103] in, For time intervals;

[0104] The positioning assistance unit, based on the coordinates of the corners of the eyes, nose, and mouth output by the facial key point analysis module, delineates anatomical sub-regions around the eyes (from the inner canthus to the outer canthus), the nose (from the bridge of the nose to the nasal ala), and the mouth (from the corner of the mouth to the vermilion border of the lips). It then analyzes the skin characteristics of each anatomical sub-region in a targeted manner to improve the detection accuracy of local abnormalities (such as excessively wide eye spacing accompanied by abnormal eyelid pigmentation).

[0105] In an embodiment of the present invention, the facial key point analysis module includes:

[0106] The key point annotation unit, based on a cascaded convolutional neural network, automatically detects and annotates several key points on the face (including anatomical landmarks such as the brow peak, corner of the eye, tip of the nose, corner of the mouth, and mandibular border), forming a two-dimensional coordinate set that reflects the facial morphology.

[0107] The noise filtering unit uses the DBSCAN density clustering algorithm to process the key point coordinates of different frames of the same newborn, identifies and removes abnormal displacement points caused by crying expressions, and performs mean filtering on the coordinate sequence through a spatiotemporal sliding window to improve the stability of the key point trajectory.

[0108] In the DBSCAN algorithm, for key point coordinates... Its core distance Calculated using the following formula:

[0109] ;

[0110] in, For point of The set of points in the neighborhood, The minimum number of points required for the core point. For point To the Nearest neighbor The distance;

[0111] The 3D reconstruction unit, combining multi-view video from the same time point, uses triangulation to calculate the 3D spatial coordinates of key points. A three-dimensional geometric model of the face is constructed to analyze the three-dimensional structural features of the bridge of the nose and facial symmetry.

[0112] Coordinates of key points from two perspectives and , three-dimensional coordinates are calculated by triangulation , the formula is as follows:

[0113] ;

[0114] wherein, and respectively, pseudo-inverses of the projection matrix of the two perspective cameras, is a homography matrix between the two perspectives;

[0115] The trend analysis unit calculates the Euclidean distance change, angle change (such as the angle between the double-eye connecting line and the horizontal line), and relative position offset (such as the displacement of the nose tip relative to the eyebrow center) of the key points at adjacent time points, generates the dynamic evolution track of the facial key points, and identifies the evolution mode of the rare disease related facial feature ratio abnormality (such as low ear position, short philtrum) or contour deformity (such as small chin).

[0116] For the key point coordinates and of two time points and , the Euclidean distance change is calculated by the following formula:

[0117] ;

[0118] The time series modeling unit models the key point three-dimensional coordinate sequence by using a long short-term memory network, learns the time dependence of the key point track in the normal growth process (such as the normal rate of the nasal ridge height with the increase of the age), and establishes an abnormal detection model of the pathological evolution track.

[0119] In the LSTM network, the update formula of the memory cell is as follows:

[0120] ;

[0121] wherein, is the forgetting gate output, is the input gate output, is the candidate memory cell, is the memory cell at the previous time.

[0122] In the embodiments of the present application, the dynamic modeling module comprises:

[0123] The growth stage division unit divides the evolution process after birth into three stages according to the development speed of the facial features of the newborn: early stage (0-7 days, facial edema subsides), middle stage (8-28 days, facial feature shape stabilizes), and late stage (29 days and later, contour feature appears), each stage corresponds to different feature change sensitivity and pathological feature performance probability.

[0124] The dynamic feature extraction unit calculates the change amplitude (difference between current value and previous stage average), fluctuation frequency (number of times the feature value exceeds the normal range) and abnormal feature occurrence frequency (such as persistent skin vascular abnormalities) of skin features (such as pigment spot area) and key point features (such as interocular distance) for each growth stage, and generates quantitative indicators reflecting the dynamics of the features;

[0125] For feature values The change amplitude in a certain growth stage is calculated by the following formula:

[0126] ;

[0127] Wherein, is the feature average of the previous growth stage;

[0128] The standard model library stores the facial feature evolution standards of known rare diseases (such as 21-trisomy syndrome, Noonan syndrome) at each growth stage, including the normal fluctuation range of feature parameters (based on healthy newborn cohort data), the typical appearance time of pathological features (such as the ear position low feature of a certain syndrome usually appears in the middle stage) and the evolution curve (reference trajectory of feature value change with age), which is used for real-time detection of the staged comparison of data.

[0129] In an embodiment of the present application, the feature fusion unit includes:

[0130] The static feature fusion module concatenates or weightedly fuses the skin texture features (such as roughness, pigment distribution vector) and key point three-dimensional structure features (such as facial distance proportion vector) in a single frame image through attention mechanism, forming a multi-dimensional static feature vector containing surface features and geometric features;

[0131] When fused through attention mechanism, the fused feature vector is calculated by the following formula:

[0132] ;

[0133] Wherein, is the th feature vector, is the weight calculated by attention mechanism, ;

[0134] The dynamic feature fusion module integrates feature change data at different time points (such as the weekly growth rate of skin lesion area and the monthly change of key point spatial location), and combines growth stage weights (giving higher weight to skin edema-related features in the early stage and focusing on contour feature weights in the later stage) to generate dynamic evolution features. The growth stage weights are automatically adjusted according to the newborn's age to highlight the key diagnostic features of each growth stage.

[0135] For dynamic characteristic changes Dynamic features after fusion Calculated using the following formula:

[0136] ;

[0137] in, Indicates the growth stage. This represents the total number of growth stages. For the first Weights for each growth stage For the first The dynamic characteristics of each growth stage;

[0138] The normalization module standardizes the fused static and dynamic features, and uses the Z-score normalization method to unify the feature dimensions and units, eliminate scale differences between different feature types, and provide input data with a consistent format for subsequent diagnostic models.

[0139] The Z-score normalization formula is as follows:

[0140] ;

[0141] in, These are the original eigenvalues. The characteristic mean, The characteristic standard deviation is denoted as .

[0142] In an embodiment of the present invention, the comprehensive diagnostic unit includes:

[0143] The weight allocation module dynamically adjusts the weight coefficients of skin features, key point features, and their dynamic evolution features based on feature type (static features reflect the immediate state, while dynamic features reflect the evolution trend) and growth stage (early skin feature weight > structural features, and late stage weight > structural features) through gradient descent algorithm, giving priority to features that are highly correlated with the current age.

[0144] For weighting coefficients The formula is updated using the gradient descent algorithm as follows:

[0145] ;

[0146] in, For learning rate, The loss function;

[0147] The multimodal comparison module uses support vector machine (SVM) and random forest algorithms to classify static feature vectors and identify the structural / texture matching degree with known rare diseases;

[0148] Simultaneously, dynamic time warping technology is used to match real-time feature evolution trajectories with pathological curves in the standard model library, calculate the similarity distance of time series, and achieve dual verification of static structure and dynamic evolution.

[0149] In the SVM algorithm, for the linearly separable case, the decision function is:

[0150] ;

[0151] in, For the weight vector, For the input feature vector, For bias, It is a symbolic function;

[0152] For the random forest algorithm, the final prediction result It is obtained by voting or averaging the prediction results of multiple decision trees. For classification problems, the voting method is used:

[0153] ;

[0154] in, It is the first The prediction results of the decision tree It is the number of decision trees. Indicates taking the mode;

[0155] In dynamic time warping, two time series are calculated. and distance First, define the local distance matrix. Then, the cumulative distance matrix is ​​solved using dynamic programming. :

[0156] ;

[0157] Final similarity distance ;

[0158] in, This represents a real-time time series of dynamic evolution of newborn facial features. Represents real-time time series The first in Each feature data point denotes the index of the data point in the sequence, denotes a pre-constructed time series of rare disease facial feature standards, denotes a standard time series of the th feature data point, denotes the index of the data point in the standard sequence, denotes a real-time time series of the th data point in the standard time series ;

[0159] denotes a local distance between the th data point in the real-time time series and the th data point in the standard time series ; denotes a cumulative distance matrix element in the dynamic time warping process, used to represent the cumulative distance of the optimal matching path from the start point of the real-time sequence to the th data point, from the start point of the standard sequence to the th data point;

[0160] denotes the cumulative distance of the previous real-time data point and the current standard data point, denotes the cumulative distance of the current real-time data point and the previous standard data point, denotes the cumulative distance of the previous real-time data point and the previous standard data point, denotes the final element of the cumulative distance matrix, i.e., the total cumulative distance of the real-time time series and the standard time series after complete matching;

[0161] a report generation module generates a visual diagnostic report based on the comparison results, including: feature matching heat map (annotating abnormal facial feature positions), three-dimensional facial model (displaying key point spatial distribution abnormalities), dynamic evolution curve (comparing real-time data with standard pathological curve), clinical suggestions (follow-up frequency, further examination items), and labeling the contribution of each feature to the diagnostic result;

[0162] the calculation of feature contribution in the report is based on the score of feature matching and the total score of all features :

[0163] ​ ;

[0164] In the embodiments of the application, further comprising:

[0165] The model optimization module periodically imports the clinical data of newly diagnosed cases in the neonatal department of the hospital, updates the rare disease feature model library and dynamic evolution standard through the transfer learning technology, adjusts the feature weight and the comparison algorithm for the newly found rare disease phenotype, and improves the generalization recognition ability of the system to the rare disease.

[0166] The visualization module generates a dynamic feature analysis report containing a time axis, supports doctors to view the facial images of the newborn at different ages, the skin feature segmentation results, the three-dimensional coordinate changes of the key points, and compares the normal development curve with the pathological evolution mode by sliding the time axis.

[0167] The embodiments of the application are preferred embodiments, but are not limited thereto, and those skilled in the art can easily understand the spirit of the application according to the above embodiments, and make different inferences and changes, as long as they do not deviate from the spirit of the application, they are within the protection scope of the application.

Claims

1. An AI image recognition-based neonatal facial feature rare disease auxiliary diagnosis system, characterized in that, Comprise: Skin analysis module, for the facial skin area of newborn, complete detection, segmentation, feature classification and dynamic change analysis, identify abnormal texture, pigment distribution or vascular features; Facial key point analysis module, label facial key anatomical points and calculate three-dimensional spatial coordinates, analyze the relative position, geometric relationship of key points and its evolution trend over time; Dynamic modeling module, build the dynamic evolution model of newborn facial features with age, correlate the time dependence of growth and development stage and rare disease characteristics; Feature fusion unit, integrate the output results of skin analysis module, facial key point analysis module and dynamic modeling module, generate comprehensive feature vector containing static features and dynamic evolution features; Comprehensive diagnosis unit, combined with clinical information, multi-modal feature model and growth and development standard, cross-verify the fused feature vector, generate diagnosis report containing disease suggestion, feature matching basis and follow-up plan; The dynamic modeling module comprises: Growth stage division unit, according to the development speed of newborn facial features, the evolution process after birth is divided into three stages: early, middle and late, each stage corresponds to different feature change sensitivity and pathological feature performance probability; Dynamic feature extraction unit, for each growth stage, calculate the change amplitude, fluctuation frequency and abnormal feature appearance frequency of skin features and key point features, generate quantitative indicators reflecting feature dynamics; Standard model library, store the facial feature evolution standard of known rare diseases in each growth stage, including the normal fluctuation range of feature parameters, the typical appearance time of pathological features and evolution curve, used for real-time detection of phased comparison of data; The feature fusion unit comprises: Static feature fusion module, combine skin texture features and key point three-dimensional structure features in single frame image in series, or weighted fusion through attention mechanism, form multi-dimensional static feature vector containing surface features and geometric features; Dynamic feature fusion module, integrate feature change data at different time points, generate dynamic evolution features combined with growth stage weight, the growth stage is divided into early and late according to the age of newborn, the growth stage weight is automatically adjusted according to the age of newborn, highlighting the key diagnostic features of each growth stage; Normalization processing module, standardize the fused static features and dynamic features, use Z-score normalization method to unify feature dimension and dimension, eliminate the scale difference of different feature types, provide input data with consistent format for subsequent diagnosis model.

2. The AI image recognition-based neonatal facial feature rare disease auxiliary diagnosis system according to claim 1, characterized in that, Also include: Image acquisition module, continuously collect clinical video stream of newborn at different time points after birth, the clinical video stream contains multiple consecutive images to record the dynamic change of facial features with growth and development; Image preprocessing module, perform face region positioning, geometric correction and brightness optimization on each frame of image in video stream, eliminate the influence of shooting angle deviation and uneven light on image quality; Image recognition model, based on deep learning framework, automatically extract multi-dimensional features of texture, contour and color of newborn face through convolutional neural network, form standardized feature expression; The feature matching module performs similarity calculation on the real-time extracted facial features and the pre-constructed rare disease facial feature library, and outputs the disease type with the highest matching degree and a confidence score.

3. The AI image recognition-based neonatal facial feature rare disease auxiliary diagnosis system according to claim 1, characterized in that, The skin analysis module includes: The skin detection unit establishes a skin color model in the YCbCr color space, fits the color distribution of normal baby skin through a Gaussian mixture clustering algorithm, automatically determines the skin color segmentation threshold by combining the Otsu threshold method, locates the facial skin area and excludes non-skin pixels; The skin segmentation unit uses the GrabCut interactive segmentation algorithm to refine the boundary of the detected skin area, optimizes the probability distribution of foreground / background pixels through iteration, and introduces a conditional random field model to optimize the continuity of the segmentation boundary, ensuring the complete extraction of the skin lesion area; The skin classification unit constructs a skin feature classification network based on the DenseNet architecture, inputs the segmented skin area image, and outputs the classification results of pigmentation, vascular malformation, and texture abnormal skin phenotypes, while analyzing the change rate of each phenotype at different time points; The region analysis unit quantifies the spatial and temporal features of skin lesions, and judges whether it conforms to the skin phenotype evolution rule related to rare diseases by comparing the growth curve of normal skin features; The positioning assistance unit draws the anatomical sub-regions of the eye, nose, and mouth based on the coordinates of the eye corner, nose wing, and mouth corner output by the facial key point analysis module, analyzes the skin features of each anatomical sub-region, and improves the detection accuracy of local abnormalities.

4. The AI image recognition-based neonatal facial feature rare disease auxiliary diagnosis system according to claim 3, characterized in that, The facial key point analysis module includes: The key point labeling unit automatically detects and labels several key points of the face based on a cascade convolutional neural network, forming a two-dimensional coordinate set reflecting the facial morphology; The noise filtering unit processes the key point coordinates of different frames of the same newborn using the DBSCAN density clustering algorithm, identifies and eliminates abnormal displacement points caused by crying expressions, and performs mean filtering on the coordinate sequence through a spatio-temporal sliding window to improve the stability of the key point trajectory; The three-dimensional reconstruction unit combines multiple perspective videos at the same time point, calculates the three-dimensional spatial coordinates (X, Y, Z) of the key points using the triangulation method, constructs a three-dimensional geometric model of the face, and analyzes the three-dimensional structural features of the nose bridge height and facial symmetry; The trend analysis unit calculates the Euclidean distance change, angle change, and relative position offset of the key points at adjacent time points, generates the dynamic evolution trajectory of the facial key points, and identifies the evolution pattern of the facial feature proportion abnormality or contour deformity related to rare diseases; The time series modeling unit models the three-dimensional coordinate sequence of the key points using a long short-term memory network, learns the time dependence of the key point trajectory in the normal growth process, and establishes an abnormality detection model for the pathological evolution trajectory.

5. The AI image recognition-based neonatal facial feature rare disease auxiliary diagnosis system according to claim 1, characterized in that, The comprehensive diagnosis unit includes: The weight allocation module dynamically adjusts the weight coefficients of the skin features, key point features, and their dynamic evolution features based on the feature type and growth stage through a gradient descent algorithm, and gives priority to features with high relevance to the current age; The multi-modal comparison module uses support vector machine (SVM) and random forest algorithms to classify static feature vectors and identify the structural / texture matching degree with known rare diseases; Meanwhile, the dynamic time warping technique is used to match the real-time feature evolution trajectory with the pathological curve in the standard model library, to calculate the similarity distance of the time series, and to realize the dual verification of static structure and dynamic evolution. In dynamic time warping, two time series G = [g1, g2, ..., g] are calculated. m ] and U=[u1,u2,…,u n When the distance D is equal to the distance ], first define the local distance matrix d(i,j)=|g i -u j Then, the cumulative distance matrix DTW(i,j) is solved using dynamic programming: The final similarity distance D = DTW(m, n); wherein G represents a real-time collected neonatal facial feature dynamic evolution time series, g i represents the i-th feature data point in the real-time time series G, i represents the index of the data point in the sequence, U represents a pre-constructed standard evolution time series of rare disease facial features, u j represents the j-th feature data point in the standard time series U, j represents the index of the data point in the standard sequence, D represents the similarity distance calculated by the dynamic time warping technique between the real-time time series G and the standard time series U; d(i,j) denotes the local distance between the ith data point g i and the jth data point u j of the standard time series U, DTW(i,j) denotes the cumulative distance matrix element in the dynamic time warping process, which is used to represent the cumulative distance of the optimal matching path from the starting point g1 to the ith data point of the real-time series, and from the starting point u1 to the jth data point of the standard series. DTW(i-1, j) represents the cumulative distance between the previous real-time data point and the current standard data point, DTW(i, j-1) represents the cumulative distance between the current real-time data point and the previous standard data point, DTW(i-1, j-1) represents the cumulative distance between the previous real-time data point and the previous standard data point, and DTW(m, n) represents the final element of the cumulative distance matrix, that is, the total cumulative distance after the complete matching of the real-time time series G and the standard time series U. The report generation module generates a visual diagnostic report according to the comparison result, including: feature matching heat map, three-dimensional face model, dynamic evolution curve, clinical suggestion, and labeling the contribution of each feature to the diagnostic result.

Images