Autism assessment method, assessment device, electronic device and storage medium

Abstract

The present application discloses an autism assessment method, an autism assessment device, an electronic device and a computer storage medium, the autism assessment method comprising: performing key point recognition on a facial image of a user to be assessed to obtain coordinate information of each key point; determining the symmetry feature of the face of the user to be assessed based on the coordinate information, the symmetry feature is used to characterize the symmetry of the left and right sides of the face of the user to be assessed; inputting the symmetry feature into a pre-trained assessment model to obtain an assessment result of the user to be assessed. The method can assist professional medical personnel in early screening of ASD.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular relates to an autism assessment method, an autism assessment device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Autism Spectrum Disorder (ASD), also known as autism spectrum disorder or autism spectrum disorder, is a series of complex neurodevelopmental disorders that are more common in children. Its symptoms include abnormal language ability, abnormal communication ability, narrow interests and stubborn behavior patterns. Among this spectrum disorder, childhood autism is the most serious type of childhood mental illness.

[0003] However, the causes of ASD are complex, and there are currently no effective drugs or treatments. Therefore, early screening for ASD is particularly important. Screening for ASD often relies on the diagnosis and identification of professional medical personnel. This method is not only time-consuming, but also due to the shortage of relevant medical personnel, most ASD patients cannot be screened in the early stages. Summary of the invention

[0004] The present application provides an autism assessment method, an autism assessment device, an electronic device and a computer-readable storage medium, which can assist medical personnel in achieving early screening of ASD.

[0005] In a first aspect, the present application provides a method for assessing autism, comprising:

[0006] Perform key point recognition on the facial image of the user to be evaluated to obtain the coordinate information of each key point;

[0007] Determine the symmetry feature of the face of the user to be evaluated based on the coordinate information, where the symmetry feature is used to characterize the symmetry between the left side and the right side of the face of the user to be evaluated;

[0008] The above symmetry features are input into a pre-trained evaluation model for processing to obtain the evaluation results of the above-mentioned user to be evaluated.

[0009] In a second aspect, the present application provides an autism assessment device, comprising:

[0010] A recognition module is used to identify key points of the face image of the user to be evaluated and obtain the coordinate information of each key point;

[0011] A determination module, used to determine the symmetry feature of the face of the user to be evaluated based on the coordinate information, wherein the symmetry feature is used to characterize the symmetry between the left side and the right side of the face of the user to be evaluated;

[0012] The evaluation module is used to input the above symmetry features into a pre-trained evaluation model for processing to obtain the evaluation results of the above-mentioned user to be evaluated.

[0013] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method of the first aspect are implemented.

[0015] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it implements the steps of the method of the first aspect.

[0016] Compared with the prior art, the beneficial effects of the present application are as follows: by identifying the key points of the user to be evaluated in the face image, the coordinate information of each key point is obtained; based on the coordinate information, the symmetry features of the face of the user to be evaluated can be determined, and the symmetry features can be processed using a pre-trained evaluation model to obtain the evaluation result of the user to be evaluated, and the evaluation result can provide auxiliary information for professional medical personnel, allowing medical personnel to perform early ASD screening for more users, so that users who actually suffer from ASD can be screened out at an early stage and receive timely assistance and intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 It is a flowchart of an autism assessment method provided in an embodiment of the present application;

[0019] Figure 2 This is a schematic diagram of key points of a face provided in an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a first connecting line, a second connecting line, and a third connecting line in a face image provided by an embodiment of the present application;

[0021] Figure 4is a schematic diagram of a region group in a face image provided by an embodiment of the present application;

[0022] Figure 5 is a statistical diagram of the differences between ASD children and non-ASD children in the six symmetry features provided in the embodiments of the present application;

[0023] Figure 6 is a schematic diagram of the structure of an autism assessment device provided in an embodiment of the present application;

[0024] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] The autism assessment method provided in the embodiment of the present application can be applied to smart cars equipped with controllers and other electronic devices, such as laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other electronic devices. The embodiment of the present application does not impose any restrictions on the specific types of electronic devices.

[0027] Since 1943, ASD has gradually been correctly recognized by people and has attracted widespread attention. With the improvement of people's awareness, the United Nations General Assembly passed a resolution in December 2007, and from 2008, April 2 of each year will be positioned as "World Autism Awareness Day" to improve people's research and diagnosis of autism. So far, there has been no breakthrough in the treatment of ASD, and no effective therapeutic drugs or treatment methods have been found. For this type of disease, early screening and early intervention are particularly important. However, the current screening of ASD mainly relies on the diagnosis and identification of professional medical personnel, which is time-consuming and lacks relevant professionals, reducing the possibility of most patients being screened in the early stage, and also making these patients lose the opportunity for early intervention. In response to this problem, the inventor found through research that the facial asymmetry of ASD children is much higher than that of non-ASD children. That is, by analyzing the facial symmetry of the user, it can provide reference information for medical personnel to diagnose ASD, so as to shorten the screening time of each patient and improve the screening efficiency. Based on this, the present application proposes an autism assessment method, which obtains assessment results based on the analysis and evaluation of the user's facial symmetry to assist medical personnel in early screening of ASD, improve screening efficiency, and allow more ASD patients to have the opportunity to be screened at an early stage and receive timely assistance and intervention.

[0028] In order to illustrate the technical solution proposed in this application, a specific embodiment is provided below for illustration.

[0029] Figure 1 A schematic flow chart of the autism assessment method provided in the present application is shown. As an example but not a limitation, the method can be applied to any of the above-mentioned electronic devices.

[0030] Step 110: perform key point recognition on the facial image of the user to be evaluated to obtain coordinate information of each key point.

[0031] The key point recognition of the face is a key step in the field of face recognition and analysis. It is the premise and breakthrough for solving other face problems such as automatic face recognition, expression analysis, 3D face reconstruction, and 3D animation. Similarly, for the facial symmetry analysis of the user to be evaluated, the key point recognition can also be performed on the face image of the user to be evaluated to obtain the coordinate information of each key point, thus laying the foundation for subsequent analysis.

[0032] Alternatively, key points can be identified by the CHEHRA method. Figure 2 , Figure 2 A schematic diagram of facial key points is shown. After obtaining the diagram, inaccurate key points can be manually adjusted to improve recognition accuracy.

[0033] Step 120: Determine the symmetry features of the face of the user to be evaluated based on the coordinate information.

[0034] The symmetry of the face of the user to be evaluated can be reflected by the symmetry features of the left and right sides of the face of the user to be evaluated, that is, the symmetry features are used to represent the symmetry of the left and right sides of the face of the user to be evaluated. In a face image, the left and right sides of the face can be divided by drawing the facial midline. The facial midline can be determined in a variety of ways, for example, based on the bridge of the nose, or based on the line connecting the pupils of both eyes. After the facial midline is drawn, the facial area to the left of the facial midline is the left side of the face, and the facial area to the right of the facial midline is the right side of the face.

[0035] Step 130: Input the symmetry feature into the pre-trained evaluation model for processing to obtain the evaluation result of the user to be evaluated.

[0036] After obtaining the symmetry feature, the pre-trained evaluation model can be used to process the symmetry feature to obtain the evaluation result of the user to be evaluated. The evaluation result can provide auxiliary information for the judgment of medical personnel, so as to efficiently perform early screening of ASD for the user to be evaluated. Among them, the evaluation model can be trained in a supervised learning manner. After the labeled sample image is processed, the sample symmetry feature can be obtained. After identification by the evaluation model, the training result is obtained. For each sample image, the closer the training result is to the label of the sample image, the higher the evaluation accuracy of the current evaluation model. When the error between the training result and the label of the sample image is less than the preset error threshold, it means that the evaluation model has converged, and the trained evaluation model can be obtained and applied in actual scenarios.

[0037] For the evaluation results, it can be a binary result, that is, yes or no, where yes means that the subject to be evaluated is more likely to suffer from ASD, otherwise it means that the evaluated user is less likely to suffer from ASD. After obtaining the evaluation results, medical personnel can further diagnose the user to be evaluated with a yes evaluation result to obtain a more accurate diagnosis result. In order to be able to give more valuable evaluation results, the evaluation results can be multi-classification results, for example, one classification result corresponds to a percentage interval, that is, the evaluation results can be presented in the form of percentages. The larger the percentage, the greater the possibility that the user to be evaluated suffers from ASD; the smaller the percentage, the smaller the possibility that the user to be evaluated suffers from ASD. In this way, when there are more users to be evaluated, the order of seeing the users to be evaluated can be determined based on the size of the percentage, and priority is given to seeing the users to be evaluated with a larger percentage, so that the users to be evaluated with urgent situations can complete the early screening of ASD as soon as possible, thereby improving the efficiency and timeliness of early screening of ASD.

[0038] The embodiment of the present application obtains the coordinate information of each key point by identifying the key points of the user to be evaluated in the face image; based on the coordinate information, the symmetry features of the face of the user to be evaluated can be determined, and then the symmetry features are processed using a pre-trained evaluation model to obtain the evaluation result of the user to be evaluated. The evaluation result can provide auxiliary information for professional medical personnel, allowing medical personnel to perform early ASD screening for more users, so that users who actually suffer from ASD can be screened out at an early stage and receive timely assistance and intervention.

[0039] In some embodiments, in order to obtain a more accurate symmetry feature, before the above step 110, the following steps are further included:

[0040] Step A1: Obtain an initial face image of the user to be evaluated.

[0041] The initial image of the user to be evaluated can be acquired through professional equipment, such as hardware equipment supporting the evaluation method provided in this application, which may include a camera and a collection room. The user to be evaluated can put his head into the collection room and collect it through the camera. The initial face image collected by this method is the face image used for subsequent recognition (the image size meets the requirements, and the left eye and the right eye are on the same horizontal line). Of course, when the user to be evaluated cannot collect images through the supporting hardware equipment, he can also provide the initial face image independently to improve the convenience of evaluation.

[0042] Step A2: scaling the initial face image according to a preset ratio.

[0043] For different initial facial images provided by different users, when shooting, the distances between the shooting device and the face of the user to be evaluated are different, so the faces in these initial facial images are large and small. In order to facilitate subsequent processing, the sizes of the faces in the images can be unified, and the initial facial images can be scaled to a preset ratio, for example, the distance between the eyes in the initial facial images can be scaled to a fixed value.

[0044] Step A3: performing a rotation operation on the scaled initial face image to obtain a face image in which the left eye and the right eye of the user to be evaluated are on the same horizontal line.

[0045] In addition to the pre-processing of scaling the initial face image, the initial face image may also be rotated to obtain the face image, wherein the rotation operation may be to adjust the left eye and the right eye of the user to be evaluated in the initial face image to the same horizontal line, thereby providing a reference for the subsequent extraction of symmetry features.

[0046] It is understandable that there is no fixed order between the above two operations. The initial face image after the scaling operation and the rotation operation are performed is the face image for face recognition. By performing the above two operations on the initial face image, the accuracy of subsequent key point recognition and symmetry feature extraction can be improved.

[0047] In some embodiments, the symmetry of the left and right sides of the face of the user to be evaluated may be characterized by an angle feature, that is, the symmetry feature may include an angle feature. The above step 120 specifically includes:

[0048] Step 121a: Calculate a first angle between a first line connecting the eyebrows on the left side of the face and the eyebrows on the right side of the face in the face image and a horizontal line based on the coordinate information.

[0049] Step 122a, and / or, calculating a second angle between a second connecting line between the nose on the left side of the face and the nose on the right side of the face in the facial image and the horizontal line based on the coordinate information.

[0050] Step 123a, and / or, calculating a third angle between a third line connecting the lips on the left side of the face and the lips on the right side of the face in the facial image and the horizontal line based on the coordinate information.

[0051] Step 124a: determine an angle feature based on the first angle and / or the second angle and / or the third angle.

[0052] For a harmonious face, the features on the left side of the face and the corresponding features on the right side of the face can be considered to be on the same horizontal line. Therefore, if the angle between the line connecting a feature on the left side of the face and the corresponding feature on the right side of the face and the horizontal line is larger, the symmetry between the left side of the face and the right side of the face is considered to be worse. In other words, the symmetry of the face can be characterized by angle features.

[0053] The angle feature may include an eyebrow angle feature and / or a nose angle feature and / or a lip angle feature. That is, the angle feature may be a single feature or a comprehensive feature, wherein the comprehensive feature may more accurately characterize the symmetry of the left and right sides of the face of the user to be evaluated from the angle aspect.

[0054] After processing in step A3, the left eye and the right eye in the face image are on the same horizontal line. On this basis, the angle formed by the line connecting each feature (eyebrow, nose and lip) on the left side of the face and each feature (eyebrow, nose and lip) on the right side of the face and the horizontal line can be calculated in turn, thereby determining the angle feature. Figure 3 , Figure 3The schematic diagram of the first line, the second line and the third line in the face image is shown. For the eyebrows, the midpoint coordinates of the eyebrows on the left side of the face and the midpoint coordinates of the eyebrows on the right side of the face can be determined according to the coordinate information, and the first line can be constructed according to the midpoint coordinates of the eyebrows on both sides of the face. The eyebrow angle feature, that is, the first angle between the first line and the horizontal line, can be calculated through the first line. For the nose, the endpoint coordinates of the nose on the left side of the face and the endpoint coordinates of the nose on the right side of the face can be determined according to the coordinate information, and the second line can be constructed according to the endpoint coordinates of the noses on both sides of the face. The nose angle feature, that is, the second angle between the second line and the horizontal line, can be calculated through the second line. For the lips, the endpoint coordinates of the lips on the left side of the face and the endpoint coordinates of the lips on the right side of the face can be determined according to the coordinate information, and the third line can be constructed according to the endpoint coordinates of the lips on both sides of the face. The lip angle feature, that is, the third angle between the third line and the horizontal line, can be calculated through the third line.

[0055] In some embodiments, for each feature, its angle feature may be determined by the following formula:

[0056]

[0057] Among them, (x LF ,y LF ) represents the coordinate information of the midpoint or endpoint of the feature on the left face, (x RF ,y RF ) represents the coordinate information of the midpoint or endpoint of the feature on the right side of the face.

[0058] In some embodiments, the symmetry of the left and right faces of the user to be evaluated may be characterized by regional features, that is, the symmetry features may include regional features. The above step 120 specifically includes:

[0059] Step 121b: determining a region group corresponding to a preset feature from the face image based on the coordinate information.

[0060] Step 121b: determining a group feature of each region group based on the first region and the second region.

[0061] Step 121b: Determine the regional features based on the group features.

[0062] The facial features (excluding ears) are the most representative facial features in a face image and are often used for facial recognition. For a symmetrical face, the area of ​​each facial feature on the left side of the face and the area on the right side of the face can be considered as a mirror image. Therefore, if the mirror effect between a facial feature on the left side of the face and the corresponding facial feature on the right side is worse, it can be considered that the symmetry between the left and right sides of the face is worse, that is, the symmetry of the face can be characterized by regional features.

[0063] The preset features include at least one of the above-mentioned facial features. For each preset feature, a corresponding region group can be determined from the face image according to the coordinate information, and the region group includes a first region of the left face of the corresponding preset feature in the face image and a second region of the right face. For example, the region formed by the eyes on the left face is the first region, and the region formed by the eyes on the right face is the second region, and the two regions together are the region group corresponding to the eyes.

[0064] See also Figure 4 , Figure 4 A schematic diagram of four area groups in a face image is shown. For each area group, a group feature of the area group can be determined based on the first area and the second area included in the area group. When there is only one area group, the group feature is the area feature; when there are more than two area groups, the combination of more than two group features is the area feature.

[0065] In some embodiments, the above step 121b specifically includes:

[0066] For each zone group:

[0067] A first distance between each pixel point in the first area and the preset center line, and a second distance between each pixel point in the second area and the preset center line are calculated.

[0068] A group characteristic of the group of regions is determined based on the first distance and the second distance.

[0069] The preset centerline may be the facial centerline mentioned in step 120. By calculating a first distance between each pixel point in the first region and the facial centerline, and a second distance between each pixel point in the second region and the preset centerline, the group feature of the region group may be determined based on the two distances.

[0070] In some embodiments, the distance between a pixel point and a preset center line can be calculated by the following formula:

[0071] D(i,j)=|j M -j|

[0072] Among them, j M represents the coordinate information of the point on the preset center line corresponding to the pixel point, and j represents the coordinate information of the pixel point.

[0073] After calculating the distance between each pixel and the preset center line, the product of the gray value of the pixel and the distance can be calculated. The calculation formula is as follows:

[0074] GD(i,j)=f(i,j).*D(i,j)

[0075] Among them, f(i,j) represents the grayscale value of the pixel.

[0076] After calculating the product of the gray value and distance of each pixel, the regional features can be determined based on the product:

[0077]

[0078] Wherein, LF represents the first region of the left side of the face, and RF represents the second region of the right side of the face. It can be understood that the preset feature can be a single feature or a combination of 2-4 features. For a combination of more than 2 features, the group features of the regional group corresponding to each feature can be calculated based on the above formula, and then multiple group features can be combined to obtain the final regional feature.

[0079] When calculating the group characteristics of each regional group, the above steps can be used for calculation.

[0080] In some embodiments, the symmetry of the left and right faces of the user to be evaluated may be characterized by a centroid feature, that is, the symmetry feature may include a centroid feature. The above step 120 specifically includes:

[0081] Step 121c: determine a first centroid of the left side of the face and a second centroid of the right side of the face in the facial image based on the coordinate information.

[0082] Step 122c, calculate the Euclidean distance between the first centroid and the second centroid to obtain the centroid feature.

[0083] The centroid represents the mass center of an object. For a grayscale image, the centroid can be considered as the grayscale value center. That is, the centroid can reflect the center of the grayscale value distribution of the grayscale image, which can reflect the integrity of the image. It can be considered that the distance between the first centroid of the image of the left face and the second centroid of the image of the right face can reflect the symmetry of the face of the user to be evaluated as a whole.

[0084] Specifically, after calculating the two centroids, the Euclidean distance between the two centroids can be calculated based on the coordinate information corresponding to the two centroids, and then the centroid features can be determined according to the Euclidean distance. The larger the Euclidean distance, the worse the symmetry of the user's face; the smaller the Euclidean distance, the better the symmetry of the user's face.

[0085] In some embodiments, the coordinate information of the first mass center or the second mass center can be calculated by the following formula:

[0086]

[0087]

[0088] Among them, x z The horizontal coordinate of the center of mass, yz represents the vertical coordinate of the centroid, f(i, j) represents the grayscale value of the pixel, M represents the length of the image, and N represents the width of the image.

[0089] In some embodiments, the symmetry of the left and right faces of the user to be evaluated may be characterized by singular value features, that is, the symmetry features may include singular value features. The above step 120 specifically includes:

[0090] Step 121d: Calculate a first matrix of singular values ​​of the left side of the face and a second matrix of singular values ​​of the right side of the face in the face image based on the coordinate information.

[0091] Step 122d: Calculate the difference between the trace of the first matrix and the trace of the second matrix to obtain singular value features.

[0092] Singular values ​​have the characteristics of stability and rotation invariance, and are an effective algebraic feature of images. For a symmetrical face, the traces of the singular values ​​on both sides of the face should be the same. When the traces of the singular values ​​on both sides of the face deviate, it means that the face is not symmetrical. Therefore, singular values ​​are a powerful feature for measuring facial symmetry. When evaluating the symmetry of a face based on singular values, two matrices of the singular values ​​of both sides of the face, namely the first matrix and the second matrix, can be determined based on the coordinate information, and then the difference between the traces of the two matrices is calculated, and the difference is determined as the singular value feature. The larger the difference, the worse the symmetry, and the smaller the difference, the better the symmetry.

[0093] In some embodiments, the first matrix or the second matrix is ​​determined by the following formula:

[0094] M=U∑V′

[0095] Where M represents a m*n size facial image. U is an m*m unitary matrix; Σ is a semi-positive m*n diagonal matrix; V' is an n*n unitary matrix. This decomposition is called the singular value decomposition of M. The elements on the diagonal of Σ are the singular values ​​of M.

[0096] In some embodiments, the symmetry of the left and right sides of the face of the user to be evaluated may be characterized by structural features, that is, the symmetry features may include structural features. The above step 120 specifically includes:

[0097] Step 121e: Calculate a first grayscale mean value of the left facial image and a second grayscale mean value of the right facial image in the facial image based on the coordinate information.

[0098] Step 122e: determine a first grayscale variance based on the grayscale value of each pixel in the left facial image and the first grayscale mean.

[0099] Step 123e: determine a second grayscale variance based on the grayscale value of each pixel in the right facial image and the second grayscale mean.

[0100] Step 124e: determining the structural features based on the first grayscale mean, the second grayscale mean, the first grayscale variance, and the second grayscale variance.

[0101] The higher the symmetry of the two images, the higher the structural similarity of the two images. That is to say, the symmetry of the face of the user to be evaluated can be evaluated based on the structural similarity between the left facial image and the right facial image in the face image. Specifically, the grayscale means of the left facial image and the right facial image can be calculated based on the coordinate information to obtain the first grayscale mean of the left facial image and the second grayscale mean of the right facial image, and then the grayscale variance of each side facial image can be calculated based on the two grayscale means. For the left facial image, the first grayscale variance can be calculated based on the grayscale value of each pixel in the image and the first grayscale mean; for the right facial image, the second grayscale variance can be calculated based on the grayscale value of each pixel in the image and the second grayscale mean. After obtaining the two grayscale variances, they can be combined with the two grayscale means calculated above to determine the similarity between the left facial image and the right facial image, and finally the structural features of the above facial image can be obtained based on the similarity.

[0102] In some embodiments, the first grayscale mean or the second grayscale mean may be calculated by the following formula:

[0103]

[0104] Among them, M represents the number of pixels, x i Represents the grayscale value of a pixel.

[0105] The first grayscale variance and the second grayscale variance can be calculated by the following formula:

[0106]

[0107] The structural characteristics can be calculated by the following formula:

[0108]

[0109] Among them, μ x Represents the first grayscale mean, μ y represents the second grayscale mean, σ x represents the first grayscale variance, σ y represents the second grayscale variance, is the similarity of the structure of the left face image and the right face image, that is, the structural feature is the dissimilarity of the structure of the left face image and the right face image.

[0110] In some embodiments, in order to improve the accuracy of the evaluation result, the symmetry feature includes at least two features of angle feature, area feature, centroid feature, singular value feature and structural feature. Based on this, before the above step 130, it also includes:

[0111] B1. Perform normalization on each symmetry feature.

[0112] B2. Perform feature splicing on the normalized symmetry features to obtain comprehensive features.

[0113] B3. Input the symmetry feature into the pre-trained evaluation model to obtain the evaluation result of the user to be evaluated: input the comprehensive feature into the pre-trained evaluation model to obtain the evaluation result.

[0114] Relying only on a single symmetry feature to evaluate the symmetry of the left and right faces of the user to be evaluated may ignore local features or overall features, resulting in low accuracy of the evaluation result and reducing the evaluation result of the user to be evaluated. Therefore, local features can be combined with overall features to improve the accuracy of the evaluation result.

[0115] In view of the fact that the value ranges of different features are generally different, if you want to combine two or more symmetry features together and avoid ignoring symmetry features with small values, you can first pre-process each symmetry feature. For example, the zero-mean standard method is used to normalize the symmetry features so that the mean of each symmetry feature after processing is 0 and the standard deviation is 1. In order to obtain more representative features, the normalized symmetry features can be combined to obtain comprehensive features, where the combination method can be splicing. Using the evaluation model to process the comprehensive feature can obtain an evaluation result with higher accuracy.

[0116] See also Figure 5 , Figure 5 A statistical graph showing the differences in six symmetry features between ASD children and non-ASD children is shown; from the graph, it can be seen that the rationality of the symmetry features selected in this application is that by processing these features, more reliable auxiliary information can be provided to medical staff to improve the reliability and accuracy of early screening of ASD.

[0117] In some embodiments, when stitching features, in addition to combining the above-mentioned symmetry features, feature combination may be further performed in combination with position information and local features based on the above-mentioned symmetry features.

[0118] The facial components can be first divided from the face image according to the facial features, for example, the left eye and the right eye are one facial component, and the nose on the left side of the face and the nose on the right side of the face are one facial component. For each facial component, it is divided into multiple facial sub-regions, and the local binary pattern is extracted from each facial sub-region to obtain a row vector. For each pre-processed facial image, the corresponding row vectors are spliced ​​to obtain a new row vector. Based on all the row vectors corresponding to the facial image data set, a feature matrix is ​​formed as a local feature. Among them, the local binary pattern (LBP) is a visual operator used for classification in the field of computer vision. It is used to compare local features of an image. The difference between the center point pixel and its neighboring pixels can be marked by a threshold.

[0119] In some embodiments, in order to facilitate the evaluation model to process the comprehensive features, after obtaining the comprehensive features, the comprehensive features can also be subjected to feature selection and / or dimensionality reduction. Among them, feature selection can be performed by the maximum correlation and minimum redundancy method: first, find the features with the greatest correlation in the comprehensive features; then eliminate the redundant features by the minimum redundancy method; through these two combined operations, select the most representative features from the comprehensive features to form a feature subset, so that the correlation between the features in the selected feature subset is the largest and the redundancy is the smallest, and the redundant features can be eliminated to reduce overfitting and enhance the generalization ability of the method. In addition, the comprehensive features can also be subjected to dimensionality reduction by the principal component method: the comprehensive features are recombined into a new set of mutually unrelated features. This operation can reduce the number of features without reducing useful information, and make the features independent of each other, eliminate features with low discrimination, thereby evaluating the accuracy, improving the computational efficiency and saving computer resources.

[0120] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0121] In order to verify the effect that can be achieved by this application, the inventor obtained a data set of ASD children and non-ASD children from Kaggle for testing. A total of 110 facial images of ASD children and 117 facial images of non-ASD children were used. The original facial images were used for feature extraction, and the features obtained were used to evaluate whether the two channels in each facial image had ASD using a binary classification method.

[0122] The test results show that medical staff can diagnose each two-pass by combining the evaluation results, and the diagnostic efficiency is greatly improved. In addition, only one child with ASD in the data set was classified as a child without ASD, and two children without ASD were classified as children with ASD. The accuracy of the evaluation results can be as high as 98.68%, that is, the evaluation results obtained by the method of the present application can provide reliable auxiliary information for medical staff.

[0123] Corresponding to the autism assessment method described in the above embodiment, Figure 6 A structural block diagram of an autism assessment device 6 provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0124] Reference Figure 6 , the autism assessment device 6 comprises:

[0125] The recognition module 61 is used to perform key point recognition on the face image of the user to be evaluated and obtain the coordinate information of each key point;

[0126] A determination module 62, configured to determine a symmetry feature of the face of the user to be evaluated based on the coordinate information, wherein the symmetry feature is used to characterize the symmetry between the left side and the right side of the face of the user to be evaluated;

[0127] The evaluation module 63 is used to input the symmetry feature into the pre-trained evaluation model for processing to obtain the evaluation result of the user to be evaluated.

[0128] Optionally, the evaluation device 6 may further include:

[0129] An acquisition module, used to acquire an initial face image of the user to be evaluated;

[0130] A scaling module, used to scale the initial face image according to a preset ratio;

[0131] The rotation module is used to perform a rotation operation on the scaled initial face image to obtain a face image in which the left eye and the right eye of the user to be evaluated are on the same horizontal line.

[0132] Optionally, the symmetry feature includes an angle feature, and the first determination module 62 may include:

[0133] A first calculation unit, configured to calculate a first angle between a first connecting line between an eyebrow on a left side of a face and an eyebrow on a right side of a face in a face image and a horizontal line based on the coordinate information;

[0134] and / or, calculating a second angle between a second connecting line between a nose on the left side of the face and a nose on the right side of the face in the face image and a horizontal line based on the coordinate information;

[0135] and / or, calculating a third angle between a third line connecting the lips on the left side of the face and the lips on the right side of the face in the facial image and the horizontal line based on the coordinate information;

[0136] The first determining unit is configured to determine an angle feature based on the first angle and / or the second angle and / or the third angle.

[0137] Optionally, the symmetry feature includes a regional feature, and the first determination module 62 may include:

[0138] A second determining unit is used to determine a region group corresponding to a preset feature from the face image based on the coordinate information, the preset feature includes at least one feature of eyebrows, eyes, nose and lips, and the region group includes a first region on the left side of the face and a second region on the right side of the face corresponding to the preset feature in the face image;

[0139] a third determining unit, configured to determine a group feature of each region group based on the first region and the second region;

[0140] The fourth determining unit is configured to determine the regional feature based on the group feature.

[0141] Optionally, the symmetry feature includes a centroid feature, and the first determination module 62 may include:

[0142] A fifth determining unit, configured to determine a first centroid of a left face and a second centroid of a right face in the face image based on the coordinate information;

[0143] The second calculation unit is used to calculate the Euclidean distance value between the first centroid and the second centroid to obtain the centroid feature.

[0144] Optionally, the symmetry feature includes a singular value feature, and the first determination module 62 may include:

[0145] A third calculation unit, used for calculating a first matrix of singular values ​​of a left side face and a second matrix of singular values ​​of a right side face in the face image based on the coordinate information;

[0146] The fourth calculation unit is used to calculate the difference between the trace of the first matrix and the trace of the second matrix to obtain the singular value feature.

[0147] Optionally, the symmetry feature includes a structural feature, and the first determination module 62 may include:

[0148] A fifth calculation unit, configured to calculate a first grayscale mean value of a left facial image and a second grayscale mean value of a right facial image in the facial image based on the coordinate information;

[0149] a sixth determining unit, configured to determine a first grayscale variance based on the grayscale value of each pixel in the left facial image and the first grayscale mean;

[0150] A seventh determining unit, configured to determine a second grayscale variance based on the grayscale value of each pixel in the right facial image and the second grayscale mean;

[0151] An eighth determining unit is used to determine the structural feature based on the first grayscale mean, the second grayscale mean, the first grayscale variance, and the second grayscale variance.

[0152] Optionally, the symmetry feature includes at least two features of an angle feature, a region feature, a centroid feature, a singular value feature, and a structure feature. The autism assessment device 6 may further include:

[0153] A normalization module, used for performing normalization processing on each symmetry feature;

[0154] A splicing module is used to splice the normalized symmetry features to obtain comprehensive features;

[0155] The above evaluation module is specifically used to: input comprehensive features into a pre-trained evaluation model to obtain evaluation results.

[0156] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0157] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 7 As shown, the electronic device 7 of this embodiment includes: at least one processor 70 ( Figure 7 Only one is shown in the figure) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on at least one processor 70. When the processor 70 executes the computer program 72, the steps in any of the above-mentioned autism assessment method embodiments are implemented, for example Figure 1 Steps 110-130 are shown.

[0158] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0159] In some embodiments, the memory 71 may be an internal storage unit of the electronic device 7, such as a hard disk or memory of the electronic device 7. In other embodiments, the memory 71 may also be an external storage device of the electronic device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Further, the memory 71 may also include both an internal storage unit of the terminal device 7 and an external storage device. The memory 71 is used to store operating devices, application programs, boot loaders (BootLoader), data and other programs, such as program codes of computer programs. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0160] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0161] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0162] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0163] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0164] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0165] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0166] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for assessing autism, characterized in that: include: Perform key point recognition on the facial image of the user to be evaluated to obtain the coordinate information of each key point; Determine, based on the coordinate information, a symmetry feature of the face of the user to be evaluated, wherein the symmetry feature is used to characterize the symmetry between the left side and the right side of the face of the user to be evaluated; Inputting the symmetry feature into a pre-trained evaluation model for processing to obtain an evaluation result of the user to be evaluated; Before performing key point recognition on the facial image of the user to be evaluated, the method further includes: Obtaining an initial facial image of the user to be evaluated; Scaling the initial face image according to a preset ratio; Performing a rotation operation on the scaled initial face image to obtain the face image, in which the left eye and the right eye of the user are on the same horizontal line; The symmetry feature includes a structural feature and at least one of an angle feature, a region feature, a centroid feature, and a singular value feature; with respect to the structural feature, determining the symmetry feature of the face of the user to be evaluated based on the coordinate information includes: Calculate a first grayscale mean of a left facial image and a second grayscale mean of a right facial image in the facial image based on the coordinate information; Determine a first grayscale variance based on the grayscale value of each pixel in the left facial image and the first grayscale mean; Determine a second grayscale variance based on the grayscale value of each pixel in the right facial image and the second grayscale mean; Determining the structural feature based on the first grayscale mean, the second grayscale mean, the first grayscale variance, and the second grayscale variance; The first grayscale value or the second grayscale value is calculated by the following formula: Wherein, M represents the number of pixels, and x i Represents the gray value of a pixel; The first grayscale variance or the second grayscale variance is calculated by the following formula: The structural features are calculated by the following formula: The μ x represents the first grayscale mean, the μ y represents the second grayscale mean, the σ x represents the first grayscale variance, the σ y represents the second grayscale variance, the represents the structural similarity between the left facial image and the right facial image, and S represents the structural feature.

2. The evaluation method according to claim 1, characterized in that: The symmetry feature includes an angle feature, and determining the symmetry feature of the face of the user to be evaluated based on the coordinate information includes: Calculating a first angle between a first line connecting an eyebrow on the left side of the face and an eyebrow on the right side of the face in the face image and the horizontal line based on the coordinate information; and / or, calculating a second angle between a second line connecting a nose on the left side of the face and a nose on the right side of the face in the facial image and the horizontal line based on the coordinate information; And / or, calculating a third angle between a third line connecting the lips on the left side of the face and the lips on the right side of the face in the facial image and the horizontal line based on the coordinate information; The angle feature is determined based on the first angle and / or the second angle and / or the third angle.

3. The evaluation method according to claim 1, characterized in that: The symmetry feature includes a region feature, and determining the symmetry feature of the face of the user to be evaluated based on the coordinate information includes: Determine a region group corresponding to a preset feature from the facial image based on the coordinate information, the preset feature including at least one of eyebrows, eyes, nose and lips, and the region group including a first region on the left side of the facial image and a second region on the right side of the facial image corresponding to the preset feature; determining a group characteristic of each of the area groups based on the first area and the second area; The region characteristics are determined based on the set of characteristics.

4. The evaluation method according to claim 1, characterized in that: The symmetry features include centroid features, The determining the symmetry feature of the face of the user to be evaluated based on the coordinate information includes: Determine a first centroid of the left side of the face and a second centroid of the right side of the face in the facial image based on the coordinate information; The Euclidean distance between the first centroid and the second centroid is calculated to obtain the centroid feature.

5. The evaluation method according to claim 1, characterized in that: The symmetry feature includes a singular value feature, and determining the symmetry feature of the face of the user to be evaluated based on the coordinate information includes: Calculate a first matrix of singular values ​​of the left side of the face and a second matrix of singular values ​​of the right side of the face in the face image based on the coordinate information; The difference between the trace of the first matrix and the trace of the second matrix is ​​calculated to obtain the singular value feature.

6. The evaluation method according to any one of claims 1 to 5, characterized in that: The symmetry feature includes at least two features of angle feature, area feature, centroid feature, singular value feature and structure feature. Before inputting the symmetry feature into the pre-trained evaluation model, the method further includes: Performing normalization processing on each of the symmetry features; Performing feature splicing on each of the normalized symmetry features to obtain a comprehensive feature; The symmetry feature is input into the pre-trained evaluation model to obtain the evaluation result of the user to be evaluated: The comprehensive features are input into a pre-trained evaluation model to obtain the evaluation result.

7. An autism assessment device, characterized in that: include: A recognition module is used to identify key points of the face image of the user to be evaluated and obtain the coordinate information of each key point; A determination module, configured to determine a symmetry feature of the face of the user to be evaluated based on the coordinate information, wherein the symmetry feature is used to characterize the symmetry between the left side and the right side of the face of the user to be evaluated; An evaluation module, used for inputting the symmetry feature into a pre-trained evaluation model for processing to obtain an evaluation result of the user to be evaluated; The evaluation device further comprises: An acquisition module, used to acquire an initial face image of the user to be evaluated; A scaling module, used to scale the initial face image according to a preset ratio; A rotation module, used to perform a rotation operation on the scaled initial face image to obtain the face image, in which the left eye and the right eye of the user are on the same horizontal line; The symmetry feature includes a structural feature and at least one of an angle feature, a region feature, a centroid feature, and a singular value feature; and for the structural feature, the determination module includes: A fifth calculation unit, configured to calculate a first grayscale mean value of a left facial image and a second grayscale mean value of a right facial image in the facial image based on the coordinate information; a sixth determining unit, configured to determine a first grayscale variance based on the grayscale value of each pixel in the left facial image and the first grayscale mean; a seventh determining unit, configured to determine a second grayscale variance based on the grayscale value of each pixel in the right facial image and the second grayscale mean; an eighth determining unit, configured to determine the structural feature based on the first grayscale mean, the second grayscale mean, the first grayscale variance, and the second grayscale variance; The first grayscale value or the second grayscale value is calculated by the following formula: The M represents the number of pixels, and the x i Represents the gray value of a pixel; The first grayscale variance or the second grayscale variance is calculated by the following formula: The structural features are calculated by the following formula: The μ x represents the first grayscale mean, the μ y represents the second grayscale mean, the σ x represents the first grayscale variance, the σ y represents the second grayscale variance, the represents the structural similarity between the left facial image and the right facial image, and S represents the structural feature.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the evaluation method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the evaluation method according to any one of claims 1 to 6 is implemented.

Images