Physique recognition processing method, apparatus and device, and readable storage medium

Abstract

The invention belongs to the technical field of digital medical treatment, provides a physique recognition processing method and device, computer equipment and a computer readable storage medium, and aims to solve the problem of low accuracy of physique recognition processing in the prior art. According to the voice data, the voice features are obtained, and then according to the facial features and the voice features, the users are classified by adopting a physique recognition model based on a classification algorithm to obtain the physique categories to which the users belong, so that physique recognition based on machine learning is realized, and therefore, traditional Chinese medicine and artificial intelligence are combined; according to the embodiment of the invention, the method achieves the classification of the constitution category of the user through the quantized facial features and voice features, can avoid the non-objectivity of the constitution classification of the user through the subjective feeling, improves the objectivity of the constitution classification of the user, and improves the accuracy of constitution recognition processing.

Description

Technical Field

[0001] The present application relates to the field of digital medical technology, and in particular to a method, device, computer equipment and computer-readable storage medium for physical constitution identification. Background Art

[0002] Traditional Chinese medicine believes that people with different physiques will have different performances, and there are generally five different types of people. When traditional Chinese medicine provides health advice or health advice to users, such as physical and mental adjustment advice, for example, when traditional Chinese medicine psychology adjusts the user's emotional distress, it will first identify the user's physical type, and then provide the user with health advice or health advice based on the user's physical type.

[0003] In traditional technology, when physical constitution is identified, people's physical constitution is mainly classified based on the subjective feelings of relevant personnel, such as naked eye observation. However, this physical constitution classification method is subject to human subjective judgment and lacks objectivity, which reduces the accuracy of physical constitution identification. Summary of the invention

[0004] The present application provides a method, apparatus, computer device and computer-readable storage medium for processing physical constitution identification, which can solve the technical problem of low accuracy in processing physical constitution identification in traditional technologies.

[0005] In a first aspect, the present application provides a method for processing physical constitution identification, comprising: obtaining a facial image of a user and obtaining voice data of the user; obtaining facial features based on the facial image, and obtaining voice features based on the voice data; and classifying the user based on the facial features and the voice features using a physical constitution identification model based on a classification algorithm to obtain the physical constitution category to which the user belongs.

[0006] In the second aspect, the present application provides a processing device for physical constitution identification, comprising: a first acquisition unit, used to acquire a facial image of a user and acquire voice data of the user; a second acquisition unit, used to acquire facial features based on the facial image, and acquire voice features based on the voice data; a classification unit, used to classify the user according to the facial features and the voice features, using a physical constitution identification model based on a classification algorithm, to obtain the physical constitution category to which the user belongs.

[0007] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the processing method for physical constitution identification when executing the computer program.

[0008] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the processing method for physical constitution identification.

[0009] The present application provides a processing method, device, computer equipment and computer-readable storage medium for constitution identification. The processing method obtains facial images and voice data of a user, and obtains facial features based on the facial image, thereby uniformly quantifying facial features based on face recognition, and obtains voice features based on voice data, thereby realizing uniformly quantifying voice features based on voice processing, and then classifies the user according to the facial features and voice features using a constitution identification model based on a classification algorithm to obtain the constitution category to which the user belongs, thereby realizing constitution identification based on machine learning, combining traditional Chinese medicine with artificial intelligence, and using quantified facial features and voice features, and based on a machine learning algorithm and a unified quantified constitution determination standard to classify the user's constitution category, which can avoid the objectivity of relevant personnel in classifying the user's constitution based on subjective feelings, improve the objectivity of classifying the user's constitution, and improve the accuracy of constitution identification processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A schematic diagram of a process flow of a method for processing physical constitution identification provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of the first sub-flow of the method for processing physical constitution identification provided in an embodiment of the present application;

[0013] Figure 3 A schematic diagram of a second sub-flow of the method for processing physical constitution identification provided in an embodiment of the present application;

[0014] Figure 4 A schematic diagram of a third sub-flow of the method for processing physical constitution identification provided in an embodiment of the present application;

[0015] Figure 5 A schematic block diagram of a processing device for physical constitution identification provided in an embodiment of the present application;

[0016] Figure 6 A schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0019] The embodiment of the present application provides a method for processing constitution identification, which can be applied to computer devices such as smart phones, tablet computers, laptops, and desktop computers, and used when processing constitution identification of people. Based on the constitution identification processing, the personality of the relevant person can be predicted, and a personality interpretation can be provided to the relevant person, or health advice or wellness advice and other physical and mental adjustment advice can be provided to the relevant person, thereby providing the public with simple and easy-to-use digital medicine related applications. For example, by developing an APP, ordinary users can obtain personality prediction and interpretation based on constitution identification, health advice or wellness advice and other physical and mental adjustment advice.

[0020] In the face of the technical problem of low accuracy in the processing of physical constitution identification in traditional technologies, the inventors proposed a method for processing physical constitution identification in the embodiment of the present application. The core idea of ​​the embodiment of the present application is: obtaining the user's facial features and voice features, and using a physical constitution identification model based on a classification algorithm to classify the user's physical constitution according to the user's facial features and voice features. Since a unified quantified facial feature based on image recognition and a voice feature based on voice recognition are used for different users, and a unified quantified judgment standard based on machine learning and a preset physical constitution identification processing algorithm is used, the user's physical constitution can be output more objectively based on artificial intelligence, avoiding the subjective factors of human judgment and improving the objectivity and accuracy of physical constitution identification processing.

[0021] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0022] See also Figure 1 , Figure 1 This is a flow chart of a method for processing physical constitution identification provided in an embodiment of the present application. Figure 1 As shown, the method includes the following steps S11-S13:

[0023] S11. Acquire a facial image of a user, and acquire voice data of the user.

[0024] Specifically, the user's physical identification process can be performed by collecting the user's facial image through a camera, for example, by taking a facial photo of the user through a camera to obtain a facial image, or by collecting a facial video of the user through a camera and then extracting a video frame to obtain the user's facial image. In other examples, the user's facial image can also be obtained from an image storage folder such as an album. The user's voice data can be collected through a recording device to obtain the voice data. In other examples, the voice data can also be obtained from a voice storage folder.

[0025] In an application example, a facial image can be collected by taking a photo by a user on the facial image collection page. After the facial image is successfully collected, the page automatically jumps to the sound collection page, and the user records voice to obtain voice data. It should be noted that the facial image and voice data collection order described in the above example is only to illustrate the facial image and voice data acquisition process, and is not used to limit the facial image and voice data collection order. In other examples, voice data can be collected first, and then facial images can be collected. This can be done in different steps or application scenarios according to actual needs.

[0026] S12. Acquire facial features according to the facial image, and acquire voice features according to the voice data.

[0027] Specifically, after acquiring the facial image, facial features are extracted according to preset facial key points. For example, facial features such as face shape, nose shape, mouth shape, chin, forehead, cheeks, etc. contained in the facial image are extracted, thereby realizing unified quantification of facial features based on face recognition.

[0028] After acquiring the speech data, the speech features are acquired according to the preset acoustic features. For example, acoustic features such as sound intensity E, spectral centroid Fc, spectral bandwidth B, and speech rate T contained in the speech data are extracted, thereby realizing unified quantization of speech features based on speech processing.

[0029] S13. Classify the user according to the facial features and the voice features, using a physical identification model based on a classification algorithm, to obtain the physical category to which the user belongs.

[0030] Specifically, according to facial features and voice features, a constitution recognition model based on a classification algorithm is adopted. For example, a decision tree algorithm is adopted to construct a constitution recognition model, or a random forest algorithm is adopted to construct a constitution recognition model, or a boosted tree algorithm is adopted to construct a constitution recognition model, etc., wherein decision trees are called Decision trees in English, random forests are called Random forests in English, and boosted trees are called Boosted trees in English. The constitution recognition model based on the classification algorithm is mainly based on the correspondence between human facial features, voice features, five internal organs and constitution, and emotions in traditional Chinese medicine to classify users and obtain the constitution category to which the users belong, that is, to determine which of the five attributes of the user's constitution in traditional Chinese medicine is, thereby realizing constitution recognition based on machine learning. Each category of constitution corresponds to different personality and other manifestations. Furthermore, according to each constitution, corresponding health suggestions, health suggestions, personality analysis and other physical and mental adjustment suggestions can be provided to users. For example, it can be applied in traditional Chinese medicine psychology. By using the identified physical constitution category of the user, the user's personality can be interpreted, and health care recommendations, health management, diet adjustments, physical constitution adjustments, etc. can be given based on the user's physical constitution. This can lay the foundation for providing the user with more accurate services in the future based on the identification of the user's physical constitution category.

[0031] In the embodiment of the present application, by acquiring the facial image and voice data of the user, and acquiring facial features based on the facial image, the facial features are uniformly quantified based on face recognition, and voice features are acquired based on the voice data, so as to realize uniform quantification of voice features based on voice processing, and then according to the facial features and voice features, a constitution recognition model based on a classification algorithm is adopted, and the facial features and voice features of different dimensions are combined to classify the user to obtain the constitution category to which the user belongs, thereby realizing constitution recognition based on machine learning, combining traditional Chinese medicine with artificial intelligence, using quantified facial features and voice features, and based on a machine learning algorithm and a unified quantified constitution determination standard, the constitution category of the user is classified, which can avoid the objectivity of relevant personnel in classifying the user's constitution through subjective feelings, improve the objectivity of the user's constitution classification, and improve the accuracy of the constitution recognition processing.

[0032] In one embodiment, acquiring facial features according to the facial image includes:

[0033] Acquire facial features contained in the facial image;

[0034] Acquire nose shape features contained in the facial image;

[0035] Acquire lip shape features contained in the facial image;

[0036] The face shape feature, the nose shape feature, and the lip shape feature are used as facial features.

[0037] Specifically, to obtain facial features contained in a facial image, the facial image can be divided into different areas, for example, the facial image can be divided into an upper area, a middle area, and a lower area from the top of the head to the chin in an equal manner or according to facial features, and then the area of ​​each part of the upper area, the middle area, and the lower area is calculated, and the facial features of the facial image are determined according to the area of ​​each part. The face shape can be determined according to the facial features. The face shapes include round face, long face, square face, heart-shaped face, diamond-shaped face, oval face and other facial shapes. The area characteristics of each part of the upper area, the middle area, and the lower area of ​​each face shape are different. For example, the ratio of the area of ​​each part of the upper area, the middle area, and the lower area of ​​each face shape is different.

[0038] To obtain the nose shape features contained in the facial image, the key points of the nose shape such as the center of the eyebrows, the center of the nose, the tip of the nose and the wings of the nose on both sides of the face image can be determined, and then the nose shape distance between different nose shape key points can be calculated, and the nose shape can be determined based on the nose shape distance.

[0039] To obtain the lip shape features contained in the facial image, the lip shape key points such as the upper and lower lips, four points, and two mouth corners of the facial image can be determined, and then the lip distance between different lip shape key points can be calculated to determine the lip shape based on the lip distance.

[0040] The face shape features, nose shape features, and lip shape features are combined to obtain a three-dimensional feature set [face shape, nose shape, mouth shape] as the facial features.

[0041] In the embodiment of the present application, by acquiring the face shape features, nose shape features, and lip shape features contained in the facial image, and taking the face shape features, nose shape features, and lip shape features as facial features, since the face shape features, nose shape features, and lip shape features can more accurately outline the contour features of the user's face, thereby accurately describing the face shape features of the facial image, based on the face shape features of different dimensions, the facial features and voice features of different dimensions are combined to classify the user to identify the user's constitution, which can further improve the accuracy of constitution identification, improve the accuracy of identifying the constitution category to which the user belongs, and also improve the accuracy of subsequent suggestions to the user based on the constitution category. For example, in traditional Chinese medicine psychology, the constitution category to which the user belongs is identified, and according to the user's constitution, the user's personality is interpreted, health conditioning suggestions, health management, diet adjustment, and constitution adjustment are performed. Since the accuracy of identifying the user's constitution category is high, the corresponding suggestions for the user's personality interpretation, health conditioning suggestions, health management, diet adjustment, and constitution adjustment are also relatively accurate, which can improve the effect of the suggestion.

[0042] In one embodiment, see Figure 2 , Figure 2 This is a schematic diagram of the first sub-flow of the method for processing physical constitution identification provided in the embodiment of the present application. Figure 2 As shown, in this embodiment, the step of obtaining the facial features contained in the facial image includes:

[0043] S21, obtaining preset facial key points of the facial image and facial topological structures between different preset facial key points;

[0044] S22, constructing a facial contour of the facial image according to the preset facial key points and the facial topological structure;

[0045] S23, dividing the facial contour into different facial regions, and calculating the facial areas of the facial regions;

[0046] S24: Determine the facial features of the facial image based on all the facial areas.

[0047] The facial topology structure describes the positional relationship and connection relationship between different preset facial key points.

[0048] Specifically, the preset facial key points of the facial image are obtained, and there is a facial topological structure between different preset facial key points. The facial topological structure describes the positional relationship and connection relationship between different preset facial key points, and the facial topological structure describes the contour of the face. In one example, the face detection tool Mediapipe can be used to detect the features of 468 preset facial key points of the face, and all the preset facial key points have a fixed facial topological structure. It should be noted that other methods can also be used to detect the key points of the face, which is not limited here.

[0049] The facial coordinates corresponding to the preset facial key points are obtained. The facial coordinates describe the positions of the preset facial key points on the face. According to the facial coordinates and the facial topology, all facial coordinates are combined into the facial contour of the facial image, thereby constructing the facial contour of the facial image.

[0050] The facial contour is divided into different facial regions, and the facial area of ​​each facial region is calculated. For example, the facial image is divided into an upper region, a middle region, and a lower region in an even manner from the top of the head to the chin or according to facial features. In one example, the line connecting the center of the eyebrows and the center of the nose can be used as the facial center axis, and the facial image can be divided into an upper region, a middle region, and a lower region in an even manner from the top of the head to the chin, and the facial area of ​​each facial region is calculated.

[0051] When determining the facial features of a facial image based on all facial areas, a facial feature set can be formed by all facial areas. The facial feature set can be described by a feature vector. For example, if the facial image is divided into an upper region, a middle region, and a lower region, and the facial area of ​​each part is s1, s2, and s3, the facial feature set can be described as: N i =[s1, s2, s3]. Obtain a preset face feature set corresponding to the preset face feature. The preset face feature set can be obtained by relevant personnel based on facial samples. The facial feature set and the preset face feature set describe features of the same dimension. Calculate the similarity between the facial feature set and the preset face feature set. According to the similarity, select the most similar preset face feature as the face feature contained in the facial image. In an example, if the facial feature set is described as: N in =[s i1 ,s i2 ,s i3 ], the preset face feature set is described as N jn =[s j1 ,s j2 ,s j3 ], the similarity between the facial feature set and the preset face shape feature set can be calculated using the following formula:

[0052] S ij =(s i1 -s j1 ) 2 +(s i2 -s j2 ) 2 +(s i2 -s j2 ) 2 ;Formula (1)

[0053] Among them, S ij Describes the similarity between the facial feature set and the preset face shape feature set, s i1 Describes the area of ​​the first facial region of the i-th facial feature set, s j1 Describe the area of ​​the first facial region of the jth preset face feature set, and so on.

[0054] In the embodiment of the present application, a facial contour is constructed through preset facial key points of a facial image and a facial topological structure between different preset facial key points, and the facial contour is divided into different facial regions. The facial features of the facial image are then determined based on the facial areas of the facial regions. Since the facial areas of the facial regions of each facial feature are different, the accuracy of determining the facial features of the facial image can be improved. Furthermore, facial features and voice features of different dimensions are combined to classify users so as to identify the users' physical constitution, which can further improve the accuracy of physical constitution identification.

[0055] In one embodiment, constructing the facial contour of the facial image according to the preset facial key points and the facial topological structure includes:

[0056] Constructing an initial facial contour of the facial image according to the preset facial key points and the facial topology structure;

[0057] The initial facial contour is adjusted to a preset image size to obtain a facial contour of the facial image.

[0058] Specifically, the initial facial contour of the facial image is constructed according to the preset facial key points and facial topology. The initial facial contour is the facial contour image extracted from the facial image. Since the initial facial contour cannot provide real image scale information, the initial facial contour is adjusted to a preset image size. For example, for the initial facial contour image, the center of the eyebrows and the center of the nose are used as the facial center axis. The initial facial contour image is adjusted to an image with a height of 120 pixels and a width of 160 pixels based on the facial center axis, thereby adjusting the initial facial contour to a preset uniform image length to obtain the facial contour of the facial image. Similarly, the preset facial feature set corresponding to the preset facial features is also obtained by adjusting the preset facial contour corresponding to the preset facial features to a preset image size.

[0059] In the embodiment of the present application, by adjusting the initial facial contour to a preset image size, different face shapes can be compared based on a unified preset image size, thereby improving the accuracy of calculating the similarity between a facial feature set and a preset face shape feature set, thereby improving the accuracy of determining the face shape features of a facial image, and then combining facial features and voice features of different dimensions to classify users to identify their physical constitution, which can further improve the accuracy of identifying the physical category to which the user belongs.

[0060] In one embodiment, see Figure 3 , Figure 3 This is a schematic diagram of the second sub-flow of the method for processing physical constitution identification provided in the embodiment of the present application. Figure 3 As shown, in this embodiment, the step of obtaining the nose shape feature contained in the facial image includes:

[0061] S31, obtaining preset nose shape key points contained in the facial image;

[0062] S32, obtaining nose shape distances between different preset nose shape key points;

[0063] S33: Determine the nose shape features included in the facial image according to the nose shape distance.

[0064] Specifically, the preset nose shape key points contained in the facial image are obtained, and the preset nose shape key points include the center of the eyebrows, the center of the nose, the tip of the nose, and the wings of the nose on both sides. According to the preset nose shape key points, different nose shape key points are connected to obtain the distance between different nose shape key points, that is, the nose shape distance. The nose shape distance includes the distance from the center of the nose to the tip of the nose, the distance between the wings of the nose on both sides, the distance from the center of the nose to the center of the eyebrows, etc. If the distance from the center of the nose to the tip of the nose is S1, the distance between the wings of the nose on both sides is S2, and the distance from the center of the nose to the center of the eyebrows is S3, according to the nose shape distance, the following vector can be used to describe the nose shape distance feature: N = [S1 / S2, S3 / S2], or the following vector can be used to describe the nose shape distance feature: N = [S1, S2, S3]. Then obtain the preset nose shape distance feature corresponding to the preset nose shape feature. The preset nose shape distance feature can be obtained by relevant personnel based on the nose shape sample. The nose shape distance feature and the preset nose shape distance feature describe the features of the same dimension. Calculate the similarity between the nose shape distance feature and the preset nose shape distance feature, which is similar to the above-mentioned calculation of the similarity between the facial feature set and the preset face shape feature set. It will not be repeated here. Then, based on the similarity, select the most similar preset nose shape feature as the nose shape feature contained in the facial image.

[0065] In the embodiment of the present application, the nose shape features of the facial image are determined by pre-setting the nose shape key points and the nose shape distance of the facial image, and the nose shape features of the facial image are determined according to the nose shape distance. Since the nose shape distance features of each nose shape feature are different, the accuracy of determining the nose shape features of the facial image can be improved. Then, the facial features and voice features of different dimensions are combined to classify the users so as to identify the users' physique, which can further improve the accuracy of physique identification.

[0066] In one embodiment, the step of obtaining the lip shape feature contained in the facial image includes:

[0067] Obtaining preset lip shape key points contained in the facial image;

[0068] Obtaining lip shape distances between different preset lip shape key points;

[0069] According to the lip shape distance, the lip shape features included in the facial image are determined.

[0070] Specifically, the preset lip key points contained in the facial image are obtained. The preset lip key points include four points of the upper and lower lips and two lip corners. The four points of the upper and lower lips are the upper and lower points of the middle axis of the upper lip and the upper and lower points of the middle axis of the lower lip. The middle axis of the upper lip and the middle axis of the lower lip are on the same branch line. According to the preset lip key points, different preset lip key points are connected to obtain the distance between different preset lip key points, that is, the lip distance. The lip distance includes the distance between the two corners of the mouth, the distance between the two points of the upper lip, the distance between the two points of the lower lip, etc. If the distance between the two corners of the mouth is S1, the distance between the two points of the upper lip is S2, and the distance between the two points of the lower lip is S3, according to the lip distance, the following vector can be used to describe the lip distance feature: N = [S2 / S1, S3 / S1], or the following vector can be used to describe the nose distance feature: N = [S1, S2, S3]. Then obtain the preset lip shape distance feature corresponding to the preset lip shape feature. The preset lip shape distance feature can be obtained by relevant personnel based on the lip shape sample. The lip shape distance feature and the preset lip shape distance feature describe the feature of the same dimension. Calculate the similarity between the lip shape distance feature and the preset lip shape distance feature, which is similar to the above calculation of the similarity between the facial feature set and the preset face feature set, and will not be repeated here. Then, based on the similarity, select the most similar preset lip shape feature as the lip shape feature contained in the facial image.

[0071] In the embodiment of the present application, the lip shape key points and lip shape distance of the facial image are preset, and the lip shape features of the facial image are determined according to the lip shape distance. Since the lip shape distance features of each lip shape feature are different, the accuracy of determining the lip shape features of the facial image can be improved. Then, the facial features and voice features of different dimensions are combined to classify the users so as to identify the users' physical constitution, which can further improve the accuracy of physical constitution identification.

[0072] In one embodiment, see Figure 4 , Figure 4 This is a schematic diagram of the third sub-flow of the method for processing physical constitution identification provided in the embodiment of the present application. Figure 4 As shown, in this embodiment, acquiring voice features according to the voice data includes:

[0073] S41, obtaining a short-time signal frame included in the voice data;

[0074] S42, calculating the short-time zero-crossing rate and short-time energy of the short-time signal frame;

[0075] S43: If the short-time zero-crossing rate is less than or equal to the short-time zero-crossing rate threshold, and the short-time energy is greater than or equal to the short-time energy threshold, extract the preset acoustic features of the short-time signal frame as speech features.

[0076] Specifically, for audio signals, according to time resolution, local or global concepts, duration, etc., audio features can be divided into long-term (Long-term in English) features, mid-term (Mid-term in English) features, and short-term (Short-term in English) features. Short-term features can also be called short-term features. The short-term signal frame contained in the voice data is obtained. The short-term signal frame is a voice frame. The short-term signal frame describes the short-term characteristics of the audio signal, which can also be called short-term features. It is mainly to read the voice data at a certain audio sampling rate, and use a certain frame length and step length as sampling points to perform short-term sampling of the voice to obtain the short-term signal frame contained in the voice data. For example, the voice data can be read at an audio sampling rate of 16000HZ, that is, 16000 sampling points per second, with a frame length of 100ms, a step length of 80ms, that is, 160 sampling points, and 128 sampling points per movement as a short-term signal frame, that is, determine how many sampling points the frame length and step length are at the sampling rate, and calculate the voice feature once for each short-term signal frame.

[0077] Calculate the short-term zero crossing rate and short-term energy of the short-term signal frame. The short-term zero crossing rate, abbreviated as st-ZCR, describes the number of times the sampled value of the sound signal passes through the zero point in each frame of the sound signal, or the number of times the signal waveform intersects the 0 axis. If 160 sampling points per frame are taken as an example, the short-term zero crossing rate can be calculated using the following formula:

[0078] Z = sum((N[1:159]*N[2:160])<0), formula (2);

[0079] Among them, Z describes the short-time zero-crossing rate, N describes the frame sequence of sampling points in each frame, N[1:159] describes the 1st to 159th values ​​of the frame, and N[2:160] describes the 2nd to 160th values ​​of the frame, thereby obtaining the short-time zero-crossing rate of the short-time signal frame.

[0080] Short-term energy, abbreviated as STE, describes the energy of a short signal frame. The short-term energy can be calculated using the following formula:

[0081] E = sum(N 2 [1:160]), formula (3);

[0082] Among them, E describes the short-time energy, and N[1:160] describes the 1st to 160th values ​​of the frame, thereby obtaining the short-time energy of the short-time signal frame. And according to all the short-time signal frames contained in the speech data, the short-time zero-crossing rate and short-time energy of all short-time signal frames are obtained, and the short-time zero-crossing rate Z[:] and short-time energy sequence E[:] of the speech data are obtained.

[0083] The short-time zero-crossing rate and short-time energy sequence of the speech data are normalized and quantized, and the short-time zero-crossing rate threshold and the short-time energy threshold are calculated using the Otsu method. The Otsu method, i.e., the OTSU algorithm, is the maximum inter-class variance method, which is used to calculate the distribution of sampling points, and then calculate the short-time zero-crossing rate threshold and the short-time energy threshold. Then, it is determined whether the short-time zero-crossing rate is less than or equal to the short-time zero-crossing rate threshold, and whether the short-time energy is greater than or equal to the short-time energy threshold. If the short-time zero-crossing rate is greater than the short-time zero-crossing rate threshold, and the short-time energy is less than the short-time energy threshold, it is determined that the speech frame is not a speaking position, that is, the speech frame is determined to be a noisy non-speaking voice. If the short-time zero-crossing rate is less than or equal to the short-time zero-crossing rate threshold, and the short-time energy is greater than or equal to the short-time energy threshold, the speech frame is determined to be a speaking position, and the preset acoustic features of the speech frame are extracted as speech features. The preset acoustic features include sound intensity, spectral centroid, spectral bandwidth, and speech rate. Among them, the extraction of acoustic features can be carried out in the following manner:

[0084] 1) Sound intensity E can be extracted using the following formula (4):

[0085]

[0086] Among them, m describes the mth sampling point, N describes the number of sampling points in each frame of speech, x(m) describes the speech signal of the mth sampling point, and the above formula describes the sum of the squares of the sampling points in each frame as the sound intensity E of each frame.

[0087] 2) Spectral centroid: The average point describing the spectral energy distribution of the speech frame is the spectral centroid, which can be calculated using the following formula:

[0088]

[0089] Among them, DFT is short-time Fourier transform, k is the number of Fourier transform points, K is [1, N], Fl describes the spectral centroid of the lth frame, N describes the number of sampling points in each frame, and the length of a sequence of length N after discrete Fourier transform is still N. Each DFT point is the corresponding length of a frequency.

[0090] 3) Spectral bandwidth, which describes the range in which the signal energy is concentrated in the spectrum, can be calculated using the following formula:

[0091]

[0092] Where DFT is short-time Fourier transform, k is the number of Fourier transform points, BW(l) describes the spectral bandwidth of the lth frame, and Fl describes the spectral centroid of the lth frame.

[0093] 4) Speaking rate, which describes the length of speaking segments in speech.

[0094] The above four features of all short-time signal frames of the speech data are averaged respectively to obtain four sound features of the speech data: [sound intensity E, spectrum centroid Fc, spectrum bandwidth B, speaking rate T].

[0095] In one embodiment, the user is classified according to the facial features and the voice features using a physical identification model based on a classification algorithm to obtain the physical category to which the user belongs, including:

[0096] Discretizing the speech features to obtain discretized speech features;

[0097] The facial features and the discretized voice features are input into a physical identification model based on a decision tree to obtain the physical classification of the user.

[0098] Specifically, the user's facial features and voice features are collected. The above two types of features of the user may include attributes of seven dimensions, which may be specifically [face shape F, nose shape N, mouth shape M, sound intensity E, spectral centroid Fc, spectral bandwidth B, and speech rate T]. The above seven attribute features constitute a feature set, which may be [f1, f2, ..., f7]. The seven features jointly determine the physical category to which the user belongs. Among them, face shape, nose shape, and mouth shape are integer features, corresponding to their respective categories respectively. The acoustic features of speech are continuous values ​​and have no specific categories. Based on this, in order to classify users based on speech features, the speech features are first discretized. The information entropy can be used to calculate the discretization threshold, that is, to calculate the information entropy of each speech feature. The speech features are discretized by the information entropy to obtain the discretized speech features. The information entropy is often used as a quantitative indicator of the information content of a system. The information entropy can be calculated using the following formula:

[0099]

[0100] Where D represents a certain sample distribution, Ent(D) describes the information entropy of the sample distribution D, y describes the feature classification, which is divided into two types of features: facial features and voice features in the embodiment of the present application, k is the discretized category, P k is the ratio of k categories after discretization. Taking the sound intensity in the speech feature as an example, all sound intensities are sorted from small to large, and the threshold is taken from small to large to divide the sound intensity feature into two categories, and the information entropy is calculated. The minimum entropy value is the optimal threshold, and the continuous value can be discretized into two categories according to the threshold. In this way, the speech feature is discretized to obtain a discretized speech feature;

[0101] After obtaining the discretized voice features, the facial features and the discretized voice features together constitute seven discrete features. A physical identification model based on a decision tree, wherein the process of using a decision tree to make decisions starts from the root node, tests the corresponding feature attributes in the item to be classified, and selects the output branch according to its value, until it reaches the leaf node, and the category stored in the leaf node is used as the decision result. Furthermore, a random forest algorithm can be used to construct a physical identification model, wherein a random forest is composed of many decision trees, and several decision trees are constructed using facial features and discretized voice features to form a random forest, and then the random forest is used to predict the user's facial features and voice features to obtain the user's physical category.

[0102] That is, the facial features and discretized voice features are input into the physical identification model based on the decision tree to obtain the physical category to which the user belongs. Among them, for the feature selection corresponding to the root node, the information gain of all facial features and discretized voice features can be calculated respectively, and the node with the largest information gain is selected as the root node. The information gain can be calculated using the following formula:

[0103]

[0104] Among them, V describes a set of features. For example, face shape can include 6 types of face shapes, V describes the set of 6 face shapes, v describes a specific feature in V, for example, v can describe a square face in the face shape feature set, D describes the sample distribution, D v is the number of feature i with value v, for example, face shape feature, the number of square faces; Ent(D) information entropy, that is, the information entropy of all samples in feature i, for example, face shape is classified as physical entropy; Ent(D v ), the information entropy of feature i is v, the face shape feature is the number of square face samples, the corresponding physical label calculates the information entropy, and the physical category of the user is obtained by voting by several decision trees of the random forest.

[0105] The embodiment of the present application adopts a decision tree-based constitution recognition model, combines facial features and voice features of different dimensions to classify users, identifies the users' constitutions, and obtains the users' constitution categories, thereby realizing constitution recognition based on machine learning. It combines traditional Chinese medicine with artificial intelligence, uses quantified facial features and voice features, and based on machine learning algorithms and adopts unified quantified constitution determination standards to classify the users' constitution categories, which can avoid the objectivity of relevant personnel in classifying users' constitutions based on subjective feelings, improve the objectivity of user constitution classification, and improve the accuracy of constitution recognition processing.

[0106] It should be noted that the processing methods for physical constitution identification described in the above embodiments can recombine the technical features contained in different embodiments as needed to obtain a combined implementation plan, but they are all within the scope of protection required by this application.

[0107] See also Figure 5 , Figure 5 A schematic block diagram of a device for processing physical constitution identification provided in an embodiment of the present application. Corresponding to the above-mentioned method for processing physical constitution identification, an embodiment of the present application also provides a device for processing physical constitution identification. Figure 5 As shown, the physical identification processing device includes a unit for executing the physical identification processing method described above, and the physical identification processing device can be configured in a computer device. Figure 5 The physical constitution identification processing device 50 includes a first acquisition unit 51, a second acquisition unit 52 and a classification unit 53.

[0108] The first acquisition unit 51 is used to acquire a facial image of a user and acquire voice data of the user;

[0109] A second acquisition unit 52, configured to acquire facial features according to the facial image, and acquire voice features according to the voice data;

[0110] The classification unit 53 is used to classify the user according to the facial features and the voice features by adopting a physical identification model based on a classification algorithm to obtain the physical category to which the user belongs.

[0111] In one embodiment, the second obtaining unit 52 includes:

[0112] A first acquisition subunit, used for acquiring facial features contained in the facial image;

[0113] A second acquisition subunit, used for acquiring nose shape features contained in the facial image;

[0114] A third acquisition subunit, used for acquiring lip shape features contained in the facial image;

[0115] The combination subunit is used to use the face shape feature, the nose shape feature and the lip shape feature as the face feature.

[0116] In one embodiment, the first obtaining subunit includes:

[0117] A fourth acquisition subunit, used to acquire preset facial key points of the facial image and facial topological structures between different preset facial key points;

[0118] A first construction subunit is used to construct a facial contour of the facial image according to the preset facial key points and the facial topological structure;

[0119] A first calculation subunit, configured to divide the facial contour into different facial regions and calculate the facial areas of the facial regions;

[0120] The first determining subunit is used to determine the face shape features of the face image according to all the face areas.

[0121] In one embodiment, the first construction subunit includes:

[0122] A second construction subunit is used to construct an initial facial contour of the facial image according to the preset facial key points and the facial topological structure;

[0123] The adjusting subunit is used to adjust the initial facial contour to a preset image size to obtain the facial contour of the facial image.

[0124] In one embodiment, the second obtaining subunit includes:

[0125] A fifth acquisition subunit, used for acquiring preset nose shape key points contained in the facial image;

[0126] a sixth obtaining subunit, configured to obtain nose shape distances between different preset nose shape key points;

[0127] The second determining subunit is used to determine the nose shape features included in the facial image according to the nose shape distance.

[0128] In one embodiment, the third acquisition subunit includes:

[0129] A seventh acquisition subunit, used for acquiring preset lip key points contained in the facial image;

[0130] An eighth acquisition subunit, used for acquiring lip shape distances between different preset lip shape key points;

[0131] The third determining subunit is used to determine the lip shape features contained in the facial image according to the lip shape distance.

[0132] In one embodiment, the second obtaining unit 52 includes:

[0133] A ninth acquisition subunit, configured to acquire a short-time signal frame included in the voice data;

[0134] A calculation subunit, used for calculating the short-time zero-crossing rate and short-time energy of the short-time signal frame;

[0135] The extraction subunit is used to extract the preset acoustic features of the short-time signal frame as speech features if the short-time zero-crossing rate is less than or equal to the short-time zero-crossing rate threshold and the short-time energy is greater than or equal to the short-time energy threshold.

[0136] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the processing device and each unit of the above-mentioned physical constitution identification can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0137] At the same time, the division and connection method of each unit in the above-mentioned physical constitution identification processing device is only used for illustration. In other embodiments, the physical constitution identification processing device can be divided into different units as needed, and the units in the physical constitution identification processing device can also adopt different connection orders and methods to complete all or part of the functions of the above-mentioned physical constitution identification processing device.

[0138] The above-mentioned processing device for physical constitution identification can be implemented in the form of a computer program. The computer program can be used in Figure 6 Runs on the computer device shown.

[0139] See also Figure 6 , Figure 6 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a computer device such as a desktop computer or a server, or may be a component or part in other devices.

[0140] See also Figure 6 The computer device 500 includes a processor 502, a memory and a network interface 505 connected via a system bus 501, wherein the memory may include a non-volatile storage medium 503 and an internal memory 504, and the memory may also be a volatile storage medium.

[0141] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute the above-mentioned physical constitution identification processing method.

[0142] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0143] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the above-mentioned physical constitution identification processing method.

[0144] The network interface 505 is used to communicate with other devices over the network. Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the computer device 500 to which the present application scheme is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structure and function of the memory and the processor are the same as those of the present application scheme. Figure 6 The embodiments shown are consistent and will not be described again here.

[0145] The processor 502 is used to run the computer program 5032 stored in the memory to implement the steps of the method for processing physical constitution identification described in the above embodiment.

[0146] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0147] Those skilled in the art will appreciate that all or part of the processes in the method for implementing the above embodiment can be completed by a computer program, which can be stored in a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0148] Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the following steps:

[0149] A computer program product, when running on a computer, enables the computer to execute the steps of the processing method for physical constitution identification described in the above embodiments.

[0150] The computer-readable storage medium may be an internal storage unit of the aforementioned device, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device.

[0151] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] The storage medium is a physical, non-transient storage medium, for example, it can be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, etc., which can store computer programs.

[0153] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0154] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0155] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present application can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for an electronic device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0157] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for physical constitution identification, characterized in that: include: Acquire a facial image of a user and acquire voice data of the user; Acquire facial features according to the facial image, and acquire voice features according to the voice data; According to the facial features and the voice features, a physical identification model based on a classification algorithm is used to classify the user to obtain the physical category to which the user belongs.

2. The method for body constitution identification according to claim 1, characterized in that: The step of acquiring facial features according to the facial image includes: Acquire facial features contained in the facial image; Acquire nose shape features contained in the facial image; Acquire lip shape features contained in the facial image; The face shape feature, the nose shape feature, and the lip shape feature are used as facial features.

3. The method for body constitution identification according to claim 2, characterized in that: The step of obtaining the facial features contained in the facial image comprises: Acquire preset facial key points of the facial image and facial topological structures between different preset facial key points; Constructing a facial contour of the facial image according to the preset facial key points and the facial topology structure; Dividing the facial contour into different facial regions, and calculating the facial areas of the facial regions; The facial features of the facial image are determined based on all the facial areas.

4. The method for body constitution identification according to claim 3, characterized in that: The step of constructing a facial contour of the facial image according to the preset facial key points and the facial topological structure includes: Constructing an initial facial contour of the facial image according to the preset facial key points and the facial topology structure; The initial facial contour is adjusted to a preset image size to obtain a facial contour of the facial image.

5. The method for body constitution identification according to claim 2, characterized in that: The step of obtaining the nose shape feature contained in the facial image comprises: Obtaining preset nose shape key points contained in the facial image; Obtaining nose shape distances between different preset nose shape key points; The nose shape features included in the facial image are determined according to the nose shape distance.

6. The method for body constitution identification according to claim 2, characterized in that: The step of obtaining the lip shape features contained in the facial image comprises: Obtaining preset lip shape key points contained in the facial image; Obtaining lip shape distances between different preset lip shape key points; According to the lip shape distance, the lip shape features included in the facial image are determined.

7. The method for body constitution identification according to claim 1, characterized in that: The acquiring of speech features according to the speech data includes: Acquire the short-time signal frame contained in the voice data; Calculating the short-time zero-crossing rate and short-time energy of the short-time signal frame; If the short-time zero-crossing rate is less than or equal to the short-time zero-crossing rate threshold, and the short-time energy is greater than or equal to the short-time energy threshold, the preset acoustic features of the short-time signal frame are extracted as speech features.

8. A processing device for physical constitution identification, characterized in that: include: A first acquisition unit, used to acquire a facial image of a user and acquire voice data of the user; A second acquisition unit, configured to acquire facial features according to the facial image, and acquire voice features according to the voice data; A classification unit is used to classify the user according to the facial features and the voice features, and to obtain the physical category to which the user belongs by adopting a physical identification model based on a classification algorithm.

9. A computer device, characterized in that: The computer device comprises a memory and a processor connected to the memory; the memory is used to store a computer program; the processor is used to run the computer program to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 can be implemented.

Images