Somatosensory dance game method for generating virtual character based on biological characteristics of user

Abstract

The invention discloses a somatosensory dance game method and device for generating a virtual character based on user biological characteristics and a computer readable storage medium, and the method comprises the steps: sending a reading request for reading a specified text to a user when the virtual character of the somatosensory dance game is created; collecting voice data of reading the specified text by the user; predicting biological characteristics of the user according to the voice data and a pre-trained machine learning model; and establishing a virtual character matched with the user according to the biological characteristics. The somatosensory dance game method for generating the virtual character based on the biological characteristics of the user has the advantage of intelligently and individually creating the virtual character.

Description

Technical Field

[0001] The present invention relates to the field of somatosensory game technology, and in particular to a somatosensory dance game method, device and computer-readable storage medium for generating a virtual character based on user biometrics. Background Art

[0002] With the rapid development of virtual reality (VR) and somatosensory gaming technologies, users are increasingly demanding the creation of personalized virtual characters in virtual environments. Existing somatosensory gaming and VR systems usually rely on users to manually input or select preset parameters to create virtual characters. Although this method can provide a certain degree of personalization, it has significant shortcomings in practical applications:

[0003] First, the method of manually inputting or selecting preset parameters lacks accurate recognition of the user's real biometric characteristics. Users usually need to select or input basic information such as height, body shape, gender, etc. This method is not only time-consuming and laborious, but also easily affected by the user's subjective judgment, resulting in the generated virtual character not matching the user's actual characteristics, making it difficult to fully reflect individual differences and personalized needs.

[0004] Secondly, the process of manually inputting or presetting parameters itself can easily make users feel cumbersome and complicated, especially for users who are not familiar with technical details. This not only reduces the user's enthusiasm for participation, but may also cause some users to give up using these functions, thus affecting the market acceptance of the product.

[0005] In view of the above problems, the prior art urgently needs a more intelligent and personalized virtual character creation method. Summary of the invention

[0006] The embodiment of the present application aims to realize intelligent and personalized creation of virtual characters by providing a somatosensory dance game method for generating virtual characters based on user biometrics.

[0007] To achieve the above-mentioned purpose, the embodiment of the present application provides a somatosensory dance game method for generating a virtual character based on user biometrics, comprising:

[0008] When creating a virtual character of a somatosensory dance game, a reading request for reading a specified text is sent to a user;

[0009] Collecting voice data of the user reading the specified text;

[0010] predicting a user's biometrics based on the voice data and the pre-trained machine learning model;

[0011] A virtual character matching the user is established based on the biometric characteristics.

[0012] In one embodiment, estimating the user's biometrics based on the voice data and a pre-trained machine learning model includes:

[0013] Extracting pitch features, formant features and spectrum features from the speech features;

[0014] Normalizing and reducing the dimension of the pitch features, formant features and spectrum features;

[0015] The pitch features, formant features and spectrum features after normalization and dimension reduction are fused into the first feature vector;

[0016] The first feature vector is input into a pre-trained gender recognition model to predict the gender feature of the user.

[0017] In one embodiment, estimating the user's biometric features based on the voice data and the pre-trained machine learning model further includes:

[0018] Calculating the total number of syllables and the duration of speech activity in the speech data;

[0019] Calculating speech rate features according to the total number of syllables and the duration of speech activity;

[0020] Extracting fundamental frequency features from the speech data, and calculating intonation range, intonation rising mode, intonation falling mode, intonation mean and intonation standard deviation according to the fundamental frequency features;

[0021] The intonation range, the intonation rising pattern, the intonation falling pattern, the intonation mean and the intonation standard deviation are integrated to obtain the intonation feature;

[0022] The speech rate feature, intonation feature, pitch feature, formant feature and spectrum feature are integrated into a second feature vector;

[0023] The second feature vector is input into a pre-trained age recognition model to predict the user's age feature.

[0024] In one embodiment, after inputting the second feature vector into a pre-trained age recognition model, estimating the user's biological characteristics according to the voice data and the pre-trained machine learning model further includes:

[0025] According to the recognition result of the user's gender characteristics, an age recognition model of the corresponding gender is selected from the preset model library.

[0026] In one embodiment, estimating the user's biometric features based on the voice data and the pre-trained machine learning model further includes:

[0027] Inputting the second feature vector into a pre-trained region recognition model to predict the region where the user is located;

[0028] According to the gender characteristics, age characteristics and the region where the user is located, the corresponding statistical height of the population is obtained from a preset height database;

[0029] Predicting the user's preliminary height based on the demographic height of the population and a pre-built statistical model;

[0030] The height characteristics of the user are obtained according to the preliminary height.

[0031] In one embodiment, when collecting the voice data of the user reading the specified text, a microphone array consisting of at least two microphones distributed at different heights and positions is used to collect the voice data of the user reading the specified text;

[0032] Obtaining the height characteristics of the user according to the preliminary height includes:

[0033] Calculate the user's head position based on the time difference between the user's voice reaching different microphones;

[0034] Calculate the user's physical height based on the user's head position;

[0035] Merging the physical height with the preliminary height to obtain a final height of the user;

[0036] The final height is used as the height feature of the user.

[0037] In one embodiment, the physical height is combined with the preliminary height to obtain the final height of the user, including:

[0038] According to the predefined prior distribution and likelihood function, the posterior distribution of the user's height is calculated by combining the physical height and the preliminary height. The posterior distribution calculation formula is as follows:

[0039] P(H|H physical , H pred )∝P(H physical |H)·P(H pred |H)·P(H), where

[0040] Where P(H) is the prior distribution, μ prior is the average height based on population statistics, σ prior 2 is the variance of height,

[0041] Among them, P(H physical |H) is the physical height likelihood function, H physical is the physical height predicted by the microphone array, σ physical 2 is the variance of the physical prediction error,

[0042] Among them, P(H pred |H) is the initial height likelihood function, H pred is the initial height, σ pred 2 is the error variance of the statistical model;

[0043] The mean of the posterior distribution is used as the final height. The calculation formula of the mean of the posterior distribution is as follows:

[0044]

[0045] In one embodiment, before sending a reading request for reading aloud the specified text to the user, the method further includes:

[0046] The specified text is selected from the preset text library according to the language type selected by the user.

[0047] To achieve the above-mentioned objectives, an embodiment of the present application also proposes a somatosensory dance game device for generating a virtual character based on user voice information, comprising a memory, a processor, and a somatosensory dance game program for generating a virtual character based on user voice information stored in the memory and executable on the processor, wherein when the processor executes the somatosensory dance game program for generating a virtual character based on user voice information, a somatosensory dance game method for generating a virtual character based on user biometrics as described in any one of the above items is implemented.

[0048] To achieve the above-mentioned objectives, an embodiment of the present application also proposes a computer-readable storage medium, on which is stored a somatosensory dance game program for generating a virtual character based on user voice information. When the somatosensory dance game program for generating a virtual character based on user voice information is executed by a processor, the somatosensory dance game method for generating a virtual character based on user biometrics as described in any of the above items is implemented.

[0049] The somatosensory dance game method of the technical solution of the present application, by utilizing the user's voice information and combining it with a pre-trained machine learning model, can predict the user's biological characteristics based on the voice information, and automatically generate a virtual character that highly matches the user's real characteristics. This method not only eliminates the tedious process of manually inputting or selecting preset parameters in traditional somatosensory games and virtual reality technology, but also significantly improves the personalization and realism of the virtual character. In addition, because the technical solution can automatically capture and analyze the user's voice characteristics, the character creation process becomes more natural and seamless, enhancing the user's interactive experience and immersion. Therefore, the somatosensory dance game method of the technical solution of the present application has the advantages of intelligent and personalized creation of virtual characters. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0051] Figure 1 This is a module structure diagram of an embodiment of a somatosensory dance game device for generating a virtual character based on user voice information according to the present invention;

[0052] Figure 2 The figure is a flow chart of an embodiment of a method for generating a virtual character through a somatosensory dance game based on user biometrics according to the present invention.

[0053] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0055] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] It should be noted that in the claims, any reference symbols placed between brackets shall not be constructed as limitations to the claims. The presence of "comprising" in the text does not exclude the presence of components or steps not listed in the claims. The quantifier "one" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware comprising several different components and by means of appropriately programmed computers. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of "first", "second", and "third" etc. does not indicate any order, and these words may be interpreted as names.

[0057] like Figure 1 As shown, Figure 1 It is a schematic diagram of the structure of a server 1 (also called a somatosensory dance game device for generating a virtual character based on user voice information) of a hardware operating environment involved in an embodiment of the present invention.

[0058] The server of the embodiment of the present invention is a device with display function, such as "Internet of Things devices", smart air conditioners, smart lights, smart power supplies with networking functions, AR / VR devices with networking functions, smart speakers, self-driving cars, PCs, smart phones, tablet computers, e-book readers, portable computers, etc.

[0059] like Figure 1 As shown, the server 1 includes: a memory 11, a processor 12 and a network interface 13.

[0060] The memory 11 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the server 1, such as a hard disk of the server 1. In other embodiments, the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc., equipped on the server 1.

[0061] Furthermore, the memory 11 may include an internal storage unit of the server 1 and an external storage device. The memory 11 may be used not only to store application software and various data installed on the server 1, such as the code of the somatosensory dance game program 10 for generating a virtual character based on user voice information, but also to temporarily store data that has been output or is to be output.

[0062] In some embodiments, the processor 12 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, which is used to run the program code or process data stored in the memory 11, such as executing a somatosensory dance game program 10 that generates a virtual character based on user voice information.

[0063] The network interface 13 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and is generally used to establish a communication connection between the server 1 and other electronic devices.

[0064] The network may be the Internet, a cloud network, a wireless fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), and / or a metropolitan area network (MAN). Various devices in the network environment may be configured to connect to the communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocol, and / or Bluetooth (Blue Tooth) communication protocol or a combination thereof.

[0065] Optionally, the server may further include a user interface, which may include a display, an input unit such as a keyboard, and an optional user interface may further include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be referred to as a display screen or a display unit, which is used to display information processed in the server 1 and to display a visual user interface.

[0066] Figure 1 Only the server 1 having components 11-13 and a somatosensory dance game program 10 for generating a virtual character based on user voice information is shown. It can be understood by those skilled in the art that Figure 1 The structure shown does not constitute a limitation on the server 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0067] In this embodiment, the processor 12 can be used to call the somatosensory dance game program for generating a virtual character based on user voice information stored in the memory 11, and perform the following operations:

[0068] When creating a virtual character of a somatosensory dance game, a reading request for reading a specified text is sent to a user;

[0069] Collecting voice data of the user reading the specified text;

[0070] predicting a user's biometrics based on the voice data and the pre-trained machine learning model;

[0071] A virtual character matching the user is established based on the biometric characteristics.

[0072] In one embodiment, the processor 12 may be used to call a somatosensory dance game program for generating a virtual character based on user voice information stored in the memory 11, and perform the following operations:

[0073] Extracting pitch features, formant features and spectrum features from the speech features;

[0074] Normalizing and reducing the dimension of the pitch features, formant features and spectrum features;

[0075] The pitch features, formant features and spectrum features after normalization and dimension reduction are fused into the first feature vector;

[0076] The first feature vector is input into a pre-trained gender recognition model to predict the gender feature of the user.

[0077] In one embodiment, the processor 12 may be used to call a somatosensory dance game program for generating a virtual character based on user voice information stored in the memory 11, and perform the following operations:

[0078] Calculating the total number of syllables and the duration of speech activity in the speech data;

[0079] Calculating speech rate features according to the total number of syllables and the duration of speech activity;

[0080] Extracting fundamental frequency features from the speech data, and calculating intonation range, intonation rising mode, intonation falling mode, intonation mean and intonation standard deviation according to the fundamental frequency features;

[0081] The intonation range, the intonation rising pattern, the intonation falling pattern, the intonation mean and the intonation standard deviation are integrated to obtain the intonation feature;

[0082] The speech rate feature, intonation feature, pitch feature, formant feature and spectrum feature are integrated into a second feature vector;

[0083] The second feature vector is input into a pre-trained age recognition model to predict the user's age feature.

[0084] In one embodiment, the processor 12 may be used to call a somatosensory dance game program for generating a virtual character based on user voice information stored in the memory 11, and perform the following operations:

[0085] According to the recognition result of the user's gender characteristics, an age recognition model of the corresponding gender is selected from the preset model library.

[0086] In one embodiment, the processor 12 may be used to call a somatosensory dance game program for generating a virtual character based on user voice information stored in the memory 11, and perform the following operations:

[0087] Inputting the second feature vector into a pre-trained region recognition model to predict the region where the user is located;

[0088] According to the gender characteristics, age characteristics and the region where the user is located, the corresponding statistical height of the population is obtained from a preset height database;

[0089] Predicting the user's preliminary height based on the demographic height of the population and a pre-built statistical model;

[0090] The height characteristics of the user are obtained according to the preliminary height.

[0091] In one embodiment, the processor 12 may be used to call a somatosensory dance game program for generating a virtual character based on user voice information stored in the memory 11, and perform the following operations:

[0092] Calculate the user's head position based on the time difference between the user's voice reaching different microphones;

[0093] Calculate the user's physical height based on the user's head position;

[0094] Merging the physical height with the preliminary height to obtain a final height of the user;

[0095] The final height is used as the height feature of the user.

[0096] In one embodiment, the processor 12 may be used to call a somatosensory dance game program for generating a virtual character based on user voice information stored in the memory 11, and perform the following operations:

[0097] The specified text is selected from the preset text library according to the language type selected by the user.

[0098] Based on the hardware architecture of the somatosensory dance game device for generating a virtual character based on user voice information, an embodiment of the somatosensory dance game method for generating a virtual character based on user biometrics of the present invention is proposed. The somatosensory dance game method for generating a virtual character based on user biometrics of the present invention aims to realize the intelligent and personalized creation of virtual characters.

[0099] Reference Figure 2 , Figure 2 This is an embodiment of a method for generating a virtual character by playing a somatosensory dance game based on user biological characteristics of the present invention. The method for generating a virtual character by playing a somatosensory dance game based on user biological characteristics comprises the following steps:

[0100] S10. When creating a virtual character of a somatosensory dance game, a reading request for reading a specified text is sent to a user.

[0101] Among them, a virtual character refers to a digital human image or role in a motion-sensing dance game. This image will represent the user's presence and performance in the game and interact with the game environment. A virtual character usually includes physical appearance characteristics (such as height, body shape, gender) and facial features (such as facial contours, hairstyle, etc.). In the game, this virtual character can be controlled and manipulated through a motion-sensing device (such as Kinect, Leap Motion, etc.), imitating the user's movements and performing interactions such as dancing.

[0102] The designated text is a specific text content that the game system requires the user to read aloud. The content of the designated text usually includes various types of sentences, short articles or phrases, which are designed to be representative so as to capture the user's natural voice characteristics.

[0103] Optionally, specify the text to include the following properties:

[0104] Phoneme diversity: The text content contains a variety of phonemes to ensure that the user's voice pronunciation characteristics are fully captured.

[0105] Emotional expression: The text can contain different emotional expressions, such as questions, statements, commands, etc., to capture the user's voice characteristics in different contexts.

[0106] Sentence structure: Sentences in text can be long or short and contain different grammatical structures, which helps capture the changes in the user's speaking speed and intonation in different language expressions.

[0107] Specifically, when creating a virtual character for a somatosensory dance game, the system first sends a reading request to the user. The reading request is conveyed to the user through the game interface or voice prompts, and the user needs to read the specified text according to the prompts.

[0108] It is understandable that the specified text is usually designed to contain rich voice feature elements, such as various phonemes, intonation changes, rhythm, etc. These elements can reflect the user's pronunciation habits, vocal cord characteristics, accent, etc. By requiring the user to read the specified text, the system can collect standardized voice data in a unified language environment. This standardization helps to eliminate voice feature variations caused by differences in text content, thereby improving the accuracy and consistency of machine learning models in predicting user biometrics.

[0109] S20: Collecting voice data of the user reading the specified text.

[0110] Specifically, the system can guide the user to read the specified text through a graphical interface or voice prompts. When the user reads the specified text, the system collects the voice data of the user reading the specified text through a configured microphone array or a single microphone device.

[0111] Among them, if a microphone array is used, more accurate sound source positioning and higher noise immunity can be achieved. This is because the microphone array collects the user's voice signal through multiple channels at the same time, which helps to reduce the interference of background noise.

[0112] Furthermore, for the collected speech data, high-pass and low-pass filters can be used to remove low-frequency and high-frequency noises in the speech signal.

[0113] In addition, the system can identify the valid part of the speech signal through the voice activity detection (VAD) algorithm and remove the silent or background noise part.

[0114] S30: Predicting the user's biometric characteristics based on the voice data and the pre-trained machine learning model.

[0115] Among them, pre-trained machine learning models refer to models that have been trained on a large amount of labeled data and have certain predictive capabilities. These models learn patterns and features in a large amount of data during the training process, and can output prediction results for user biometrics after given input data (such as voice features).

[0116] Optionally, the machine learning model can be at least one of a deep neural network, a convolutional neural network, a recurrent neural network, a long short-term memory network, a support vector machine, and a random forest.

[0117] Furthermore, the user's biometric characteristics include but are not limited to the user's gender, age, height, body shape, etc.

[0118] Specifically, the system first extracts multiple features from the voice data, such as pitch, intonation, formant frequency, speaking rate, etc. These features are then input into a pre-trained model, which infers the user's biometrics based on these features.

[0119] S40: establishing a virtual character matching the user according to the biometric feature.

[0120] Specifically, based on the predicted biometrics of the user, the system will generate a virtual character that matches the user's actual features. The generation of the virtual character is based on preset image templates and model libraries, which can be adjusted according to the user's biometrics. For example, if the user's biometrics show that they are tall and thin, the system will select a virtual character template that matches it and adjust the virtual character's proportions, body shape, facial features, etc. to make it closer to the user's actual features. The generated virtual character will be applied to the somatosensory dance game in real time, and the user can control the virtual character to interact and dance through somatosensory devices (such as Kinect, Leap Motion, etc.).

[0121] It can be understood that the somatosensory dance game method of the technical solution of the present application, by utilizing the user's voice information and combining it with a pre-trained machine learning model, can predict the user's biological characteristics based on the voice information, and automatically generate a virtual character that highly matches the user's real characteristics. This method not only eliminates the cumbersome process of manually inputting or selecting preset parameters in traditional somatosensory games and virtual reality technology, but also significantly improves the personalization and realism of the virtual character. In addition, because the technical solution can automatically capture and analyze the user's voice characteristics, the character creation process becomes more natural and seamless, enhancing the user's interactive experience and immersion. Therefore, the somatosensory dance game method of the technical solution of the present application has the advantages of intelligent and personalized creation of virtual characters.

[0122] In some embodiments, before sending a reading request for reading the specified text to the user, the method further includes:

[0123] The specified text is selected from the preset text library according to the language type selected by the user.

[0124] Among them, the preset text library contains specified texts in multiple language versions, and each text version contains sentences and phrases that are conducive to capturing the user's pronunciation characteristics.

[0125] Specifically, in order to improve the accuracy of virtual character generation and user experience, the system first allows users to select their preferred language type. For example, users can select different languages ​​such as English, Chinese, French, Spanish, etc. from the language options provided by the system interface.

[0126] Furthermore, according to the language type selected by the user, the system automatically selects a specified text that matches the language from a preset text library, and then sends a read request to the user. For example, if the user selects English as the language type, the system will select a specified text from the English text library and send a read request to the user.

[0127] It can be understood that by adding the steps of language selection and text matching, the method of the present application further improves the adaptability of the system to the diverse needs of users. By selecting a specified text that matches the user's language type from a preset text library, the system can accurately collect voice data in a multilingual environment, thereby improving the accuracy and personalization of virtual character generation. This not only optimizes the user's interactive experience, but also ensures the consistency and reliability of the system performance in different language environments.

[0128] In some embodiments, estimating the user's biometrics based on the voice data and the pre-trained machine learning model includes:

[0129] S31, extracting pitch features, formant features and spectrum features from the speech features.

[0130] Among them, the pitch features reflect the basic frequency of the user's voice, which is usually related to the vibration of the vocal cords. The system can use the autocorrelation method or the pitch tracking algorithm to extract the fundamental frequency (F0) from the speech signal to obtain the pitch features.

[0131] Furthermore, there are the following differences in pitch between men and women:

[0132] Physiological differences: Pitch is determined by the vibration frequency of the vocal cords (fundamental frequency, F0). Generally, men's vocal cords are longer and thicker, with a lower vibration frequency, so the fundamental frequency is usually lower than that of women. Women's vocal cords are shorter and thinner, with a higher vibration frequency, so the fundamental frequency is usually higher.

[0133] Gender differences: Statistically, the average pitch of men is usually between 85Hz and 180Hz, while the average pitch of women is usually between 165Hz and 255Hz. Therefore, pitch characteristics are one of the important signs to distinguish between men and women.

[0134] Based on the above differences, the system can infer the user's gender by analyzing the user's pitch characteristics through a machine learning model. For example, if the fundamental frequency is low, the model may tend to predict that the user is male; if the fundamental frequency is high, it may be predicted as female.

[0135] Formant Features are energy-concentrated areas in the speech spectrum that reflect the shape and size of the user's vocal tract. The system can extract multiple formant frequencies (such as F1, F2, F3) as features through linear predictive coding (LPC) or formant tracking algorithms.

[0136] Furthermore, there are the following differences in formant peaks between men and women:

[0137] Physiological differences: Formants are determined by the shape and size of the vocal tract, reflecting the physical resonant properties of the vocal tract. Men typically have longer vocal tracts (including the pharynx and oral cavity), which results in lower-frequency formants. Women typically have shorter vocal tracts, resulting in higher-frequency formants.

[0138] Gender differences: Generally, the first formant (F1) and the second formant (F2) are lower in frequency in male speech and higher in frequency in female speech. For example, a male's F1 may be between 100Hz and 900Hz, and F2 between 900Hz and 2400Hz; whereas a female's F1 and F2 are higher.

[0139] Therefore, by detecting these formant frequencies, the machine learning model can further distinguish gender. Low-frequency F1 and F2 values ​​may indicate that the user is male, while high-frequency F1 and F2 values ​​may indicate that the user is female.

[0140] Spectral features reflect the energy distribution of speech signals at different frequencies and can be extracted through short-time Fourier transform (STFT) or Mel-frequency cepstral coefficients (MFCC). Spectral features can capture the sound quality and timbre characteristics of user voices.

[0141] Furthermore, there are the following differences between men and women on the spectrum:

[0142] Physiological differences: Spectral characteristics reflect the energy distribution of speech signals in different frequency ranges. Due to the differences in the vocal tract structure between men and women, the energy of speech signals is distributed differently in the spectrum. Male speech usually has higher energy in the low-frequency part, while female speech has higher energy in the mid- and high-frequency parts.

[0143] Gender differences: By analyzing the spectral envelope or MFCC (Mel Frequency Cepstral Coefficients) of the speech signal, the difference in energy distribution between different genders can be observed. Generally, the spectrum of female speech tends to show higher energy in the higher frequency range, while the spectrum characteristics of male speech are more concentrated in the low frequency band.

[0144] Therefore, spectral features provide another dimension of gender-discriminating information for machine learning models. By analyzing the energy distribution patterns in the spectrum, the model can help confirm or enhance gender predictions based on pitch and formant features.

[0145] S32, normalizing and reducing the dimension of the pitch features, formant features and spectrum features.

[0146] The purpose of normalization is to scale the numerical ranges of different features to the same scale (such as between 0 and 1) to eliminate the scale differences between different features. This can prevent features with a larger range of eigenvalues ​​from having too much impact on the model.

[0147] Dimensionality reduction refers to using dimensionality reduction techniques to reduce feature dimensions. The purpose of this step is to remove redundant information while retaining the main components that best reflect the user's gender characteristics. The reduced feature vector will contain fewer dimensions, but the information will be more concentrated.

[0148] Specifically, after extracting various types of speech features, the system normalizes and reduces the dimensionality of these features to reduce the differences and redundancy between the features, thereby improving the prediction accuracy and efficiency of the model.

[0149] Optionally, the pitch feature and other features may be normalized using methods such as Min-Max Normalization, Z-Score Normalization, and Max Abs Scaling.

[0150] Optionally, methods such as principal component analysis (PCA), linear discriminant analysis (LDA), independent component analysis (ICA), singular value decomposition (SVD) and the like may be used to perform dimensionality reduction processing on features such as pitch features.

[0151] S33, the pitch features, formant features and spectrum features after normalization and dimensionality reduction processing are merged into a first feature vector.

[0152] Among them, by integrating the normalized and dimensionally reduced pitch features, resonance peak features and spectrum features into a comprehensive feature vector, features from different sources can be combined together for unified input into the machine learning model.

[0153] Specifically, pitch features, formant features, and spectrum features can be fused into a feature vector using methods such as feature concatenation, weighted fusion, and feature averaging.

[0154] Exemplarily, when the feature splicing method is used to merge the pitch feature, the formant feature and the spectrum feature into a first feature vector, assuming that the pitch feature vector is P, with a length of n1; the formant feature vector is F, with a length of n2; the spectrum feature vector is S, with a length of n3; then, the first feature vector V = [P, F, S], and the length of the first feature vector V is n1+n2+n3.

[0155] S34: Input the first feature vector into a pre-trained gender recognition model to predict the gender feature of the user.

[0156] Among them, the pre-trained gender recognition model can be a support vector machine (SVM), a deep neural network (DNN) or a convolutional neural network (CNN), etc. The model has been trained on a large-scale labeled dataset and can accurately map the input feature vector to the gender classification result.

[0157] Specifically, the system inputs the first feature vector into the pre-trained gender recognition model, and the model analyzes the input feature vector and outputs the user's gender prediction result. The model outputs the user's gender characteristics, usually as a binary classification (such as male / female), or it can output a probability distribution (such as the probability of male is 0.8 and the probability of female is 0.2).

[0158] It can be understood that through the feature extraction, processing and fusion process performed in steps S31 to S34, the system can accurately extract and identify gender features from the user's voice data. This process utilizes the multi-dimensional information of pitch, formant and spectral features. By inputting these features into the pre-trained machine learning model, the model can use the correlation and difference between these features to perform a comprehensive analysis, thereby accurately predicting the user's gender, thereby achieving high accuracy and robustness in gender recognition.

[0159] In some embodiments, estimating the user's biometric features based on the voice data and the pre-trained machine learning model further includes:

[0160] S110: Calculate the total number of syllables and the duration of voice activity in the voice data.

[0161] The total number of syllables refers to the number of syllables contained in the speech signal extracted from the speech data. A syllable is the basic unit of speech, usually containing a vowel or a combination of a vowel and a consonant.

[0162] The voice activity duration refers to the total duration of actual voice content in the voice signal when the user reads the specified text. The voice activity duration does not include silence or background noise, and only counts the actual speaking time.

[0163] Specifically, the system first uses the voice activity detection (VAD) algorithm to distinguish between voice activity and silence in the voice signal and extract the duration of voice activity. Next, the system calculates the total number of syllables in the voice data by detecting peaks or syllable changes in the voice. Finally, the system uses this information as a basis to further analyze the user's speaking speed characteristics. By analyzing the voice data, the system calculates the total number of syllables produced by the user when reading the specified text, as well as the duration of voice activity (that is, the actual length of time the user speaks when reading aloud).

[0164] S120: Calculate speech rate features according to the total number of syllables and the duration of speech activity.

[0165] Among them, the speech rate feature refers to the number of syllables uttered by the user per unit time, which is used to indicate the user's pronunciation speed. The speech rate feature is an important indicator reflecting the speed of speech, usually expressed in the form of "syllables / second" or "syllables / minute". The speech rate feature can reflect the user's speaking habits, which is usually related to age. For example, young people usually speak faster, while older people speak slower.

[0166] Specifically, the system obtains the speech rate feature by dividing the total number of syllables and the duration of speech activity calculated in step S110. The calculation formula is:

[0167] Speaking speed = total number of syllables / duration of speech activity.

[0168] S130, extracting fundamental frequency features from the speech data, and calculating intonation range, intonation rising pattern, intonation falling pattern, intonation mean and intonation standard deviation according to the fundamental frequency features.

[0169] Among them, the fundamental frequency feature refers to the basic frequency (F0) in the user's voice signal, which reflects the frequency of vocal cord vibration and is usually used to describe the change in pitch.

[0170] Intonation range refers to the range of fundamental frequency changes in the user's voice, representing the ups and downs of the user's voice.

[0171] The intonation rising pattern and the intonation falling pattern respectively represent the patterns or tendencies of the rising and falling fundamental frequencies in the user's voice.

[0172] The intonation mean refers to the average value of the fundamental frequency in the user's voice, reflecting the pitch level of the overall voice.

[0173] The standard deviation of intonation indicates the fluctuation range of the fundamental frequency, reflecting the stability and variation of speech.

[0174] Specifically, the system first extracts fundamental frequency features from speech data through short-time Fourier transform (STFT) or other frequency analysis methods, and then calculates the intonation range, intonation rise and fall patterns, intonation mean and intonation standard deviation based on the fundamental frequency features.

[0175] Specifically, the calculation formula of the intonation range is: intonation range=max(F0)-min(F0).

[0176] The calculation formula for the mean intonation is:

[0177]

[0178] The formula for calculating the standard deviation of intonation is:

[0179]

[0180] S140, integrating the intonation range, intonation rising pattern, intonation falling pattern, intonation mean and intonation standard deviation to obtain intonation features.

[0181] Among them, the intonation feature refers to a comprehensive feature formed by the fusion of multiple intonation-related features (such as intonation range, rising pattern, falling pattern, mean, standard deviation), which is used to describe the intonation change pattern in the user's voice.

[0182] Specifically, the system performs a fusion operation such as feature concatenation or weighted averaging on the intonation range, intonation rising pattern, intonation falling pattern, intonation mean, and intonation standard deviation calculated in step S130 to form a unified intonation feature vector, i.e., a second feature vector. The second feature vector can fully reflect the intonation characteristics of the user and serve as one of the inputs of the subsequent age recognition model.

[0183] S150: Merge the speech rate feature, intonation feature, pitch feature, formant feature and spectrum feature into a second feature vector.

[0184] Specifically, the system sequentially concatenates the speech rate features, intonation features, pitch features, formant features, and spectrum features generated in the previous steps into a unified feature vector, the second feature vector. This vector incorporates a variety of speech feature information and can provide comprehensive feature input for the machine learning model, thereby improving the model's prediction accuracy.

[0185] S160: Input the second feature vector into a pre-trained age recognition model to predict the user's age feature.

[0186] Among them, the pre-trained age recognition model refers to a machine learning model that has been trained on a large amount of labeled data and can predict the user's age based on the input feature vector.

[0187] Specifically, the system inputs the second feature vector generated in step S150 into the pre-trained age recognition model, and the model outputs the user's age prediction result by analyzing and processing these features. The model can be a machine learning algorithm such as a deep neural network (DNN), a support vector machine (SVM), a random forest, etc., which can accurately predict the user's age characteristics after being trained with a large amount of data.

[0188] It is worth noting that the age recognition model can be a classification model or a regression model. If the age recognition model is a classification model, the age output result is the age group to which the user belongs (such as 10-20 years old, 20-30 years old, etc.). If the age recognition model is a regression model, the output result is the specific age value of the user.

[0189] It can be understood that by extracting and integrating multiple speech features such as speech rate, intonation, pitch, formant and spectrum, a comprehensive second feature vector is constructed and input into the pre-trained age recognition model. The system can more accurately predict the user's age characteristics, thereby improving the personalization and realism of the virtual character. In addition, through the fusion processing of multiple features, the system can fully capture the physiological and behavioral characteristics of the user's voice, thereby improving the robustness and adaptability of the age prediction model.

[0190] In some embodiments, after inputting the second feature vector into a pre-trained age recognition model, estimating the user's biological characteristics according to the voice data and the pre-trained machine learning model further includes:

[0191] According to the recognition result of the user's gender characteristics, an age recognition model of the corresponding gender is selected from the preset model library.

[0192] Among them, the preset model library refers to a collection of multiple age recognition models pre-stored in the system. These models are specially trained and optimized according to different gender characteristics to adapt to the differences in voice characteristics between men and women.

[0193] Specifically, the system first selects an age recognition model corresponding to the gender from a preset model library based on the user's gender characteristics (such as male or female) output in step S34. The purpose of this is to use models specially trained for different genders to improve the accuracy of age prediction. Since there are significant differences between men and women in voice characteristics (such as pitch, intonation, speaking speed, etc.), using models of different genders can better capture these differences, thereby providing more accurate age prediction.

[0194] It can be understood that by selecting the age recognition model of the corresponding gender according to the gender characteristics of the user, the system can significantly improve the accuracy and reliability of age recognition by using a more targeted model to process the voice characteristics of different genders. Through such a gender classification and model selection mechanism, the system can better adapt to the individual differences of users. In addition, this method also reduces the impact of errors caused by gender characteristics on age prediction, further improving the accuracy and robustness of the model.

[0195] In some embodiments, estimating the user's biometric features based on the voice data and the pre-trained machine learning model further includes:

[0196] S210: Input the second feature vector into a pre-trained region recognition model to predict the region where the user is located.

[0197] Among them, the pre-trained region recognition model refers to a machine learning model that has been trained on large-scale labeled data and can predict the user's region based on the input feature vector.

[0198] It is worth noting that due to the diversity of languages ​​and regional cultures, people in different regions will show different patterns in speech features. For example, people in some regions speak faster, and some dialects have specific formant frequencies, which can be reflected by feature vectors.

[0199] The second feature vector is a comprehensive feature vector generated by integrating multiple speech features (such as speech rate, intonation, pitch, formant, spectrum, etc.). These features can reflect the user's pronunciation habits, intonation patterns, speech rate, pitch changes and other speech features, which often vary significantly in different geographical regions.

[0200] Then, during the model training process, the model can be specifically made to learn the mapping relationship between the second feature vector and the region where the user is located. For example, certain feature vector patterns may correspond to specific regions. In this way, by inputting new feature vectors, the model can predict the region where the user may come from based on the learned patterns and rules.

[0201] Returning to the technical solution of the present application, by inputting the second feature vector into the pre-trained region recognition model, the model analyzes these speech features to predict the user's geographic location or cultural area, and then accurately infers the region where the user is located.

[0202] S220: Obtain corresponding population statistical heights from a preset height database according to the gender characteristics, age characteristics and the region where the user is located.

[0203] The preset height database refers to a database stored in the system that contains statistical data on the height of people of different genders, age groups and regions. The database is built based on extensive demographic data and can reflect the height distribution of different groups.

[0204] Specifically, the system extracts statistical height data corresponding to the user's characteristics from a preset height database based on the user's gender, age and location. These data usually include statistical information such as average height, median height and standard deviation.

[0205] S230: Predicting the user's preliminary height based on the statistical height of the population and a pre-built statistical model.

[0206] Among them, population statistical height refers to the statistical data extracted from the preset height database, which reflects the height distribution of the population matching the user's characteristics.

[0207] Pre-built statistical models refer to mathematical models built based on large-scale data analysis and regression techniques, which are used to predict individual characteristics (such as height) based on statistical data.

[0208] Optionally, the statistical model can be any one of a linear regression model, a logistic regression model, a polynomial regression model, a Bayesian statistical model, principal component analysis, a random forest, a K-means clustering, a naive Bayes classifier, a survival analysis model, and a hidden Markov model.

[0209] Specifically, the system uses a pre-built statistical model to calculate based on the demographic height data extracted from the database and the user's gender, age and regional characteristics, and outputs a preliminary height prediction value that conforms to statistical laws.

[0210] S240: Obtain the height characteristics of the user according to the preliminary height.

[0211] Among them, the user's height feature refers to the user's height information ultimately determined by the system, which may include the user's specific height value and the relevant confidence interval or error range.

[0212] Specifically, the system uses the initial predicted height as a basis, makes fine adjustments or corrections as needed, and finally determines the user's height characteristics. This process may include combining other biometric features or further algorithm optimization to ensure the accuracy and credibility of the height characteristics.

[0213] It can be understood that since the user's region is predicted based on voice features, and the population statistical height is obtained from a preset database in combination with gender, age and region information, the system uses a pre-built statistical model to make a preliminary height prediction, thereby effectively improving the accuracy and personalization of height prediction. In addition, by integrating multi-dimensional data, the system can provide height prediction results that are more in line with the actual situation based on the user's specific characteristics. At the same time, this solution uses a statistical model to predict height, which also reduces the error caused by individual differences, making height prediction more stable and reliable.

[0214] In some embodiments, when collecting voice data of a user reading aloud the specified text, a microphone array consisting of at least two microphones distributed at different heights and positions is used to collect the voice data of the user reading aloud the specified text.

[0215] Specifically, based on the configuration of the microphone array, the system is able to infer the user's physical height by analyzing the time difference between the sound reaching different microphones, and merge it with the preliminary predicted height to obtain a more accurate final height of the user.

[0216] Further, obtaining the height characteristics of the user according to the preliminary height includes:

[0217] S310, calculating the user's head position according to the time difference between the user's voice reaching different microphones.

[0218] The time difference between the user's voice reaching different microphones refers to the time difference between the voice signal emitted by the user and the different microphones in the microphone array. Since the propagation speed of sound in the air is limited, the time it takes for the voice signal to reach different microphones from the user's head position will be different.

[0219] The user's head position refers to the position coordinates of the user's head in three-dimensional space inferred by calculating the time difference between the sound reaching different microphones.

[0220] Specifically, the system first measures the time it takes for the user's voice to reach each microphone, and then uses the time difference method (TDOA) or triangulation to calculate the user's head position. By analyzing these time differences, the system can determine the relative position of the user's head in the microphone array (including height, horizontal position, etc.), which lays the foundation for subsequent height calculation.

[0221] For example, the specific steps of calculating the user's head position by using the time difference method are as follows:

[0222] 1. Calculate the time difference of each pair of microphones receiving the voice signal. Assume that the position of microphone 1 is (x1, y1, z1) and the position of microphone 2 is (x2, y2, z2). The time for the sound to propagate from the user's head position (x, y, z) to the two microphones is t1 and t2 respectively. The time difference Δt is:

[0223]

[0224] , where v is the speed of sound in air.

[0225] 2. Head position calculation. When there are two or more microphones in the microphone array, the system can solve the position coordinates of the user's head through multiple time difference equations. Usually, more than two microphones can be used to form multiple equations, thereby solving the three-dimensional coordinates of the user's head through triangulation.

[0226] Taking a three-microphone array as an example, assume there are three microphones A(x1,y1,z1), B(x2,y2,z2) and C(x3,y3,z3), and the corresponding time difference Δt AB and Δt AC , we can list the following two equations:

[0227]

[0228] By solving this set of equations, the three-dimensional coordinates (x, y, z) of the user's head can be obtained.

[0229] S320: Calculate the user's physical height based on the user's head position.

[0230] Among them, physical height refers to the actual height of the user from the top of the head to the ground, reflecting the actual height of the user.

[0231] Specifically, the system calculates the user's physical height based on the user's head position combined with the known positions of the microphones in the microphone array.

[0232] For example, if the height of the user's head relative to the ground is known, the system can use this as an estimate of physical height. Taking into account that the user may be in different standing or sitting positions, the system can apply some adjustments or calibrations to ensure the accuracy of physical height.

[0233] For example, assuming that the user's head position is represented as (xhead, yhead, zhead), and the lowest microphone position in the microphone array is (xmic, ymic, zmic), then the user's head height h head It is the vertical coordinate value of the user's head position, that is, zhead.

[0234] Then, considering the position of the user's head relative to the microphone array (i.e., the horizontal offset may affect the measurement accuracy), the projection of the user's head relative to the microphone array in the vertical direction can be calculated, which is expressed as:

[0235] h hser =z head +z mic ,

[0236] Among them, zmic is the height coordinate of the microphone, and zhead is the height coordinate of the user's head.

[0237] Furthermore, the user's physical height Huser can be calculated by the following formula:

[0238] H user =h head +h mic ,

[0239] Among them, h mic is the distance from the lowest microphone to the ground, h head is the distance from the user's head to the lowest microphone.

[0240] S330: Merge the physical height with the preliminary height to obtain the final height of the user.

[0241] Among them, the final height is the user's comprehensive height obtained by integrating the physical height and the preliminary height, which combines the results of physical measurement and statistical prediction.

[0242] Specifically, the system combines the physical height and the preliminary height, using methods such as weighted averaging and Bayesian inference. Through this fusion method, the system can comprehensively consider the advantages and disadvantages of the two methods to obtain a more accurate estimate of the user's height. For example, the physical height and the preliminary height can be added with a certain weight to balance the results based on physical measurement and statistical prediction, thereby obtaining the user's final height.

[0243] S340: Use the final height as the height feature of the user.

[0244] Specifically, the system uses the final fused height as the user's height feature. This final height feature will be used to generate personalized virtual characters, making the generated virtual characters closer to the user's actual height, thereby improving the authenticity of the virtual experience.

[0245] It can be understood that since a microphone array is used when collecting voice data and the user's head position is calculated based on the time difference between the sound reaching different microphones, the system can infer the user's physical height based on the actual measurement results, thereby improving the accuracy of height prediction. By integrating the physical height with the preliminary predicted height, the system further eliminates the limitations of a single prediction method and ultimately obtains a more accurate user height feature. In addition, this method of combining physical measurement and statistical prediction not only improves the accuracy of the system, but also enhances the personalization and realism of the virtual character, thereby providing users with a more immersive interactive experience.

[0246] In some embodiments, fusing the physical height with the preliminary height to obtain a final height of the user includes:

[0247] S410, according to the predefined prior distribution and likelihood function, combined with the physical height and the preliminary height, calculate the posterior distribution of the user's height, the posterior distribution calculation formula is as follows:

[0248] P(H|H physical , H pred )∝P(H physical |H)·P(H pred |H)·P(H), where

[0249] Among them, P(H) is the prior distribution, which conforms to the normal distribution, μ prior is the average height based on population statistics, σ prior 2 is the variance of height,

[0250] Among them, P(H physical |H) is the physical height likelihood function, H physical is the physical height predicted by the microphone array, σ physical 2 is the variance of the physical prediction error,

[0251] Among them, P(H pred |H) is the initial height likelihood function, H pred is the initial height, σ pred 2 is the error variance of the statistical model;

[0252] S420, using the mean of the posterior distribution as the final height, the calculation formula of the mean of the posterior distribution is as follows:

[0253]

[0254] It can be understood that the above steps S410 to S420 use the Bayesian reasoning method to fuse the preliminary height based on statistical prediction and the physical height based on physical measurement. The system can comprehensively consider these two sources of information to obtain a more accurate and reliable final height estimate. In addition, this fusion method also allows the system to combine uncertainty assessment in subsequent applications, thereby improving the personalization and realism of virtual character generation. At the same time, by considering the relative importance of prior information and new evidence, Bayesian reasoning can adaptively adjust the prediction results under different conditions, enhancing the robustness and adaptability of the system.

[0255] In addition, the embodiment of the present invention also proposes a computer-readable storage medium, which can be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable storage medium includes a somatosensory dance game program 10 for generating a virtual character based on user voice information. The specific implementation of the computer-readable storage medium of the present invention is roughly the same as the specific implementation of the somatosensory dance game method for generating a virtual character based on user biometrics and the server 1, which will not be repeated here.

[0256] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0257] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0258] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0259] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0260] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0261] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for generating a virtual character based on user biometrics, characterized in that: include: When creating a virtual character of a somatosensory dance game, a reading request for reading a specified text is sent to a user; Collecting voice data of the user reading the specified text; predicting a user's biometrics based on the voice data and the pre-trained machine learning model; A virtual character matching the user is established based on the biometric characteristics.

2. The method for generating a virtual character based on user biometrics according to claim 1, characterized in that: Estimating the user's biometrics based on the voice data and the pre-trained machine learning model, including: Extracting pitch features, formant features and spectrum features from the speech features; Normalizing and reducing the dimension of the pitch features, formant features and spectrum features; The pitch features, formant features and spectrum features after normalization and dimension reduction are fused into the first feature vector; The first feature vector is input into a pre-trained gender recognition model to predict the gender feature of the user.

3. The method for generating a virtual character based on user biometrics as claimed in claim 2, characterized in that: Estimating the user's biometrics based on the voice data and the pre-trained machine learning model, further comprising: Calculating the total number of syllables and the duration of speech activity in the speech data; Calculating speech rate features according to the total number of syllables and the duration of speech activity; Extracting fundamental frequency features from the speech data, and calculating intonation range, intonation rising mode, intonation falling mode, intonation mean and intonation standard deviation according to the fundamental frequency features; The intonation range, the intonation rising pattern, the intonation falling pattern, the intonation mean and the intonation standard deviation are integrated to obtain the intonation feature; The speech rate feature, intonation feature, pitch feature, formant feature and spectrum feature are integrated into a second feature vector; The second feature vector is input into a pre-trained age recognition model to predict the age feature of the user.

4. The method for generating a virtual character based on user biometrics as claimed in claim 3, characterized in that: After inputting the second feature vector into the pre-trained age recognition model, estimating the user's biological features according to the voice data and the pre-trained machine learning model further includes: According to the recognition result of the user's gender characteristics, an age recognition model of the corresponding gender is selected from the preset model library.

5. The method for generating a virtual character based on user biometrics as claimed in claim 3, characterized in that: Estimating the user's biometrics based on the voice data and the pre-trained machine learning model, further comprising: Inputting the second feature vector into a pre-trained region recognition model to predict the region where the user is located; According to the gender characteristics, age characteristics and the region where the user is located, the corresponding statistical height of the population is obtained from a preset height database; Predicting the user's preliminary height based on the demographic height of the population and a pre-built statistical model; The height characteristics of the user are obtained according to the preliminary height.

6. The method for generating a virtual character based on user biometrics as claimed in claim 5, characterized in that: When collecting voice data of the user reading the specified text, using a microphone array consisting of at least two microphones distributed at different heights and positions to collect the voice data of the user reading the specified text; Obtaining the height characteristics of the user according to the preliminary height includes: Calculate the user's head position based on the time difference between the user's voice reaching different microphones; Calculate the user's physical height based on the user's head position; Merging the physical height with the preliminary height to obtain a final height of the user; The final height is used as the height feature of the user.

7. The method for generating a virtual character based on user biometrics according to claim 6, characterized in that: Merging the physical height with the preliminary height to obtain a final height of the user includes: According to the predefined prior distribution and likelihood function, the posterior distribution of the user's height is calculated by combining the physical height and the preliminary height. The posterior distribution calculation formula is as follows: P(H|H physical , H pred ) ∝ P ( H physical |H) · P(H pred |H) · P(H), where, Where P(H) is the prior distribution, μ prior is the average height based on population statistics, σ prior 2 is the variance of height, Among them, P(Hphhysica1l H) is the physical height likelihood function, H physical is the physical height predicted by the microphone array, σ physical 2 is the variance of the physical prediction error, Among them, P(H pred |H) is the initial height likelihood function, H pred is the initial height, σ pred 2 is the error variance of the statistical model; The mean of the posterior distribution is used as the final height. The calculation formula of the mean of the posterior distribution is as follows:

8. The method for generating a virtual character based on user biometrics as claimed in claim 6, characterized in that: Before sending a reading request for reading the specified text to the user, the method further includes: The specified text is selected from the preset text library according to the language type selected by the user.

9. A somatosensory dance game device for generating a virtual character based on user voice information, characterized in that: The invention comprises a memory, a processor and a somatosensory dance game program for generating a virtual character based on user voice information, which is stored in the memory and can be run on the processor. When the processor executes the somatosensory dance game program for generating a virtual character based on user voice information, the somatosensory dance game method for generating a virtual character based on user biometrics as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a somatosensory dance game program for generating a virtual character based on user voice information. When the somatosensory dance game program for generating a virtual character based on user voice information is executed by a processor, a somatosensory dance game method for generating a virtual character based on user biometrics as described in any one of claims 1-8 is implemented.

Images