Somatosensory dance game method for generating virtual character based on user tone

Abstract

The invention discloses a somatosensory dance game method and device for generating a virtual character based on user timbre and a computer readable storage medium, and the method comprises the steps: activating a preset voice assistant to carry out interactive communication with a user when a virtual game character is created; collecting voice data when the user communicates with the voice assistant; generating a timbre model matched with the timbre of the user according to the voice data and a pre-trained machine learning model; and generating a voice and an image of a virtual character based on the tone model. The somatosensory dance game method for generating the virtual character based on the tone of the user has the advantage of intelligently and individually creating the virtual character.

Description

Technical Field

[0001] The present invention relates to the field of somatosensory game technology, and in particular to a somatosensory dance game method, device and computer-readable storage medium for generating a virtual character based on user timbre. Background Art

[0002] With the rapid development of virtual reality (VR) and somatosensory gaming technologies, users are increasingly demanding the creation of personalized virtual characters in virtual environments. Existing somatosensory gaming and VR systems usually rely on users to manually input or select preset parameters to create virtual characters. Although this method can provide a certain degree of personalization, it has significant shortcomings in practical applications:

[0003] First, the method of manually inputting or selecting preset parameters lacks accurate recognition of the user's real biometric characteristics. Users usually need to select or input basic information such as height, body shape, gender, etc. This method is not only time-consuming and laborious, but also easily affected by the user's subjective judgment, resulting in the generated virtual character not matching the user's actual personality characteristics, making it difficult to fully reflect individual differences and personalized needs.

[0004] Secondly, the process of manually inputting or presetting parameters itself can easily make users feel cumbersome and complicated, especially for users who are not familiar with technical details. This not only reduces the user's enthusiasm for participation, but may also cause some users to give up using these functions, thus affecting the market acceptance of the product.

[0005] In view of the above problems, the prior art urgently needs a more intelligent and personalized virtual character creation method. Summary of the invention

[0006] The embodiment of the present application aims to realize the automated and intelligent creation of virtual characters in the somatosensory game by providing a somatosensory dance game method for generating a virtual character based on the user's timbre.

[0007] To achieve the above-mentioned purpose, the embodiment of the present application provides a somatosensory dance game method for generating a virtual character based on the user's timbre, comprising:

[0008] When creating a virtual game character, activate the preset voice assistant to interact with the user;

[0009] Collect voice data when users communicate with voice assistants;

[0010] Generating a timbre model that matches the user's timbre based on the voice data and the pre-trained machine learning model;

[0011] The voice and image of the virtual character are generated based on the timbre model.

[0012] In one embodiment, activating a preset voice assistant to interact with a user includes:

[0013] The voice assistant conducts preliminary communication with the user in a preset tone and a first preset text, wherein the preset tone and the first preset text are designed for the purpose of building user trust and / or eliminating the user's vigilance;

[0014] Collecting voice data of the user during the initial communication process, and analyzing the user's tone characteristics based on the voice data;

[0015] When the user's voice data in the initial communication meets the preset conditions, the voice assistant further communicates with the user based on the analyzed user tone characteristics and the second preset text, where the second preset text is designed for the purpose of guiding the user to express his emotions.

[0016] In one embodiment, the preset condition refers to judging that the user is in a relaxed state based on the voice data of the user in the preliminary communication, and the judgment of the relaxed state includes at least one of the following characteristics:

[0017] The user's speaking speed is maintained within the predetermined speaking speed range;

[0018] The user's pitch is maintained within the preset pitch range;

[0019] The user's tone of voice changes at a frequency lower than a preset tone threshold;

[0020] The user's voice rhythm is maintained within the preset voice range;

[0021] The emotion recognition algorithm detects that the emotional characteristics in the user's voice meet the preset standards for a calm or pleasant state.

[0022] In one embodiment, the voice assistant further communicates with the user based on the analyzed user voice characteristics and the second preset text, including:

[0023] Asking the user open questions or guiding topics based on the user's tone characteristics and the second preset text;

[0024] Collecting the user's voice answers to the open-ended questions or guiding topics, and analyzing the user's emotional state;

[0025] Adaptively respond to users' emotional expressions based on the analysis results.

[0026] In one embodiment, before the voice assistant further communicates with the user based on the analyzed user voice tone feature and the second preset text, the method further includes:

[0027] According to the tone characteristics of the user, a second preset text adapted to the current emotional state and tone characteristics is selected from a plurality of pre-designed text libraries; or

[0028] Based on the user's answers and tone characteristics in the initial communication stage, the second preset text is generated using natural language generation technology.

[0029] In one embodiment, activating a preset voice assistant to interactively communicate with a user further includes:

[0030] When the voice assistant further communicates with the user, when it detects that the user's voice data based on the preset emotional theme meets the preset requirements, the logic of ending the communication is triggered;

[0031] After triggering the logic of ending the communication, the voice assistant guides the user to the next game session based on the third preset text and ends the current communication with the user.

[0032] In one embodiment, generating a timbre model that matches the user's timbre based on the voice data and a pre-trained machine learning model includes:

[0033] Preprocessing the collected user voice data, wherein the preprocessing includes cleaning background noise and removing non-voice components, and extracting key timbre features of the user voice;

[0034] Input the preprocessed speech features into the pretrained machine learning model;

[0035] Based on the user's voice features and the output of the pre-trained model, a timbre model that matches the user's timbre is generated.

[0036] In one embodiment, generating the voice and image of the virtual character based on the timbre model includes:

[0037] Through text-to-speech conversion technology, the text or game script input by the user is combined with the timbre model to generate a virtual character voice that matches the user's timbre characteristics;

[0038] Dynamically adjust and generate the emotional expression of the virtual character's voice according to the current situation and game scene of the virtual character;

[0039] Select a virtual character template that matches the user's voice from the preset virtual character library;

[0040] The timbre features in the timbre model are mapped to the visual features of the virtual character template to generate an image of the virtual character.

[0041] To achieve the above-mentioned objectives, an embodiment of the present application also proposes a somatosensory dance game device for generating a virtual character based on the user's timbre, comprising a memory, a processor, and a somatosensory dance game program for generating a virtual character based on the user's timbre stored in the memory and executable on the processor, wherein when the processor executes the somatosensory dance game program for generating a virtual character based on the user's timbre, a somatosensory dance game method for generating a virtual character based on the user's timbre as described in any one of the above items is implemented.

[0042] To achieve the above-mentioned objectives, an embodiment of the present application also proposes a computer-readable storage medium, on which is stored a somatosensory dance game program for generating a virtual character based on the user's timbre. When the somatosensory dance game program for generating a virtual character based on the user's timbre is executed by a processor, a somatosensory dance game method for generating a virtual character based on the user's timbre as described in any of the above items is implemented.

[0043] The somatosensory dance game method of the technical solution of the present application, through the interactive communication between the voice assistant and the user, the system can effectively collect the user's voice data, and use the pre-trained machine learning model to generate a timbre model that matches the user's timbre, thereby automatically generating the voice and image of the virtual character that matches the user's timbre. Through the generation of this personalized timbre model, the voice expression of the virtual character is more natural and realistic, which significantly improves the user's sense of immersion and helps to improve the user's participation and interactivity in the somatosensory dance game. In addition, this method not only eliminates the cumbersome process of manually inputting or selecting preset parameters in traditional somatosensory games and virtual reality technology, but also significantly improves the personalization and realism of the virtual character. In addition, by automatically capturing and analyzing the user's voice data, the character creation process is made more natural and senseless, thereby greatly improving the user's interactive experience and immersion. It can be seen that the somatosensory dance game of the technical solution of the present application has the advantages of creating virtual characters intelligently and personalized. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0045] Figure 1 It is a module structure diagram of an embodiment of a somatosensory dance game device for generating a virtual character based on the user's timbre according to the present invention;

[0046] Figure 2 The figure is a flow chart of an embodiment of a method for generating a virtual character through a somatosensory dance game based on user timbre according to the present invention.

[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0049] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0050] It should be noted that in the claims, any reference symbols placed between brackets shall not be constructed as limitations to the claims. The presence of "comprising" in the text does not exclude the presence of components or steps not listed in the claims. The quantifier "one" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware comprising several different components and by means of appropriately programmed computers. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of "first", "second", and "third" etc. does not indicate any order, and these words may be interpreted as names.

[0051] like Figure 1 As shown, Figure 1 It is a schematic diagram of the structure of a server 1 (also called a somatosensory dance game device for generating a virtual character based on user timbre) of a hardware operating environment involved in an embodiment of the present invention.

[0052] The server of the embodiment of the present invention is a device with display function, such as "Internet of Things devices", smart air conditioners, smart lights, smart power supplies with networking functions, AR / VR devices with networking functions, smart speakers, self-driving cars, PCs, smart phones, tablet computers, e-book readers, portable computers, etc.

[0053] like Figure 1 As shown, the server 1 includes: a memory 11, a processor 12 and a network interface 13.

[0054] The memory 11 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the server 1, such as a hard disk of the server 1. In other embodiments, the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc., equipped on the server 1.

[0055] Furthermore, the memory 11 may include an internal storage unit of the server 1 as well as an external storage device. The memory 11 may be used not only to store application software and various data installed on the server 1, such as the code of the somatosensory dance game program 10 for generating a virtual character based on the user's timbre, but also to temporarily store data that has been output or is to be output.

[0056] In some embodiments, the processor 12 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, which is used to run the program code or process data stored in the memory 11, such as executing a somatosensory dance game program 10 that generates a virtual character based on the user's timbre.

[0057] The network interface 13 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and is generally used to establish a communication connection between the server 1 and other electronic devices.

[0058] The network may be the Internet, a cloud network, a wireless fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), and / or a metropolitan area network (MAN). Various devices in the network environment may be configured to connect to the communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocol, and / or Bluetooth (Blue Tooth) communication protocol or a combination thereof.

[0059] Optionally, the server may further include a user interface, which may include a display, an input unit such as a keyboard, and an optional user interface may further include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be referred to as a display screen or a display unit, which is used to display information processed in the server 1 and to display a visual user interface.

[0060] Figure 1 Only the server 1 having components 11-13 and a somatosensory dance game program 10 for generating a virtual character based on the user's timbre is shown. It can be understood by those skilled in the art that Figure 1 The structure shown does not constitute a limitation on the server 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0061] In this embodiment, the processor 12 can be used to call the somatosensory dance game program for generating a virtual character based on the user's timbre stored in the memory 11, and perform the following operations:

[0062] When creating a virtual game character, activate the preset voice assistant to interact with the user;

[0063] Collect voice data when users communicate with voice assistants;

[0064] Generating a timbre model that matches the user's timbre based on the voice data and the pre-trained machine learning model;

[0065] The voice and image of the virtual character are generated based on the timbre model.

[0066] Based on the hardware architecture of the somatosensory dance game device for generating a virtual character based on the user's timbre, an embodiment of the somatosensory dance game method for generating a virtual character based on the user's timbre of the present invention is proposed. The somatosensory dance game method for generating a virtual character based on the user's timbre of the present invention aims to realize the automated and intelligent creation of virtual characters in somatosensory games.

[0067] Reference Figure 2 , Figure 2 This is an embodiment of a method for generating a virtual character based on user timbre in a somatosensory dance game according to the present invention. The method for generating a virtual character based on user timbre in a somatosensory dance game according to the present invention comprises the following steps:

[0068] S10: When creating a virtual game character, activate a preset voice assistant to interact with the user.

[0069] Among them, the voice assistant refers to an intelligent voice interaction system built into the game that can interact with users in natural language. The assistant has the ability to understand and respond to user voice input and is designed to guide users through the character creation process through voice.

[0070] Specifically, when the user selects the option to create a character, the system automatically starts the voice assistant. The voice assistant greets the user in a soft and friendly tone, such as "Hello, welcome to the character creation session. I will help you customize your unique character." Then, the voice assistant communicates with the user through guiding questions, such as "What kind of music do you like?", "What type of dance do you prefer?", etc. These questions can not only help users better understand the character creation process, but also provide the necessary interactive basis for voice data collection in subsequent steps.

[0071] S20: Collecting voice data when the user communicates with the voice assistant.

[0072] Among them, voice data refers to the voice signals generated by users when interacting with voice assistants. These data include characteristics such as the user's speaking speed, pitch, timbre, intonation, and emotional expression, which are used for subsequent timbre model generation.

[0073] Specifically, when the user answers the questions asked by the voice assistant through voice, the system will automatically collect these voice inputs. The system can record the user's voice through the microphone device and process it in real time in the background. The processing includes filtering background noise, removing irrelevant sound interference, and extracting key voice features. The extracted voice features will be saved in the system to provide basic data for generating the user's personalized timbre model.

[0074] S30. Generate a timbre model that matches the user's timbre based on the voice data and the pre-trained machine learning model.

[0075] Among them, the pre-trained machine learning model refers to a neural network or other machine learning model that has been trained with a large amount of voice data, which can recognize, extract and generate timbre features.

[0076] A timbre model refers to a mathematical model or algorithm that is generated based on the user's voice characteristics and can be used to simulate and reproduce the user's unique timbre.

[0077] Specifically, the system first inputs the collected user voice data into a pre-trained machine learning model. The model analyzes the voice data to extract features such as the user's pitch, timbre brightness, and intonation changes. Then, based on these features, the model generates a timbre model that matches the user's timbre. This timbre model can capture the uniqueness of the user's voice and provide a basis for generating the voice of the virtual character.

[0078] S40: Generate the voice and image of the virtual character based on the timbre model.

[0079] Among them, the voice of the virtual character refers to the vocal characteristics of the virtual character generated based on the user's timbre model. This voice will be used for the virtual character's dialogue, performance and interaction in the game.

[0080] The image of a virtual character refers to the appearance of the character in the game, including facial features, body shape, hairstyle, clothing, etc. These image features are partly generated based on the user's timbre model to achieve consistency between vision and hearing.

[0081] Specifically, the system first uses the timbre model to generate the voice of the virtual character through text-to-speech (TTS) technology. For example, when a user enters a text or the character needs to speak according to the game plot, the system will call the timbre model to convert the text into a voice similar to the user's timbre. At the same time, the system will also generate the appearance of the virtual character based on the timbre characteristics in the timbre model. For example, the system can match high-pitched tones with lighter body shapes or bright clothing, while low-pitched tones may correspond to more stable body shapes or dark clothing. The generated virtual character image can be displayed through 3D modeling tools and applied in real time in the game.

[0082] It can be understood that the somatosensory dance game method of the technical solution of the present application, through the interactive communication between the voice assistant and the user, the system can effectively collect the user's voice data, and use the pre-trained machine learning model to generate a timbre model that matches the user's timbre, thereby automatically generating the voice and image of the virtual character that matches the user's timbre. Through the generation of this personalized timbre model, the voice expression of the virtual character is more natural and realistic, which significantly improves the user's sense of immersion and helps to improve the user's participation and interactivity in the somatosensory dance game. In addition, this method not only eliminates the cumbersome process of manually inputting or selecting preset parameters in traditional somatosensory games and virtual reality technology, but also significantly improves the personalization and realism of the virtual character. In addition, by automatically capturing and analyzing the user's voice data, the character creation process is made more natural and senseless, thereby greatly improving the user's interactive experience and immersion. It can be seen that the somatosensory dance game of the technical solution of the present application has the advantages of creating virtual characters intelligently and individually.

[0083] In some embodiments, activating a preset voice assistant to interact with the user includes:

[0084] S11. The voice assistant conducts preliminary communication with the user in a preset tone and a first preset text, wherein the preset tone and the first preset text are designed for the purpose of establishing user trust and / or eliminating the user's vigilance.

[0085] Among them, the preset tone refers to the tone characteristics adopted by the voice assistant in the initial communication, which is usually soft, friendly, and kind, aiming to make users feel relaxed and trusting.

[0086] The first preset text refers to the text content used by the voice assistant in initial communication. It is designed to be simple, easy and stress-free, with the aim of establishing initial trust with the user and eliminating the user's vigilance.

[0087] Specifically, when the user starts the virtual character creation process, the voice assistant first greets the user in a friendly and relaxed tone, such as "Hello! Welcome to our game world. We will create a unique virtual character together." Then, the voice assistant asks simple, stress-free open-ended questions through the designed first preset text, such as "Have you played any interesting games recently?" or "What type of music do you like?" The purpose of these questions is to make the user feel relaxed and naturally enter a communication state, thereby laying the foundation for subsequent voice data collection and analysis.

[0088] S12: Collecting the voice data of the user during the initial communication process, and analyzing the tone characteristics of the user based on the voice data.

[0089] Among them, the tone feature refers to the user's current emotional state and expression, such as relaxation, tension, pleasure or calmness, extracted and analyzed from the user's voice data.

[0090] Specifically, when the user responds to the initial communication questions raised by the voice assistant, the system collects the user's voice data in real time through the microphone. The system processes this data, removes background noise and extracts key voice features. These features will be input into the tone analysis module. The system uses a pre-trained model or algorithm to analyze the user's tone characteristics and identify the user's current emotional state (such as whether he is in a relaxed state) to provide a basis for the next step of communication.

[0091] Optionally, the following steps may be performed to obtain the user's tone of voice features from the voice data:

[0092] 1. Speech data preprocessing: Preprocessing includes but is not limited to noise filtering and speech segmentation;

[0093] 2. Extract basic speech features:

[0094] 2.1. Extract basic speech features: Extract basic speech features from the preprocessed speech data, including speaking speed, pitch, volume, intonation, etc.

[0095] 2.2 Extraction of high-level emotional features: Extract high-level speech features related to the user's emotional state, such as intonation fluctuations, pitch change rate, timbre brightness, etc. These features can reflect the user's emotional state. At this point, emotion recognition algorithms can be used to analyze the emotional dimension of speech based on features such as Mel-frequency cepstral coefficients (MFCCs) and formants. For example, happy speech usually has a higher pitch and faster speaking speed, while angry speech may show a louder volume and irregular intonation changes.

[0096] 3. Sentiment analysis: The extracted speech features are input into the pre-trained sentiment analysis model to classify the user's speech features into different sentiment categories and output the corresponding sentiment scores or probability distributions. These results reflect the user's current emotional state and tone characteristics.

[0097] S13. When the user's voice data in the initial communication meets the preset conditions, the voice assistant further communicates with the user based on the analyzed user tone characteristics and the second preset text, where the second preset text is designed for the purpose of guiding the user to express emotions.

[0098] Among them, the preset condition refers to the system's judgment on whether the user's voice data in the initial communication meets specific requirements, such as whether the user's voice shows sufficient relaxation or positive emotions, and the tone feature analysis results meet the predetermined standards.

[0099] The second preset text refers to the text content used by the voice assistant in further communication. It is designed to guide users to express deeper emotions and usually includes open-ended questions or emotion-triggering topics.

[0100] Specifically, after the voice data in the user's initial communication meets the preset conditions, the voice assistant will automatically switch to a more in-depth communication mode. For example, if the system detects that the user's tone characteristics show a relaxed and calm state, the voice assistant may say: "It sounds like you are very interested in this type of game. Can you share with me your favorite game moment?" This open-ended question is designed to guide the user to express emotions, thereby collecting richer voice data and emotional features. These further communication data will generate a more accurate timbre model for the system, and ultimately be used to create a virtual character voice and image that is highly consistent with the user's emotional state.

[0101] In some embodiments, the preset condition refers to judging that the user is in a relaxed state based on the voice data of the user in the preliminary communication, and the judgment of the relaxed state includes at least one of the following characteristics:

[0102] 1. The user's speaking speed is maintained within the preset speaking speed range.

[0103] The speech rate refers to the number of words or syllables pronounced by the user per minute during the communication process. The predetermined speech rate range is usually set to a numerical range that reflects the natural conversation of the user, such as 100 to 150 words per minute.

[0104] Specifically, when the user has an initial communication with the voice assistant, the system will calculate the user's speaking speed in real time. If the user's speaking speed is always maintained within a predetermined range (for example, 100 to 150 words per minute), the system will determine that the user may be in a relaxed state. The stability of this speaking speed range usually indicates that the user is communicating with the voice assistant in a natural and relaxed manner, rather than speeding up or slowing down due to tension or anxiety.

[0105] 2. The user's pitch is maintained within the preset pitch range.

[0106] Among them, pitch refers to the height of the fundamental frequency in the user's voice, which is usually related to the user's emotions and physiological state. The preset pitch range is a reasonable range set according to the user's normal speaking pitch, such as 85 to 180 Hz.

[0107] Specifically, during the initial communication, the system analyzes the fundamental frequency of the user's voice to determine whether the pitch is maintained within a preset range. If the pitch remains stable and there is no obvious drastic fluctuation, the system will assume that the user is likely to be in a relaxed state. This is because when the user is relaxed, the pitch tends to be stable, and the pitch will not rise or fall as when nervous.

[0108] 3. The user's tone change frequency is lower than the preset tone threshold.

[0109] The intonation change frequency refers to the frequency of changes in the user's intonation during communication, that is, the number of ups and downs of the voice. The preset intonation threshold is a standard that limits the number of intonation changes, for example, no more than 3 times per minute.

[0110] Specifically, when analyzing the user's voice data, the system calculates the frequency of voice changes. If the user's voice change frequency is lower than a preset threshold (such as no more than 3 times per minute), the system will determine that the user's voice is stable, suggesting that the user is in a relaxed state. A lower frequency of voice changes usually indicates that the user is emotionally stable and does not frequently adjust the voice due to anxiety or excitement.

[0111] 4. The user's voice rhythm is maintained within the preset voice range;

[0112] Among them, speech rhythm refers to the speed, pauses and continuity of the user's speech during communication. The preset speech range usually reflects the speech rhythm characteristics in natural conversation, including the uniformity and coherence of speech.

[0113] Specifically, the system determines whether the speech rhythm is within the preset range by analyzing indicators such as pause intervals and pronunciation continuity in the user's speech. If the user's speech rhythm is even and there is no obvious rush or interruption, the system will think that the user may be in a relaxed state. This kind of rhythm balance is usually a manifestation of relaxation and natural communication.

[0114] 5. Through the emotion recognition algorithm, the emotional characteristics in the user's voice meet the preset standards of calm or happy state.

[0115] Among them, the emotion recognition algorithm refers to an algorithm model that judges the user's emotional state by analyzing voice features (such as speaking speed, pitch, intonation, etc.). The preset standard is based on the score of calm or pleasant emotions given by the emotion recognition model, which is usually set to a score higher than 0.7.

[0116] Specifically, the system uses an emotion recognition algorithm to process and analyze the user's voice data. If the emotion recognition model scores the user's current emotional state higher than 0.7 and shows a calm or happy state, the system will consider the user to be in a relaxed state. This score usually reflects the user's overall emotional state, ensuring the naturalness and comfort of communication.

[0117] It can be understood that by establishing user trust and eliminating vigilance through initial communication with the voice assistant, the system can collect the user's voice data and analyze its tone characteristics in a natural and relaxed environment, thereby accurately judging the user's emotional state. Based on this, the voice assistant further uses text to guide emotional expression to communicate with the user, ensuring that the collected voice data not only covers basic voice features, but also includes the user's emotional characteristics, thereby generating a more personalized and realistic virtual character voice and image, and can also capture voice features that are more in line with the user's tone. In addition, this multi-stage interactive method makes the character creation process more natural and smooth, significantly improving the user's interactive experience and immersion.

[0118] In some embodiments, the voice assistant further communicates with the user based on the analyzed user voice characteristics and the second preset text, including:

[0119] S131. Asking the user an open-ended question or a guiding topic based on the user's tone of voice and the second preset text.

[0120] Among them, open questions are those that have no fixed answers, which allow users to answer freely based on their own experience, feelings and interests. Such questions usually start with words such as "why", "how", "can you share it", etc.

[0121] For example, during the avatar creation process, the voice assistant might ask, “What kind of role do you usually like to play in games?” or “Which game scenarios are you most interested in?” There is no right or wrong answer to these questions, and users are free to answer based on their preferences and experience.

[0122] Guiding topics refer to specific topics raised by voice assistants, which are intended to guide users to have a deeper discussion or emotional expression around the topic. Such topics are usually related to the user's interests or previous answers, with the purpose of stimulating the user's emotional response or exploring deeper content.

[0123] For example, if a user shows interest in a certain game character during initial communication, the voice assistant may further ask: "What is your favorite game character? What is special about this character in your heart?" This kind of topic guides users to think deeply and express their feelings or opinions about a specific character.

[0124] Open-ended questions and guided topics can encourage users to provide longer and more natural responses. This open communication method can capture more diverse voice features, such as speaking speed, pitch changes, and emotional expressions in different contexts. These rich voice data provide more material for subsequent analysis and model training of the system, helping to generate a more accurate timbre model.

[0125] In addition, open-ended questions and guiding topics provide the system with an opportunity to analyze user feedback in real time. Through the user's answer, the system can judge the user's emotional changes and adjust the subsequent communication strategy accordingly. If the user shows strong interest or positive emotions, the system can continue to discuss the relevant topic in depth; if the user shows confusion or disinterest, the system can switch topics or adjust the tone to maintain the fluency of the conversation and the user's engagement.

[0126] Specifically, when the system analyzes that the user is currently in a relaxed state, the voice assistant may use the second preset text to ask open questions such as "It sounds like you are very interested in this game. Can you tell me about your favorite game character?" or "What strategies do you usually choose when playing this type of game?" In this way, the system aims to further understand the user's interests and emotional state, and provide more information for subsequent communication content.

[0127] S132: Collect the user's voice answers to the open-ended questions or guiding topics, and analyze the user's emotional state.

[0128] Among them, voice answers refer to the voice responses made by users to open questions or guiding topics raised by voice assistants. The voice data will include the user's detailed answers and emotional expressions.

[0129] Emotional state refers to the emotion expressed by the user during the answering process, which is obtained by analyzing the user's voice answer. For example, the user may show emotional states such as happiness, excitement, calmness or uncertainty.

[0130] Specifically, when the user answers the voice assistant's questions, the system collects the user's voice data in real time through the microphone. The system processes these voice data and extracts features such as speaking speed, pitch, volume, and intonation changes. Then, the system uses the pre-trained sentiment analysis model to analyze these voice features to identify the user's emotional state. For example, the system may recognize that the user shows obvious joy when talking about a game character, or shows confidence when describing a game strategy.

[0131] S133. Adaptively respond to the user's emotional expression based on the analysis results.

[0132] Adaptive responses refer to the targeted feedback that voice assistants give based on the user’s current emotional state. For example, voice assistants can use encouraging, affirmative, or mitigating words to keep the user’s emotional interaction in a positive state.

[0133] Specifically, when the user answers open-ended questions or guiding topics, the system continuously analyzes the user's emotional state in real time. This includes monitoring the user's tone, speaking speed, pitch, volume and other features to determine the user's current emotional state (such as pleasure, confusion, excitement, tension, etc.). This can use pre-trained emotion recognition models (such as RNN, LSTM, CNN, etc.) to generate emotion scores or classification results based on the user's voice features, and judge the changes in the user's emotions in real time.

[0134] Based on the results of sentiment analysis, the voice assistant generates adaptive response content. For example, when the user shows pleasure, the voice assistant can use positive feedback, such as "Glad that you like this topic!"; when the user shows confusion or hesitation, the voice assistant can provide comfort or guidance, such as "This sounds a bit complicated, why don't we change to a lighter topic?". This can use natural language generation (NLG) technology to dynamically generate response text that matches the user's emotional state based on sentiment analysis results and predefined response templates, and output it through speech synthesis (TTS) technology.

[0135] For example, when the system analyzes that the user shows positive emotions (such as excitement or happiness) about a topic, the voice assistant may further encourage the user to discuss the topic in depth. For example, "This is an interesting strategy! How do you think this strategy will perform in different scenarios?" If the system detects that the user is confused or uncertain about a topic, the voice assistant can choose to adjust the topic or provide helpful suggestions, such as "It sounds like this question is a bit tricky, why don't we talk about other interesting content." Through this adaptive response, the system can ensure that the interaction with the user is always natural and smooth, and enhance the user's sense of participation.

[0136] It can be understood that the system can more flexibly adjust the content of interaction with users by asking open questions or guiding topics based on the user's tone characteristics through the voice assistant, and collecting and analyzing the user's voice answers in real time, thereby identifying the user's emotional state and making adaptive responses accordingly. This dynamic adjustment not only makes communication more natural and smooth, but also captures the user's emotional characteristics more deeply and enhances the user's sense of participation and satisfaction. In addition, this multi-stage interaction method helps to enhance the emotional connection between users and virtual characters, thereby significantly enhancing the user's immersion and interactive experience in the somatosensory dance game.

[0137] In some embodiments, before the voice assistant further communicates with the user based on the analyzed user voice tone feature and the second preset text, the method further includes:

[0138] S110 . Selecting a second preset text that is compatible with the current emotional state and tone characteristics from a plurality of pre-designed text libraries according to the tone characteristics of the user.

[0139] Specifically, when the system analyzes that the user shows a happy and relaxed emotional state in the initial communication, the system will select the conversation content that matches this state from a specially designed text library. The text library may contain questions related to the user's interests, humorous comments, or positive feedback. For example, if the user expresses interest in a certain game style, the system may select "You have a deep understanding of this game style. Can you share your favorite part?" as the second preset text for further communication. This selection ensures that the subsequent communication content can be consistent with the user's emotional state and further stimulate the user's interest in participation.

[0140] It can be understood that by analyzing the user's tone characteristics and selecting a second preset text that is suitable for the user's current emotional state and tone characteristics from a pre-designed plurality of text libraries, the system can provide a dialogue content that highly matches the user's emotions in the communication between the user and the voice assistant. Therefore, the system can effectively enhance the user's emotional participation and ensure the naturalness and fluency of the dialogue process. In addition, by selecting text that matches the user's emotional state, the system can better guide the user to express his emotions, thereby improving the personalization and realism of the virtual character.

[0141] In some embodiments, before the voice assistant further communicates with the user based on the analyzed user voice tone feature and the second preset text, the method further includes:

[0142] S120: Generate a second preset text using natural language generation technology based on the user's answer and tone characteristics in the initial communication stage.

[0143] Among them, natural language generation technology (NLG) refers to the technology of automatically generating natural language text through algorithms and models, aiming to create customized conversation content based on specific input conditions.

[0144] Specifically, when the user gives specific answers in the initial communication, the system will analyze these answers and the user's tone characteristics to determine the user's interests and emotional state. Based on this information, the system uses natural language generation technology to dynamically generate dialogue text that is highly matched to the user's current state. For example, if the user expresses interest in the complex background of a certain character in the initial communication, and the voice shows curiosity and excitement, the system may generate a customized dialogue content, such as "It sounds like you are very interested in the background story of this character. What impact do you think this character's growth experience will have on the progress of the game?" In this way, the system can ensure that the content of the dialogue is not only closely related to the user's interests, but also can guide the user to further express emotions and opinions.

[0145] It can be understood that based on the user's answers and tone characteristics in the initial communication stage, the system uses natural language generation technology to generate a personalized second preset text. The system can dynamically generate highly customized dialogue content based on the user's specific feedback and emotional state. Therefore, the system can more accurately match the user's interests and emotional needs, significantly improving the user's sense of participation and satisfaction in the interaction. In addition, by generating personalized text, the system can guide users to explore specific topics in depth, further enhancing the personalized performance of virtual characters and the user's immersive experience.

[0146] In some embodiments, activating a preset voice assistant to interactively communicate with a user further includes:

[0147] S210. When the voice assistant further communicates with the user, when it is detected that the user's voice data based on the preset emotional theme meets the preset requirements, the logic of ending the communication is triggered.

[0148] Among them, preset emotional topics refer to specific topics or emotional categories related to user emotional expression that are pre-defined by the system, such as pleasure, satisfaction, confusion, anxiety, etc. When the user communicates with the voice assistant, the system focuses on the expression of these emotional topics.

[0149] Preset requirements refer to the conditions or standards set by the system to determine whether the user has fully expressed a specific emotional theme. These conditions can include the intensity of specific emotions in the user's voice data, the consistency of tone characteristics, the length of communication, the amount of voice data, etc.

[0150] Specifically, when a user communicates with a voice assistant around a preset emotional topic (such as "your favorite game character"), the system will continuously monitor the user's voice data. If the system detects that the user's emotional expression has met the preset requirements, such as the user continues to show satisfaction or pleasure in multiple answers, or the communication has reached a predetermined length of time, the system will trigger the logic to end the communication. This judgment ensures that the user's communication on a certain emotional topic has reached the level of full expression, avoiding repetition or fatigue caused by excessive communication.

[0151] S220. After triggering the logic for ending the communication, the voice assistant guides the user to the next game session based on the third preset text and ends the current communication with the user.

[0152] The third preset text refers to the conversation content pre-designed by the system to end the current communication and guide the user to the next step. This text is intended to provide users with clear guidance so that they can naturally transition from the current communication to the next part of the game.

[0153] Ending the current communication with the user means that the voice assistant ends the conversation with the user through an appropriate ending sentence and prepares for the next step.

[0154] Specifically, when the system detects that the user's emotional expression meets the preset requirements and triggers the logic of ending the communication, the voice assistant will use the third preset text to have a final conversation with the user. For example, the voice assistant may say: "Thank you very much for sharing the wonderful content! Now let's move on to the next interesting part." The voice assistant then guides the user to the next part of the game, such as character selection, scene exploration, or the beginning of a mission. In this way, the system can ensure the natural end of the communication while maintaining the continuity of the game process and the smoothness of the user experience.

[0155] It can be understood that, during the further communication between the voice assistant and the user, the user's voice data based on the preset emotional theme can be monitored in real time, and the logic of ending the communication can be triggered when the preset requirements are met. Therefore, the system can ensure that the user ends the current conversation naturally when the emotional expression is sufficient. In addition, by using the third preset text to guide the user to the next game session after the end logic is triggered, the system can ensure the natural transition of communication and the continuity of the game process, thereby significantly improving the user's sense of participation and overall experience.

[0156] In some embodiments, generating a timbre model that matches the user's timbre based on the voice data and a pre-trained machine learning model includes:

[0157] S31, preprocessing the collected user voice data, wherein the preprocessing includes cleaning background noise and removing non-voice components, and extracting key timbre features of the user voice.

[0158] Among them, background noise cleaning refers to the use of audio processing technology to remove environmental noise and interference in voice data that may affect the analysis, ensuring the purity of the voice signal.

[0159] Non-speech component removal refers to removing irrelevant parts of the speech data, such as silent segments, breathing sounds, coughing sounds, etc., to ensure that only the user's valid speech information is included in the analysis.

[0160] Key timbre feature extraction refers to extracting features that are crucial to timbre modeling from preprocessed speech data, such as pitch, timbre, prosody, formants, etc.

[0161] Specifically, the system first filters the background noise of the user's voice data, using filters (such as low-pass filters, high-pass filters) or other noise suppression algorithms to remove background sounds in the environment. Then, the system identifies and removes non-speech components in the voice signal, such as silence, breathing sounds, etc. Next, the system uses audio analysis tools to extract key timbre features. For example, the system can extract pitch through autocorrelation methods or Fourier transforms, and extract timbre features through linear predictive coding (LPC) or Mel-frequency cepstral coefficients (MFCCs).

[0162] S32. Input the preprocessed speech features into the pre-trained machine learning model.

[0163] Among them, the pre-trained machine learning model refers to a model that has been trained with a large amount of voice data, which can recognize and process voice features and generate corresponding timbre representations. The model can be a deep neural network (such as a convolutional neural network CNN, a long short-term memory network LSTM) or other machine learning algorithms suitable for processing voice data.

[0164] Specifically, the system uses the timbre features extracted from S31 as input data into the pre-trained machine learning model. The model further processes and analyzes these features through its hierarchical structure to identify the user's unique timbre characteristics. For example, the model may analyze the pitch change pattern, the brightness of the timbre, the rhythm of the intonation, etc., to generate a high-dimensional feature vector or parameter set that matches the user's voice characteristics.

[0165] S33. Generate a timbre model that matches the user's timbre based on the user's voice features and the output of the pre-trained model.

[0166] Specifically, the system matches and optimizes the output of the pre-trained model (such as high-dimensional feature vectors) with the user's original voice features to generate a timbre model. The timbre model may contain multiple parameters, such as pitch range, timbre adjustment coefficient, resonance peak frequency, etc., which are used to describe the key characteristics of the user's voice. The generated timbre model can not only reflect the user's natural voice characteristics, but can also be used in the subsequent text-to-speech (TTS) process to ensure that the voice of the virtual character is highly consistent with the user's real timbre. After the model is generated, the system can verify and fine-tune it by comparing the similarity between the original voice data and the generated voice to improve the accuracy of the model and the user's matching degree.

[0167] It can be understood that by preprocessing the collected user voice data, extracting key timbre features, and inputting them into the pre-trained machine learning model for analysis and generation, the system can generate a timbre model that highly matches the user's timbre. This timbre model can accurately reflect the user's voice characteristics, thereby providing a basis for the generation of the virtual character's voice, ensuring that the virtual character's voice performance is natural and realistic, and significantly enhancing the user's immersion and personalized experience. In addition, the generated timbre model further improves the similarity with the user's voice through matching and optimization, enhancing the emotional connection between the virtual character and the user.

[0168] In some embodiments, generating the voice and image of the virtual character based on the timbre model includes:

[0169] S41. Through text-to-speech conversion technology, the text or game script input by the user is combined with the timbre model to generate a virtual character voice that matches the user's timbre characteristics.

[0170] Among them, the text input by the user or the game script refers to the text content directly input by the user or preset in the game scenario, and the system needs to convert these contents into the voice of the virtual character.

[0171] Specifically, when a user enters text in the game (such as dialogue content) or when the game script requires a character to speak, the system will call TTS technology. The TTS system will use the previously generated timbre model to convert the text content into voice output consistent with the user's voice characteristics. The parameters in the timbre model (such as pitch, intonation, and timbre) will directly affect the characteristics of the generated voice, thereby ensuring that the voice of the virtual character is highly similar to the user's real voice and providing a personalized voice experience.

[0172] S42. Dynamically adjust and generate the emotional expression of the virtual character's voice according to the current situation and game scene of the virtual character.

[0173] Among these, the virtual character's context and game scene refer to the character's current game context (such as dancing, stage performances, interaction with other characters, etc.) and specific game scenes (such as stage, concert, rehearsal room, etc.). These factors determine the character's emotional expression method.

[0174] Dynamically adjusting the emotional expression of generated speech means that the system adjusts the intonation, pitch, speaking speed and other parameters of the speech in real time according to situational requirements to reflect the emotional state of the character in different dance scenes (such as excitement, joy, relaxation, etc.).

[0175] Specifically, the system takes into account the character's current dance context when generating the virtual character's voice. For example, in a vibrant dance performance scene, the system may adjust the voice's pitch and speed to make the voice express excitement and enthusiasm; while in a relaxed rehearsal scene, the voice will be calmer and more soothing. By adjusting the emotional expression of the voice in real time, the system ensures that the virtual character's voice performance is consistent with the dance scene and the user's emotional experience, thereby enhancing the game's immersion and the character's emotional depth.

[0176] S43: Select a virtual character template that matches the user's timbre from a preset virtual character library.

[0177] Among them, the preset virtual character library refers to the various virtual character templates stored in the system, which include different appearance features and basic designs for the system to select and customize. The virtual character template refers to the basic design framework of the virtual character, including the appearance features of the character (such as face shape, body shape, hairstyle, etc.) and basic visual elements.

[0178] Specifically, the system will select the most matching character template in the virtual character library based on the user's timbre characteristics (such as pitch, timbre brightness, intonation, etc.). For example, for a user with a high pitch and a brisk tone, the system may select a character template with a young and lively appearance; while for a user with a deeper timbre, the system may select a character template with a mature and stable appearance. Through this matching, the system ensures that the appearance of the virtual character is consistent with the user's voice characteristics, so that the visual and auditory performance of the character are coordinated.

[0179] S44, mapping the timbre features in the timbre model to the visual features of the virtual character template to generate an image of the virtual character.

[0180] Among them, timbre feature mapping refers to the conversion of parameters in the timbre model (such as pitch, timbre brightness, intonation, etc.) into corresponding visual features (such as the character's facial contour, hairstyle color, clothing style, etc.).

[0181] The visual characteristics of a virtual character refer to the character's appearance design elements, including face shape, body shape, hairstyle, skin color, clothing, etc. These elements will be adjusted according to the timbre characteristics.

[0182] Specifically, the system associates and maps the characteristic values ​​of the timbre model (such as pitch, timbre brightness, intonation, volume, etc.) with the visual features of the virtual character template based on a preset mapping relationship. Pitch can be mapped to the body shape or facial contour of the virtual character, timbre brightness can be mapped to the character's hair color or clothing color, and intonation can be mapped to the character's expression style or decorative details.

[0183] For example, the following is a specific mapping relationship that can be selected:

[0184] Pitch and body type: A higher pitch might map to a thinner, lighter body type, while a lower pitch might map to a thicker, more solid body type.

[0185] Tone brightness and hair / clothing color: bright tones are mapped to bright hair colors or bright clothing, and dark tones are mapped to dark hair colors or low-key clothing.

[0186] Tone of voice and facial expressions / decorative details: Frequent changes in tone of voice can be mapped to richer, dynamic facial expressions or complex clothing details, while steady tones can be mapped to simpler, calmer expressions and designs.

[0187] Volume and character aura / posture: A louder volume may map to a character's more confident and domineering posture, while a lower volume may map to a character's gentler and more reserved temperament.

[0188] Furthermore, when performing actual mapping, the timbre features need to be normalized and converted into a numerical range suitable for mapping to visual features. For example, the pitch is converted into a range of 0 to 1, and the timbre brightness is also normalized accordingly.

[0189] After the standardization process is completed, the standardized timbre feature values ​​are input into a pre-designed mapping model, and the corresponding visual features are generated through the model. For example, the standardized value of pitch can directly affect the character's height parameter, and the timbre brightness can directly adjust the hue of the character's hair color. In specific implementation, a linear mapping or a rule-based mapping model can be used to convert the timbre feature values ​​into specific character appearance parameters, and these parameters are applied to the 3D model of the virtual character. Or the standardized value of the timbre brightness can be mapped to the RGB parameter of the color value to adjust the brightness of the hair color or clothing.

[0190] By mapping the timbre features in the timbre model to the visual features of the virtual character, the close combination of timbre features and visual features makes the performance of the virtual character more coordinated and consistent. This consistency reduces the user's cognitive conflict and makes the character's performance more natural and coherent, thereby improving the overall user experience.

[0191] It can be understood that by combining the user's timbre model with text-to-speech conversion technology, dynamically generating a virtual character voice that is consistent with the user's voice characteristics, and generating a matching virtual character image through timbre feature mapping, the system can provide the user with a highly personalized virtual character that closely matches the user's voice characteristics. Therefore, the user's immersion and emotional connection in the game are significantly enhanced, and the visual and auditory performance of the virtual character is also highly coordinated, improving the overall quality of the gaming experience.

[0192] In addition, the embodiment of the present invention also proposes a computer-readable storage medium, which can be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable storage medium includes a somatosensory dance game program 10 for generating a virtual character based on the user's timbre. The specific implementation of the computer-readable storage medium of the present invention is roughly the same as the specific implementation of the somatosensory dance game method for generating a virtual character based on the user's timbre and the server 1, which will not be repeated here.

[0193] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0195] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0196] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0197] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0198] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A somatosensory dance game method for generating a virtual character based on user timbre, characterized in that: include: When creating a virtual game character, activate the preset voice assistant to interact with the user; Collect voice data when users communicate with voice assistants; Generating a timbre model that matches the user's timbre based on the voice data and the pre-trained machine learning model; The voice and image of the virtual character are generated based on the timbre model.

2. The method for generating a virtual character based on user timbre according to claim 1, characterized in that: Activate the preset voice assistant to interact with the user, including: The voice assistant conducts preliminary communication with the user in a preset tone and a first preset text, wherein the preset tone and the first preset text are designed for the purpose of building user trust and / or eliminating the user's vigilance; Collecting voice data of the user during the initial communication process, and analyzing the user's tone characteristics based on the voice data; When the user's voice data in the initial communication meets the preset conditions, the voice assistant further communicates with the user based on the analyzed user tone characteristics and the second preset text, where the second preset text is designed for the purpose of guiding the user to express his emotions.

3. The method for generating a virtual character based on user timbre according to claim 2, characterized in that: The preset condition refers to judging that the user is in a relaxed state based on the voice data of the user in the preliminary communication, and the judgment of the relaxed state includes at least one of the following characteristics: The user's speaking speed is maintained within the predetermined speaking speed range; The user's pitch is maintained within the preset pitch range; The user's tone of voice changes at a frequency lower than a preset tone threshold; The user's voice rhythm is maintained within the preset voice range; The emotion recognition algorithm detects that the emotional characteristics in the user's voice meet the preset standards for a calm or pleasant state.

4. The method for generating a virtual character based on user timbre according to claim 2, characterized in that: The voice assistant further communicates with the user based on the analyzed user voice characteristics and the second preset text, including: Asking the user open questions or guiding topics based on the user's tone characteristics and the second preset text; Collecting the user's voice answers to the open-ended questions or guiding topics, and analyzing the user's emotional state; Adaptively respond to users' emotional expressions based on the analysis results.

5. The method for generating a virtual character based on user timbre according to claim 4, characterized in that: Before the voice assistant further communicates with the user based on the analyzed user voice tone feature and the second preset text, the method further includes: According to the tone characteristics of the user, a second preset text adapted to the current emotional state and tone characteristics is selected from a plurality of pre-designed text libraries; or Based on the user's answers and tone characteristics in the initial communication stage, the second preset text is generated using natural language generation technology.

6. The method for generating a virtual character based on user timbre according to claim 4, characterized in that: Activate the preset voice assistant to interact with the user, including: When the voice assistant further communicates with the user, when it detects that the user's voice data based on the preset emotional theme meets the preset requirements, the logic of ending the communication is triggered; After triggering the logic of ending the communication, the voice assistant guides the user to the next game session based on the third preset text and ends the current communication with the user.

7. The method for generating a virtual character based on user timbre according to any one of claims 1 to 6, characterized in that: Generating a timbre model that matches the user's timbre according to the voice data and the pre-trained machine learning model, including: Preprocessing the collected user voice data, wherein the preprocessing includes cleaning background noise and removing non-voice components, and extracting key timbre features of the user voice; Input the preprocessed speech features into the pretrained machine learning model; Based on the user's voice features and the output of the pre-trained model, a timbre model that matches the user's timbre is generated.

8. The method for generating a virtual character based on user timbre according to claim 7, characterized in that: Generating the voice and image of the virtual character based on the timbre model includes: Through text-to-speech conversion technology, the text or game script input by the user is combined with the timbre model to generate a virtual character voice that matches the user's timbre characteristics; Dynamically adjust and generate the emotional expression of the virtual character's voice according to the current situation and game scene of the virtual character; Select a virtual character template that matches the user's voice from the preset virtual character library; The timbre features in the timbre model are mapped to the visual features of the virtual character template to generate an image of the virtual character.

9. A somatosensory dance game device for generating a virtual character based on the user's timbre, characterized in that: The invention comprises a memory, a processor and a somatosensory dance game program for generating a virtual character based on the user's timbre, which is stored in the memory and can be run on the processor. When the processor executes the somatosensory dance game program for generating a virtual character based on the user's timbre, a somatosensory dance game method for generating a virtual character based on the user's timbre as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a somatosensory dance game program for generating a virtual character based on the user's timbre, and when the somatosensory dance game program for generating a virtual character based on the user's timbre is executed by a processor, a somatosensory dance game method for generating a virtual character based on the user's timbre as described in any one of claims 1-8 is implemented.

Images