Self-adaptive adjustment method for personalized role modeling

Abstract

The invention discloses a personalized role modeling adaptive adjustment method and device and a computer readable storage medium, and the method comprises the steps: processing music audio data, and extracting a static theme feature and a music time sequence emotion vector; acquiring and processing player voice signals to extract static age features and real-time player state vectors; performing weighted fusion on the music time sequence emotion vector and the real-time player state vector, and generating a fusion condition vector in combination with the static theme feature and the static age feature; inputting the fusion condition vector into a preset conditional generation model to generate a modeling parameter vector; and applying the modeling parameter vector to a basic character modeling model of the player to adjust the basic character modeling of the player in real time. According to the invention, while the identity recognition sense of the player is ensured, the deep and time sequence dynamic personalized immersion experience is realized.

Description

Technical Field

[0001] This invention relates to the field of virtual character design technology, and in particular to a method, device, and computer-readable storage medium for adaptive adjustment of personalized character appearance. Background Technology

[0002] In the fields of motion-sensing games, virtual character interaction, and immersive entertainment experiences, users' demands for personalized and dynamic virtual character appearances are increasing. Currently, mainstream virtual social platforms or game systems typically provide users with a rich library of character clothing, hairstyles, and makeup. Users mainly customize the appearance of their virtual characters through manual selection, purchases from the in-game store, or unlocking by completing specific tasks. This method forms the technological foundation for existing character personalization.

[0003] To enhance the dynamism of the experience, several attempts have emerged within the industry. Specifically, a common approach is music-driven animation generation, such as using multimodal frameworks to generate dance videos synchronized with music from static images. However, this approach primarily focuses on real-time rendering of motion sequences, neglecting dynamic adjustments to character design, such as materials or makeup, resulting in a disconnect between music and visual appearance. Another approach is audio-driven avatar customization, such as changing accessories based on the virtual environment context. However, this customization is often static, applied permanently once generated, and cannot respond to emotional fluctuations in the music or real-time user input.

[0004] In summary, existing technologies have significant shortcomings in the dynamic integration of music and character design, as well as in maintaining character continuity, failing to achieve precise and adaptive adjustments to the user's basic character appearance. Therefore, how to achieve a dual dynamic and personalized adaptive adjustment method that responds to changes in music temporal sequence and the player's real-time emotions, while maintaining the continuity of the player's basic character image, has become a pressing technical challenge for those skilled in the art. Summary of the Invention

[0005] This application provides a personalized character appearance adaptive adjustment method, aiming to solve the technical problems in the prior art such as the disconnect between virtual character appearance and music atmosphere, the disruption of the continuity of the player's basic image, and the static and monotonous adaptation effect.

[0006] To achieve the above objectives, embodiments of this application provide a personalized character appearance adaptive adjustment method, including:

[0007] Process music audio data to extract static thematic features that represent the overall style of the music, as well as temporal emotional vectors that represent the real-time fluctuations of the music.

[0008] Acquire and process player voice signals to extract static age features representing the player's physiological attributes, and real-time player state vectors representing the player's real-time emotional state;

[0009] The music temporal emotion vector and the real-time player state vector are weighted and fused together, and combined with the static theme features and the static age features to generate a fusion condition vector;

[0010] The fusion condition vector is input into a preset conditional generation model to generate a modeling parameter vector;

[0011] The modeling parameter vector is applied to the player's base character model to adjust the player's base character model in real time.

[0012] In one embodiment, processing music audio data to extract static thematic features representing the global style of the music, and a temporal emotional vector representing the real-time fluctuations of the music, includes:

[0013] Extract the Mel spectrogram of the music audio data;

[0014] The Mel spectrogram is processed using a pre-trained convolutional recurrent neural network to generate the music temporal sentiment vector; and

[0015] The metadata of the music audio data is processed by a pre-trained natural language processing model to generate the static topic features.

[0016] In one embodiment, the Mel spectrogram is processed by a pre-trained convolutional recurrent neural network to generate the music temporal sentiment vector, including:

[0017] Based on the Mel spectrogram, a time-series feature sequence is extracted within a predetermined time window;

[0018] Based on the temporal feature sequence, local features are extracted through the convolutional layer of the convolutional recurrent neural network, and temporal dependencies are captured through the recurrent layer of the convolutional recurrent neural network to generate intermediate sentiment representations.

[0019] Based on the intermediate sentiment representation, the intermediate sentiment representation is mapped through a fully connected layer to generate the music temporal sentiment vector, wherein the music temporal sentiment vector includes an arousal dimension value and a pleasure dimension value.

[0020] In one embodiment, acquiring and processing the player's voice signal to extract static age features representing the player's physiological attributes and a real-time player state vector representing the player's real-time emotional state includes:

[0021] Extract the Mel frequency cepstral coefficients, fundamental frequency, and formant frequency of the player's voice signal;

[0022] The static age feature is generated by processing the Mel frequency cepstral coefficients, the fundamental frequency, and the formant frequencies using a pre-trained deep neural network; and

[0023] The real-time acoustic features of the player's voice signal are extracted within a rolling time window, and the real-time acoustic features are processed by a pre-trained convolutional bidirectional long short-term memory network to generate the real-time player state vector.

[0024] In one embodiment, real-time acoustic features of the player's voice signal are extracted within a rolling time window, and the real-time acoustic features are processed through a pre-trained convolutional bidirectional long short-term memory network to generate the real-time player state vector, including:

[0025] Within the rolling time window, the spectroscopic and prosodic features of the player's voice signal are calculated to generate the real-time acoustic features;

[0026] Based on the real-time acoustic features, local patterns are extracted through the convolutional layers of the convolutional bidirectional long short-term memory network, and the temporal context before and after are fused through the bidirectional long short-term memory layers of the convolutional bidirectional long short-term memory network to generate real-time emotion embeddings.

[0027] Based on the real-time emotion embedding, the real-time emotion embedding is weighted through an attention mechanism to generate the real-time player state vector, wherein the real-time player state vector includes an arousal dimension value and a pleasure dimension value.

[0028] In one embodiment, the music temporal emotion vector and the real-time player state vector are weighted and fused, and combined with the static theme features and the static age features to generate a fusion condition vector, including:

[0029] Based on the real-time player state vector, the music contribution weight of the music temporal emotion vector and the player contribution weight of the real-time player state vector are calculated by a preset weight controller. The weight controller performs smoothing processing on the music feature sequence and player state sequence within the historical time window so that the music contribution weight and the player contribution weight have temporal continuity.

[0030] A weighted music temporal sentiment vector is generated by multiplying the music contribution weight by the music temporal sentiment vector.

[0031] A weighted real-time player state vector is generated by multiplying the player contribution weight by the real-time player state vector.

[0032] The weighted music temporal emotion vector, the weighted real-time player state vector, the static theme feature, and the static age feature are concatenated to generate the fusion condition vector.

[0033] In one embodiment, the fusion condition vector is input into a preset conditional generation model to generate a modeling parameter vector, including:

[0034] Based on the fusion condition vector, a noise vector is sampled from a standard normal distribution;

[0035] The encoder of the conditional generative model compresses the fusion conditional vector and the noise vector to generate a latent representation;

[0036] The latent representation is reconstructed by the decoder of the conditional generative model to generate the styling parameter vector, which includes at least one of clothing self-illumination intensity, clothing self-illumination color, clothing material UV roll speed, hair highlight intensity, makeup density, and accessory visibility.

[0037] In one embodiment, the modeling parameter vector is applied to the player's base character model to adjust the player's base character model in real time, including:

[0038] Extract the parameter values ​​from the shape parameter vector;

[0039] By using the shader script of the preset rendering engine, the parameter values ​​are set to the material properties and component properties of the basic character model, so as to adjust the material, lighting effects, makeup intensity and accessory visibility of the basic character model.

[0040] Based on the adjusted basic character model, the basic character model is rendered through the rendering engine to achieve real-time adjustment of the player's basic character model.

[0041] To achieve the above objectives, this application also proposes a personalized character appearance adaptive adjustment device, including a memory, a processor, and a personalized character appearance adaptive adjustment program stored in the memory and executable on the processor. When the processor executes the personalized character appearance adaptive adjustment program, it implements the personalized character appearance adaptive adjustment method as described in any of the above claims.

[0042] To achieve the above objectives, embodiments of this application also propose a computer-readable storage medium storing a personalized character appearance adaptive adjustment program, wherein the personalized character appearance adaptive adjustment program, when executed by a processor, implements the personalized character appearance adaptive adjustment method as described in any of the above claims.

[0043] The personalized character model adaptive adjustment method proposed in this application has the following beneficial effects:

[0044] 1. It ensures the continuity of character appearance and solves the technical problem of player identity discontinuity:

[0045] This application achieves adjustment rather than replacement of the player's basic character model by applying styling parameter vectors and using the rendering engine's shader scripts to set material properties and component properties (such as lighting effects, makeup intensity, or accessory visibility). This method overcomes the drawback of frequently replacing the entire character model due to music or scene changes, which leads to a disconnect in the player experience, thus ensuring that players maintain a sense of identity with their basic character while enjoying dynamic visual effects.

[0046] 2. It achieves a deep sense of immersive, dynamic temporal progression, overcoming the limitations of static adaptation:

[0047] This application utilizes pre-trained convolutional recurrent neural networks and other temporal sentiment analysis models to process music audio into temporal sentiment vectors representing the real-time fluctuations of the music. These temporal vectors (rather than static labels) are used to drive the generation of character appearance parameters, allowing the character's appearance (e.g., from a low-glow blue to a high-speed gold effect) to adjust synchronously with the emotional fluctuations of the music from verse to chorus. This overcomes the static matching defect where the player's image remains the same throughout the entire song, achieving a beat-by-beat dynamic fusion of auditory and visual elements, greatly enhancing immersion.

[0048] 3. It has created a highly personalized experience integrating humans, machines, and entertainment, filling the gap in multimodal fusion:

[0049] This application processes player voice signals in parallel, extracting static age features representing physiological attributes and real-time player state vectors representing immediate emotions. These two voiceprint features, along with the music temporal emotion vector, participate in the generation of a fusion conditional vector. This allows the final character design to not only respond to the music but also match the player's inherent identity (age) and real-time emotions (such as cheers). This overcomes the limitations of relying solely on a single modality or manual selection, achieving a deep, highly personalized adaptation.

[0050] 4. An intelligent and adaptive fusion strategy has been implemented, improving the system's flexibility and smoothness:

[0051] This application introduces a pre-defined weight controller to weight and fuse the music temporal emotion vector and the real-time player state vector. This weight controller dynamically adjusts the contribution weights of the music and the player based on the player's state (e.g., excitement) and performs smoothing based on a historical timeline window. This allows the driving source of appearance adjustments (music or player) to be dynamically switched, and the visual transitions have temporal continuity, resulting in a smarter strategy and smoother effects.

[0052] 5. It enhances the creativity and automation of shape generation, overcoming the limitations of static rule bases:

[0053] This application employs a pre-trained conditional generative model that maps fused conditional vectors to styling parameter vectors (such as clothing luminescence intensity and UV rolling speed). By introducing noise vectors for sampling, this model can automatically learn complex mapping relationships and produce diverse results. This overcomes the shortcomings of relying on manually exhaustively enumerated label-styling rule bases, freeing it from rigid matching constraints. While achieving full automation, it continuously brings players a creative and unpredictable sense of style surprise. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0055] Figure 1 This is a module structure diagram of an embodiment of the personalized character model adaptive adjustment device of the present invention;

[0056] Figure 2 This is a flowchart illustrating an embodiment of the personalized character appearance adaptive adjustment method of the present invention.

[0057] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0059] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0060] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The quantifier "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of "first," "second," and "third," etc., does not indicate any order and can be interpreted as names.

[0061] like Figure 1 As shown, Figure 1 This is a schematic diagram of the hardware operating environment server 1 (also called a personalized character appearance adaptive adjustment device) involved in the embodiment of the present invention.

[0062] The server in this embodiment of the invention includes devices with display functions such as "Internet of Things devices", smart air conditioners, smart lights, smart power supplies with network connectivity, AR / VR devices with network connectivity, smart speakers, autonomous vehicles, PCs, smartphones, tablets, e-book readers, and portable computers.

[0063] like Figure 1 As shown, the server 1 includes: a memory 11, a processor 12, and a network interface 13.

[0064] The memory 11 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the server 1, such as the hard disk of the server 1. In other embodiments, the memory 11 can also be an external storage device of the server 1, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the server 1.

[0065] Furthermore, the memory 11 may include both internal storage units of the server 1 and external storage devices. The memory 11 can be used not only to store application software and various types of data installed on the server 1, such as the code of the personalized character appearance adaptive adjustment program 10, but also to temporarily store data that has been output or will be output.

[0066] In some embodiments, processor 12 may be a central processing unit (CPU), controller, microcontroller, microprocessor or other data processing chip, used to run program code stored in memory 11 or process data, such as executing personalized character model adaptive adjustment program 10.

[0067] The network interface 13 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface), which is typically used to establish communication connections between the server 1 and other electronic devices.

[0068] The network can be the Internet, a cloud network, a Wi-Fi network, a Personal Area Network (PAN), a Local Area Network (LAN), and / or a Metropolitan Area Network (MAN). Various devices in the network environment can be configured to connect to the communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Li-Fi, 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access points (APs), device-to-device communication, cellular communication protocols, and / or Bluetooth communication protocols, or combinations thereof.

[0069] Optionally, the server may also include a user interface, which may include a display, an input unit such as a keyboard, and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be referred to as a screen or display unit, used to display information processed in server 1 and to display a visual user interface.

[0070] Figure 1 Only server 1, which includes components 11-13 and a personalized character appearance adaptive adjustment program 10, is shown. Those skilled in the art will understand that... Figure 1 The structure shown does not constitute a limitation on server 1 and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0071] In this embodiment, the processor 12 can be used to call the personalized character appearance adaptive adjustment program stored in the memory 11 and perform the following operations:

[0072] Process music audio data, extract static thematic features that represent the global style of the music, and generate music temporal emotion vectors that represent the real-time fluctuations of the music.

[0073] Acquire and process player voice signals to extract static age features representing the player's physiological attributes, and real-time player state vectors representing the player's real-time emotional state;

[0074] The music temporal emotion vector and the real-time player state vector are weighted and fused together, and combined with the static theme features and the static age features to generate a fusion condition vector;

[0075] The fusion condition vector is input into a preset conditional generation model to generate a modeling parameter vector;

[0076] The modeling parameter vector is applied to the player's base character model to adjust the player's base character model in real time.

[0077] Based on the hardware architecture of the aforementioned personalized character appearance adaptive adjustment device, an embodiment of the personalized character appearance adaptive adjustment method of the present invention is proposed. The personalized character appearance adaptive adjustment method of the present invention aims to solve the technical problems in the prior art, such as the disconnect between virtual character appearance and musical atmosphere, the disruption of the continuity of the player's basic image, and the static and singular adaptation effect.

[0078] Reference Figure 2 , Figure 2 This is an embodiment of the personalized character appearance adaptive adjustment method of the present invention, which includes the following steps:

[0079] S10. Process music audio data, extract static thematic features that represent the overall style of the music, and music temporal emotion vectors that represent the real-time fluctuations of the music.

[0080] Here, a static theme feature is a feature vector used to characterize the overall content, style, or atmosphere of the music audio data. For example, this feature vector can quantitatively indicate whether the music belongs to themes such as "summer," "urban," or "cyberpunk." This static theme feature remains unchanged throughout the music playback, providing a global stylistic tone for subsequent styling adjustments.

[0081] A musical temporal emotion vector is a dynamically changing vector that is used to finely depict the emotional fluctuations of music at different points in time (e.g., verses and choruses). Specifically, this vector can be a two-dimensional vector, including arousal (representing energy level, ranging from low to high) and valence (representing emotional polarity, ranging from negative to positive) dimensions, thereby capturing emotional changes from calm and tension to excitement and explosion.

[0082] This step serves as the input source throughout the method, ensuring that subsequent fusion modules receive the timing and static information driven by the music.

[0083] In step S11, the Mel spectrogram of the music audio data is extracted.

[0084] Specifically, the system first preprocesses the input music audio data (such as MP3 or WAV files), including decoding, resampling to a specific sampling rate (e.g., 16kHz), and framing. For example, the audio signal can be divided into several short frames (e.g., frame length 25 milliseconds, frame shift 10 milliseconds).

[0085] Subsequently, a Short-Time Fourier Transform (STFT) is performed on each frame of the signal to obtain the spectrum. The spectrum is then filtered and logarithmically calculated using a Mel filter bank, ultimately generating a two-dimensional Mel spectrogram. This Mel spectrogram can visually represent the energy distribution of the audio signal at different frequencies and times, forming the basis for subsequent temporal sentiment analysis.

[0086] For example, taking a game of motion-sensing music and dance game as an example, if player A inputs an EDM track with a sampling rate of 44.1kHz, an STFT window size of 1024 samples, and 128 Mel filters, then M(f,t) captures the energy distribution of low-frequency rhythm drum beats and high-frequency synthesizer melodies. At this time, M(f,t) shows a gradual increase in amplitude in the low-frequency band (f<20) of the verse, quantifying the accumulated tension.

[0087] In step S12, the Mel spectrogram is processed by a pre-trained convolutional recurrent neural network to generate the music temporal emotion vector.

[0088] This step is a concrete implementation of temporal sentiment analysis on the Mel spectrogram generated in step S11. Here, the pre-trained convolutional recurrent neural network (CRNN) is an end-to-end temporal modeling architecture that combines convolutional neural networks (CNNs) to extract local spectral patterns with recurrent neural networks (RNNs) to capture long-range dependencies, used to map the emotional dynamics of the music from the Mel spectrogram. This network acts as the core engine for real-time music sentiment extraction, ensuring that the generated vectors reflect instantaneous changes in the audio, such as rhythmic build-up or harmonic release, and supporting temporal alignment for subsequent fusion.

[0089] In some embodiments, step S11 is implemented through the following steps S111-S113:

[0090] S111. Based on the Mel spectrogram, extract the time-series feature sequence within a predetermined time window.

[0091] Specifically, the complete Mel spectrum generated in step S11 (Where F is the number of Mel frequency bands and T is the total number of frames) Segment along the time axis T. Slide using a predetermined time window with a width of w frames and a step size of s frames (e.g., w = 128 frames, s = 10 frames).

[0092] The temporal feature sequence X_k extracted at each time step k can be represented as:

[0093] X k =M[:,k·s:k·s+w],where,

[0094] X_k is the input to the subsequent convolutional recurrent neural network at the current time step k.

[0095] S112. Based on the temporal feature sequence, local features are extracted through the convolutional layer of the convolutional recurrent neural network, and temporal dependencies are captured through the recurrent layer of the convolutional recurrent neural network to generate intermediate sentiment representations.

[0096] Specifically, in this convolutional recurrent neural network (CRNN), the convolutional layers (e.g., multiple layers of two-dimensional convolutional kernels) first operate on the input X_k to extract local patterns (i.e., "local features") in the frequency domain and short-time domain. A two-dimensional convolution operation C can be summarized as:

[0097]

[0098] Where K is the convolution kernel weight, b is the bias term, and * represents the convolution operation. After the pooling layer, the output feature map is flattened in the frequency dimension but preserved in the time dimension, forming a feature sequence. (where D_feat is the feature dimension extracted by the convolutional layer).

[0099] Subsequently, the characteristic sequence C′ k This information is fed into recurrent layers (e.g., bidirectional long short-term memory networks (Bi-LSTM) or gated recurrent units (GRUs) to capture long-range "temporal dependencies." For example, in an LSTM unit, computing the current hidden state h_k involves:

[0100] i k =σ(W i C′ k +U i h k-1 +b i );

[0101] f k =σ(W f C′ k +U f h k-1 +b f );

[0102] ok=σ(W o C′ k +U o h k-1 +b o );

[0103]

[0104] h k =o k ☉tanh(c k );

[0105] Where σ is the Sigmoid activation function and ⊙ is the element-wise product. After the recurrent layer has run through all time steps, its final hidden state (or the sequence of all hidden states) is considered as the "intermediate sentiment representation".

[0106] S113. Based on the intermediate emotion representation, the intermediate emotion representation is mapped through a fully connected layer to generate the music temporal emotion vector.

[0107] Specifically, the intermediate sentiment representation H generated in S112 (e.g., the hidden state h_T at the last time step of the recurrent layer) is input into one or more fully connected layers (DenseLayer) for nonlinear mapping. This mapping operation can be represented as:

[0108] V_output = W_fc·H + b_fc, where V_output is the final generated music temporal emotion vector.

[0109] Music temporal emotion vectors include arousal and valence dimensions, which are typically normalized to a specific range (e.g., [-1, 1]) to characterize the real-time position of music in a two-dimensional emotion space.

[0110] It is understandable that through window sequence extraction, CNN-GRU dependency capture, and FC emotion mapping in sub-steps S111-S113, step S12 achieves fine temporal analysis of the Mel spectrum, thus accurately quantifying the arousal and pleasure dynamics of the music (such as the arousal jump from 0.6 to 0.9), thereby improving the emotional resolution, linking the dynamic weight calculation of the fusion module, and improving the musical responsiveness and immersive continuity of the styling adjustment.

[0111] In step S13, the metadata of the music audio data is processed by a pre-trained natural language processing model to generate the static topic features.

[0112] This step, executed in parallel with S11-S12, is used to extract the global style of the music. Here, metadata can include the music's lyrics, genre tags (such as "rock," "electronic"), song title, or album description.

[0113] Natural Language Processing (NLP) models can be a combination of pre-trained word embedding models (such as Word2Vec) and topic models (such as LDA), or more advanced pre-trained language models (such as BERT). This NLP model is responsible for parsing semantic information from metadata, such as extracting the theme "summer" from lyrics like "sunshine, beach," or extracting the themes "urban" or "cyberpunk" from the genre tag "Synthwave," and ultimately outputting a static topic feature that quantifies these global styles (e.g., a 128-dimensional embedding vector).

[0114] It is understandable that by adopting the parallel processing scheme of steps S11-S13, the music audio data is parsed into temporal Mel spectrograms and static metadata, and processed using CRNN and NLP models respectively. Therefore, this method can simultaneously obtain static thematic features representing the overall style of the music (used to establish the tone of the design) and dynamic emotional vectors representing the real-time fluctuations of the music (used to drive the real-time changes of the design), thereby achieving comprehensive quantification of the music in multiple dimensions, improving the temporal resolution of emotional analysis, and thus providing a robust music input foundation for the fusion module, improving the immersiveness and response accuracy of the overall design adjustment.

[0115] S20. Acquire and process player voice signals to extract static age features representing the player's physiological attributes, and real-time player state vectors representing the player's real-time emotional state.

[0116] This step involves parallel analysis of player speech segments captured by the microphone to obtain features in two different dimensions.

[0117] The static age feature is a low-dimensional embedding vector representing the player's biological age group, such as "youth (18-35 years old)," inferred from long-term averaged acoustic parameters. It provides a personalized baseline to accommodate styling preferences at different life stages. This feature serves as a global anchor throughout the method, ensuring that styling adjustments take into account the player's biologically constant attributes.

[0118] The real-time player state vector is a vector that changes dynamically over time and is used to capture the player's immediate emotional reactions (such as excitement or tension) during game interactions.

[0119] In some embodiments, step S20 is implemented through steps S21-S23:

[0120] S21. Extract the Mel frequency cepstral coefficients, fundamental frequency, and formant frequency of the player's voice signal.

[0121] To extract static features for characterizing physiological properties, the system can analyze speech signals over a relatively long period (e.g., a cumulative 5-10 seconds) and calculate their statistical properties. Among these,

[0122] Mel-frequency cepstral coefficients (MFCCs): These can be calculated by pre-emphasizing, framing, windowing (e.g., Hamming window w(n)), performing Fast Fourier Transform (FFT), Mel filtering, and Discrete Cosine Transform (DCT) on the speech signal s(n). The i-th MFCC coefficient C_i(k) of the k-th frame can be summarized as:

[0123]

[0124] Where E j (k) is the energy output of the k-th frame on the j-th Mel filter, where M is the number of filters.

[0125] Fundamental frequency (F0): Represents the fundamental frequency of vocal cord vibration, which can be estimated from the time-domain signal using methods such as the autocorrelation function (ACF) or the YIN algorithm.

[0126] Formants: These represent the resonant frequencies of the vocal tract (oral and nasal cavities), and can be estimated using linear predictive coding (LPC) analysis, i.e., finding the LPC polynomials. The frequency of the root.

[0127] S22. The Mel frequency cepstral coefficients, the fundamental frequency, and the formant frequencies are processed by a pre-trained deep neural network to generate the static age features.

[0128] Specifically, the system concatenates the statistical values ​​of various acoustic features extracted in step S21 (e.g., the mean and variance of MFCC coefficients, the mean and jitter of F0, and the average position of the first three formants F1-F3) to form a high-dimensional static acoustic feature vector X_static. This X_static vector is input into a pre-defined deep neural network (DNN) (e.g., a multilayer perceptron MLP) age classification model. This model performs layer-by-layer abstraction through nonlinear layers (e.g., the ReLU activation function).

[0129] H l =ReLU(W l ·H l-1 +b l ),

[0130] Among them, H l-1 It is the output of the previous layer (H0 = X) static ), W l and b l These are the weights and biases of the l-th layer. The last layer typically uses the Softmax activation function to output the classification probability, or directly outputs an embedding vector as the static age feature Y. age .

[0131] S23. Extract the real-time acoustic features of the player's voice signal within the rolling time window, and process the real-time acoustic features through a pre-trained convolutional bidirectional long short-term memory network to generate the real-time player state vector.

[0132] Specifically, this step is executed in parallel to extract dynamic emotions. In particular, step S23 can be achieved through the following steps S231-S233:

[0133] S231. Calculate the spectroscopic features and prosodic features of the player's voice signal within the rolling time window to generate the real-time acoustic features.

[0134] Specifically, the system analyzes the player's voice signal s(n,t) within a rolling time window (e.g., window length W_r = 1 second, step S_r = 0.2 seconds).

[0135] Spectral characteristics: Calculate the Mel spectrum M(t) of the signal within this window. This calculation process is consistent with the Mel spectrum extraction method described in step S11, but here it is based on a short-time rolling window.

[0136] Prosodic features: Calculate the pitch f_0(t) and energy (e.g., root-mean-square logarithmic energy E(t)) within this window. For example, the energy E(t) can be calculated as:

[0137] Where N_w is the number of sampling points within the window.

[0138] Subsequently, the extracted spectral features M(t) and prosodic features (e.g., statistical values ​​or sequences of f_0(t) and E(t)) are combined to generate the real-time acoustic feature X. realtime (t).

[0139] S232. Based on the real-time acoustic features, local patterns are extracted through the convolutional layers of the convolutional bidirectional long short-term memory network, and the preceding and following temporal contexts are fused through the bidirectional long short-term memory layers of the convolutional bidirectional long short-term memory network to generate real-time emotion embeddings.

[0140] Specifically, real-time acoustic features X realtime (t) is input into a pre-trained convolutional bidirectional long short-term memory network (CNN-BiLSTM) and undergoes the following computation:

[0141] First, the convolutional layers of a convolutional bidirectional long short-term memory network (e.g., a 2D CNN module) will affect X. realtime (t) is used to extract local patterns in the time-frequency domain. This process is implemented using a convolutional kernel K_cnn: C (t) =ReLU(K cnn *X realtime (t)+b cnn ).

[0142] After convolution and pooling, the output feature map C(t) is reshaped (e.g., the frequency dimension is flattened) into a feature sequence C'_t.

[0143] Next, the feature sequence C'_t is fed into a bidirectional long short-term memory (Bi-LSTM) layer. This layer contains a forward LSTM and a backward LSTM to fuse the preceding and following temporal contexts, as follows:

[0144] and

[0145]

[0146] Output at time t This refers to real-time emotion embedding that incorporates contextual information.

[0147] S233. Based on the real-time emotion embedding, the real-time emotion embedding is weighted by an attention mechanism to generate the real-time player state vector, wherein the real-time player state vector includes an arousal dimension value and a pleasure dimension value.

[0148] Specifically, to aggregate information from the real-time sentiment embedding sequence (H = [h_1, h_2, ..., h_T]), the system employs an attention mechanism. This mechanism calculates the similarity between a context vector u (e.g., a trainable parameter) and each h_t to generate attention weights α_t, which are calculated as follows:

[0149] e t =tanh(W a h t +b a ),

[0150]

[0151] Where W_a and b_a are trainable parameters.

[0152] Subsequently, the H sequence is weighted and summed using these weights to obtain a fixed-dimensional weighted sentiment representation v, calculated as follows:

[0153]

[0154] Finally, the weighted sentiment representation v is fed into one (or more) fully connected layers for mapping to generate the real-time player state vector V. player (t). Specifically, the obtained real-time player state vector includes an arousal dimension value and a valence dimension value, which can be represented as:

[0155] V player (t)=[V Arousal V Valence ].

[0156] For example, in Player A's game, the system captures the "youthful calm" tone by extracting MFCC / F0 / Formants in step S21; it emphasizes "youthful vitality" by generating static age features in step S22; and it extracts spectral patterns using CNN, fuses context using BiLSTM, and applies attention-weighted V in step S23. player (t1) = [0.2, 0.4] (calm and prepared type) vs V player (t2) = [0.8, 0.9] (passionate release type), thus quantifying the player's complex state from "holding their breath" to "bursting out" to support visual synchronization with the music.

[0157] It is understandable that through acoustic extraction, MLP age regression and CNN-BiLSTM temporal weighting in sub-steps S21-S23, step S20 accurately characterizes the player's static physiology and dynamic emotions, thus improving the physiological-emotional differentiation of the state vector, thereby strengthening the personalized weight of the fusion and realizing the player-centered responsiveness and continuous immersion of the appearance adjustment.

[0158] S30. The music temporal emotion vector and the real-time player state vector are weighted and fused, and combined with the static theme feature and the static age feature to generate a fusion condition vector.

[0159] This process dynamically balances the music and the player's contribution to form a unified input signal that drives subsequent character creation, thereby ensuring that adjustments both respond to the music rhythm and match the player's immediate reactions.

[0160] Specifically, the fusion conditional vector is a high-dimensional composite vector that concatenates a static component with a dynamic weighted component. This vector is designed based on a multimodal fusion framework, which can smoothly integrate temporal signals in the emotional space and support precise conditional inputs for conditional models: the static dimension provides global constraints, while the dynamic dimension captures instantaneous interactions, avoiding the dominance of a single modality.

[0161] In some embodiments, step S30 can be implemented by the following steps S31-S34:

[0162] S31. Based on the real-time player state vector, calculate the music contribution weight of the music temporal emotion vector and the player contribution weight of the real-time player state vector through a preset weight controller. The weight controller performs smoothing processing based on the music feature sequence and player state sequence within the historical time window to ensure that the music contribution weight and the player contribution weight have temporal continuity.

[0163] Specifically, the system employs a weighted controller to dynamically allocate the proportions of musical emotion and player emotion during the fusion process. This controller can be a small neural network or a set of preset logical rules, and its input can be based on the real-time player state vector V. player (t). For example, when V is detected. player When the arousal value (e.g., the player's excited cheer) of (t) increases significantly and exceeds a certain threshold, the weight controller can dynamically increase the value of the player contribution weight β and correspondingly decrease the value of the music contribution weight α, so that the styling adjustment is more responsive to the player's immediate emotions at this time.

[0164] In a specific implementation, the two weight values ​​satisfy the constraint: α(t) + β(t) = 1.

[0165] In addition, to avoid frequent changes in weights due to instantaneous signal mutations, which would cause the adjustment of modeling parameters (see S50) to appear abrupt or flickering, the weight controller can also perform smoothing based on the music feature sequence and player state sequence within the historical time window.

[0166] For example, the weight controller first calculates the instantaneous weights α and β at the current time t, and then smooths the weights using an exponential moving average (EMA) algorithm:

[0167] α′(t)=(1-λ)·α′(t-1)+in·α(t);

[0168] B'(t)=(1-λ).β'(t-1)+λ·β(t);

[0169] Where α′(t) and β′(t) are the final smoothed weights, λ is a smoothing coefficient (e.g., λ = 0.1), and α′(t-1) and β′(t-1) are the smoothed weights of the previous time step.

[0170] This approach ensures that the weighting of music contribution and player contribution has temporal continuity.

[0171] S32. Based on the music contribution weight multiplied by the music temporal sentiment vector, a weighted music temporal sentiment vector is generated.

[0172] Specifically, the system obtains the smoothed music contribution weight α'(t) generated in step S31, and multiplies it element-wise with the music temporal emotion vector V_emotion(t) generated in step S10:

[0173] V′ emotion (t)=α′(t)·V emotion (t).

[0174] The resulting V'_emotion(t) is the weighted music temporal emotion vector.

[0175] S33. Generate a weighted real-time player state vector by multiplying the player contribution weight by the real-time player state vector.

[0176] Similarly, the system obtains the smoothed player contribution weight β′(t) generated in step S31 and multiplies it element-wise with the real-time player state vector V_player(t) generated in step S20:

[0177] V′ player (t)=β′(t)·V player (t).

[0178] The resulting V'_player(t) is the weighted real-time player state vector.

[0179] S34. Concatenate the weighted music temporal emotion vector, the weighted real-time player state vector, the static theme feature, and the static age feature to generate the fusion condition vector.

[0180] Finally, the system concatenates the two dynamic weighted vectors (V'_emotion(t) and V'_player(t)) with the two static feature vectors (i.e., the static theme feature E_theme generated in step S10 and the static age feature E_age generated in step S20) to form a feature vector with a higher dimension:

[0181] F fused (t)=Concat(V′ emotion (t),V′ player (t),E theme E age ).

[0182] The F_fused(t) is the fusion condition vector, which will be used as the input condition for the conditional generative model in the next step S40.

[0183] Specifically, the weights α / β in step S31 serve as scalar inputs to steps S32 and S33, ensuring that the weighted output is continuous with the history (EMA bridges t-1 to t); the Concat step in S34 connects the weighted dynamics and statics in series to form a closed loop—V_emotion / V_player→α / β→weighted vector→F_fused(t), where Δ_seq feedback strengthens temporal stability and avoids sudden changes interfering with rendering.

[0184] For example, in player A's motion-sensing music and dance game, step S31 calculates α_smooth = 0.75 based on the low norm (V_player) at t1, and smooths the history Δ_seq = 0.35; S32 generates V_emotion_weighted(t1) = [0.45, 0.225] to capture the verse rhythm; S33 generates V_player_weighted(t1) = [0.05, 0.1] to incorporate player A's breath-holding; S34 concatenates F_fused(t1) to quantify the composite state of "musical tension dominating". Moving to the chorus t2, Δ_seq drops to 0.15 (synchronization).

[0185] α_smooth=0.4, V_emotion_weighted(t2)=[0.36,0.32]

[0186] vs

[0187] V_player_weighted(t2)=[0.48,0.54],

[0188] F_fused(t2) highlights the balance between the "player's outburst" and the music's climax, contrasting the calm and building momentum of t1 with the passionate release of t2, thus enhancing the visual feedback of the dance synchronization.

[0189] It is understandable that through the weight smoothing calculation, element weighting and vector concatenation in steps S31-S34, step S30 achieves the dynamic balance and fusion of music and player signals, thereby accurately capturing the temporally continuous interaction mode, thus improving the robustness and personalization of the fusion vector, and enhancing the responsiveness and immersive continuity of the overall shape adjustment.

[0190] S40. Input the fusion condition vector into the preset conditional generation model to generate the modeling parameter vector.

[0191] Specifically, the conditional generative model can be a deep neural network based on an encoder-decoder architecture (e.g., a conditional variational autoencoder C-VAE). This step utilizes this encoder-decoder deep neural network to infer targeted adjustment instructions from the fused signal, thereby achieving the creative evolution of visual elements while maintaining the continuity of the basic shape.

[0192] In some embodiments, step S40 is achieved through the following steps S41-S43:

[0193] S41. Based on the fusion condition vector, sample the noise vector from the standard normal distribution.

[0194] Specifically, to ensure that the generated modeling parameters exhibit subtle variations under the same input conditions (i.e., to create a sense of "surprise"), the system first samples a noise vector z from a multidimensional standard normal distribution N(0,I): z ~ N(0,I). This noise vector z serves as the source of randomness and participates in the generation process together with the fusion condition vector F_fused(t).

[0195] S42. The encoder of the conditional generative model compresses the fusion conditional vector and the noise vector to generate a latent representation.

[0196] Specifically, the encoder network E_Φ(·) receives two inputs: the fusion condition vector F_fused(t) generated in step S34 and the noise vector z sampled in step S41.

[0197] An encoder network (e.g., composed of multiple fully connected or convolutional layers) processes the concatenated joint input [F_fused(t), z], compresses it, and maps it to a low-dimensional latent space to generate the latent representation l:

[0198] l = E φ ([F fused (t),z]), where Φ is the parameter of the encoder network.

[0199] S43. The latent representation is reconstructed through the decoder of the conditional generative model to generate the styling parameter vector, which includes at least one of clothing self-illumination intensity, clothing self-illumination color, clothing material UV roll speed, hair highlight intensity, makeup density, and accessory visibility.

[0200] Specifically, the decoder network D_θ(·) receives the latent representation l generated in step S42 as input. The decoder (e.g., composed of multiple upsampling layers or fully connected layers) performs the opposite operation to the encoder, "decoding" or "reconstructing" the latent representation, expanding it from the low-dimensional latent space and mapping it back to the target parameter space to generate the final modeling parameter vector P_final(t):

[0201] P final (t)=D θ (l), where θ is the parameter of the decoder network.

[0202] The P_final(t) is a vector that defines specific visual adjustment values, including at least one of the following styling parameters: clothing emissivity intensity, clothing emissivity color, clothing material UV roll rate, hair highlight intensity, makeup density, and accessory visibility. For example, P_final(t) = [P_1, P_2, ..., P_6], where P_1 corresponds to emissivity intensity (a floating-point number [0,1]) and P_6 corresponds to accessory visibility (a floating-point number or Boolean value [0,1]).

[0203] Let's take Player A's game (for example, a real-time dance feedback game) as an example:

[0204] 1. At time t1 (e.g., the tense build-up phase of the musical verse):

[0205] The fusion condition vector F_fused(t1) generated in step S34 mainly reflects the "high tension" of the music and the "high focus" of the player.

[0206] S41: The system samples the noise vector z1 from N(0,I).

[0207] S42: The encoder E_Φ receives both F_fused(t1) and z1 as input, compresses them together, and generates a latent representation l1 located in a specific region of the latent space. The position of l1 is mainly determined by F_fused(t1), so l1 itself embodies the "tension" feature.

[0208] S43: The decoder D_θ receives l1 for reconstruction and outputs the styling parameter vector P_final(t1). The specific values ​​of this vector (e.g., [clothing self-illumination intensity = 0.2, self-illumination color = (0,0,1)(blue), UV scrolling speed = 0.1]) are rendered as a "blue low glow" and slow dynamic effect, visually reflecting a "calm and poised" style.

[0209] 2. At time t2 (e.g., the climax of the chorus):

[0210] The fusion condition vector F_fused(t2) generated in step S34 changed drastically, mainly reflecting the "high arousal" of the music and the "high excitement" of the player (e.g., the player cheered in sync).

[0211] S41: The system independently samples to obtain a new noise vector z2.

[0212] S42: Encoder E_Φ receives a new input combination F_fused(t2) and z2, compresses them to generate a latent representation l2 that is far from l1. The position of l2 is mainly determined by F_fused(t2), reflecting the characteristic of "high release".

[0213] S43: The decoder D_θ receives l2 for reconstruction and outputs a distinctly different styling parameter vector P_final(t2) (e.g., [clothing self-illumination intensity = 1.0, self-illumination color = (1, 0.8, 0) (gold), UV scrolling speed = 5.0]). This vector is rendered as a dynamic effect of "gold high speed", highlighting the "passionate release" style.

[0214] It is understandable that, through the S41-S43 process, the system can generate P_final(t1) and P_final(t2) with significantly different parameters based on the change of F_fused(t) (from t1 to t2). This parameter jump from "blue low glow" to "gold high speed" visually enhances the visual transition from "deep and restrained" to "dazzling and expansive," thus providing players with precise and highly immersive visual feedback for actions from build-up to explosive release in real-time dance games. Therefore, it achieves diverse and consistent model generation, thereby improving the personalized differentiation of visual elements and enhancing the immersive response and continuous creativity of overall adjustments.

[0215] S50. Apply the modeling parameter vector to the player's basic character model to adjust the player's basic character model in real time.

[0216] This process is responsible for transforming the abstract modeling parameter vector $P_{final}(t)$ generated in step S40 into real-time character appearance changes visible to the player in the game interface. At the same time, it strictly maintains the geometric structure and texture core of the basic model and avoids any replacement operations, thereby ensuring the continuity of the player's appearance in dynamic scenes.

[0217] Specifically, the basic character model is a preset 3D mesh (such as face shape, hairstyle and clothing in FBX format), which carries the player's custom identity and serves as the rendering base in Unity or Unreal Engine; the model parameter vector P_final(t) is directly passed in from step S40, providing [0,1] normalized instructions for fine-tuning Shader properties.

[0218] In some embodiments, step S50 can be implemented by the following steps S51-S53:

[0219] S51. Extract the parameter values ​​from the modeling parameter vector.

[0220] Specifically, the system script that executes this step (e.g., a C# or C++ script running in the game engine) first obtains the modeling parameter vector P_final(t) for the current time point t from the conditional generative model in step S40.

[0221] The script then parses the P_final(t) vector to extract the specific parameter values ​​corresponding to each dimension. For example, based on the vector structure defined in step S43, the script will extract P_1 (e.g., a floating-point number [0,1] representing the self-illumination intensity of the clothing), P_2 (e.g., an RGB color value representing the self-illumination color of the clothing), ..., P_6 (e.g., a floating-point number [0,1] representing the visibility of accessories).

[0222] S52. Using the shader script of the preset rendering engine, set the parameter values ​​to the material properties and component properties of the basic character model to adjust the material, lighting effects, makeup density, and accessory visibility of the basic character model.

[0223] Specifically, after S51 extracts the parameter values, the system applies these values ​​to the rendered object using a pre-defined rendering engine (such as Unity or Unreal Engine). Basic character models (e.g., the player's T-shirt model and face model) are pre-bound with custom shaders that support parametric adjustments. The system script then calls the rendering engine's API to pass the parameter values ​​extracted by S51 to the corresponding shader script's material properties (ShaderProperties).

[0224] For example, this process can be implemented in the Unity engine through the following API calls:

[0225] RenderMaterial.SetFloat("_EmissionStrength",P_1); (Sets the self-emission strength of the clothing).

[0226] RenderMaterial.SetColor("_EmissionColor",P_2); (Sets the color of the clothing's self-emission).

[0227] RenderMaterial.SetFloat("_ScrollSpeed",P_3); (Sets the material's UV scrolling speed).

[0228] FaceMaterial.SetFloat("_MakeupOpacity",P_5); (Sets the makeup intensity).

[0229] Furthermore, the script can adjust component properties such as the visibility of accessories by manipulating the activation state of the GameObject:

[0230] AccessoryGameObject.SetActive(P_6>0.5f); (Sets the visibility of the accessory).

[0231] S53. Based on the adjusted basic character model, the basic character model is rendered through the rendering engine to achieve real-time adjustment of the player's basic character model.

[0232] After the material properties and component properties of the model are updated in step S52, the relevant data of these adjusted basic character model (e.g., updated shader parameters) will be submitted to the graphics processing unit (GPU) in the next rendering cycle (i.e. the next frame) of the rendering engine.

[0233] When the GPU executes the rendering pipeline, it uses the latest parameter values ​​set in S52 (e.g., the luminous intensity determined by P_1 and the luminous color determined by P_2) to calculate the final pixel color of the model surface.

[0234] Since the process of generating and applying this parameter (S40-S53) is executed in real time and continuously (e.g., updated frame by frame or beat by beat as the music timing vector V_emotion(t) changes), the appearance of the character observed by the player on the display device (such as clothing lighting effects and makeup intensity) will also undergo smooth or drastic dynamic changes, thus visually presenting the real-time adjustment of the player's basic character appearance.

[0235] It is understandable that through parameter extraction, shader attribute setting and frame-by-frame rendering in S51-S53, step S50 transforms the blending command into smooth visual feedback, thereby achieving fine dynamic adjustment of the basic shape (such as the gradient of glow from 0.2 to 1.0), which in turn enhances the player's sense of identity and temporal synchronization, and improves the continuous immersion and personalized expressiveness of the overall method.

[0236] In summary, the personalized character model adaptive adjustment method of this application has the following beneficial effects:

[0237] 1. It ensures the continuity of character appearance and solves the technical problem of player identity discontinuity:

[0238] This application achieves adjustment rather than replacement of the player's basic character model by applying styling parameter vectors and using the rendering engine's shader scripts to set material properties and component properties (such as lighting effects, makeup intensity, or accessory visibility). This method overcomes the drawback of frequently replacing the entire character model due to music or scene changes, which leads to a disconnect in the player experience, thus ensuring that players maintain a sense of identity with their basic character while enjoying dynamic visual effects.

[0239] 2. It achieves a deep sense of immersive, dynamic temporal progression, overcoming the limitations of static adaptation:

[0240] This application utilizes pre-trained convolutional recurrent neural networks and other temporal sentiment analysis models to process music audio into temporal sentiment vectors representing the real-time fluctuations of the music. These temporal vectors (rather than static labels) are used to drive the generation of character appearance parameters, allowing the character's appearance (e.g., from a low-glow blue to a high-speed gold effect) to adjust synchronously with the emotional fluctuations of the music from verse to chorus. This overcomes the static matching defect where the player's image remains the same throughout the entire song, achieving a beat-by-beat dynamic fusion of auditory and visual elements, greatly enhancing immersion.

[0241] 3. It has created a highly personalized experience integrating humans, machines, and entertainment, filling the gap in multimodal fusion:

[0242] This application processes player voice signals in parallel, extracting static age features representing physiological attributes and real-time player state vectors representing immediate emotions. These two voiceprint features, along with the music temporal emotion vector, participate in the generation of a fusion conditional vector. This allows the final character design to not only respond to the music but also match the player's inherent identity (age) and real-time emotions (such as cheers). This overcomes the limitations of relying solely on a single modality or manual selection, achieving a deep, highly personalized adaptation.

[0243] 4. An intelligent and adaptive fusion strategy has been implemented, improving the system's flexibility and smoothness:

[0244] This application introduces a pre-defined weight controller to weight and fuse the music temporal emotion vector and the real-time player state vector. This weight controller dynamically adjusts the contribution weights of the music and the player based on the player's state (e.g., excitement) and performs smoothing based on a historical timeline window. This allows the driving source of appearance adjustments (music or player) to be dynamically switched, and the visual transitions have temporal continuity, resulting in a smarter strategy and smoother effects.

[0245] 5. It enhances the creativity and automation of shape generation, overcoming the limitations of static rule bases:

[0246] This application employs a pre-trained conditional generative model that maps fused conditional vectors to styling parameter vectors (such as clothing luminescence intensity and UV rolling speed). By introducing noise vectors for sampling, this model can automatically learn complex mapping relationships and produce diverse results. This overcomes the shortcomings of relying on manually exhaustively enumerated label-styling rule bases, freeing it from rigid matching constraints. While achieving full automation, it continuously brings players a creative and unpredictable sense of style surprise.

[0247] Furthermore, this invention also proposes a computer-readable storage medium, which can be any one or any combination of several of the following: hard disk, multimedia card, SD card, flash memory card, SMC, read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, etc. The computer-readable storage medium includes a personalized character appearance adaptive adjustment program 10. The specific implementation of the computer-readable storage medium of this invention is largely the same as the aforementioned personalized character appearance adaptive adjustment method and the specific implementation of server 1, and will not be repeated here.

[0248] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0249] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0250] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0251] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0252] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0253] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for adaptive adjustment of personalized character appearance, characterized in that, include: Process music audio data to extract static thematic features that represent the overall style of the music, as well as temporal emotional vectors that represent the real-time fluctuations of the music. Acquire and process player voice signals to extract static age features representing the player's physiological attributes, and real-time player state vectors representing the player's real-time emotional state; The music temporal emotion vector and the real-time player state vector are weighted and fused together, and combined with the static theme features and the static age features to generate a fusion condition vector; The fusion condition vector is input into a preset conditional generation model to generate a modeling parameter vector; The modeling parameter vector is applied to the player's base character model to adjust the player's base character model in real time.

2. The personalized character appearance adaptive adjustment method as described in claim 1, characterized in that, Processing music audio data, extracting static thematic features representing the overall style of the music, and temporal emotional vectors representing the real-time fluctuations of the music, including: Extract the Mel spectrogram of the music audio data; The Mel spectrogram is processed using a pre-trained convolutional recurrent neural network to generate the music temporal sentiment vector; and The metadata of the music audio data is processed by a pre-trained natural language processing model to generate the static topic features.

3. The personalized character appearance adaptive adjustment method as described in claim 2, characterized in that, The Mel spectrogram is processed using a pre-trained convolutional recurrent neural network to generate the music temporal sentiment vector, including: Based on the Mel spectrogram, a time-series feature sequence is extracted within a predetermined time window; Based on the temporal feature sequence, local features are extracted through the convolutional layer of the convolutional recurrent neural network, and temporal dependencies are captured through the recurrent layer of the convolutional recurrent neural network to generate intermediate sentiment representations. Based on the intermediate sentiment representation, the intermediate sentiment representation is mapped through a fully connected layer to generate the music temporal sentiment vector, wherein the music temporal sentiment vector includes an arousal dimension value and a pleasure dimension value.

4. The personalized character appearance adaptive adjustment method as described in claim 1, characterized in that, The system acquires and processes player voice signals to extract static age features representing the player's physiological attributes, and a real-time player state vector representing the player's real-time emotional state, including: Extract the Mel frequency cepstral coefficients, fundamental frequency, and formant frequency of the player's voice signal; The static age feature is generated by processing the Mel frequency cepstral coefficients, the fundamental frequency, and the formant frequencies using a pre-trained deep neural network; and The real-time acoustic features of the player's voice signal are extracted within a rolling time window, and the real-time acoustic features are processed by a pre-trained convolutional bidirectional long short-term memory network to generate the real-time player state vector.

5. The personalized character appearance adaptive adjustment method as described in claim 4, characterized in that, Real-time acoustic features of the player's voice signal are extracted within a rolling time window, and these features are processed by a pre-trained convolutional bidirectional long short-term memory network to generate the real-time player state vector, including: Within the rolling time window, the spectroscopic and prosodic features of the player's voice signal are calculated to generate the real-time acoustic features; Based on the real-time acoustic features, local patterns are extracted through the convolutional layers of the convolutional bidirectional long short-term memory network, and the temporal context before and after are fused through the bidirectional long short-term memory layers of the convolutional bidirectional long short-term memory network to generate real-time emotion embeddings. Based on the real-time emotion embedding, the real-time emotion embedding is weighted through an attention mechanism to generate the real-time player state vector, wherein the real-time player state vector includes an arousal dimension value and a pleasure dimension value.

6. The personalized character appearance adaptive adjustment method as described in claim 1, characterized in that, The music temporal emotion vector and the real-time player state vector are weighted and fused, and combined with the static theme features and the static age features to generate a fusion condition vector, including: Based on the real-time player state vector, the music contribution weight of the music temporal emotion vector and the player contribution weight of the real-time player state vector are calculated by a preset weight controller. The weight controller performs smoothing processing on the music feature sequence and player state sequence within the historical time window so that the music contribution weight and the player contribution weight have temporal continuity. A weighted music temporal sentiment vector is generated by multiplying the music contribution weight by the music temporal sentiment vector. A weighted real-time player state vector is generated by multiplying the player contribution weight by the real-time player state vector. The weighted music temporal emotion vector, the weighted real-time player state vector, the static theme feature, and the static age feature are concatenated to generate the fusion condition vector.

7. The personalized character appearance adaptive adjustment method as described in claim 1, characterized in that, The fusion condition vector is input into a preset conditional generation model to generate a modeling parameter vector, including: Based on the fusion condition vector, a noise vector is sampled from a standard normal distribution; The encoder of the conditional generative model compresses the fusion conditional vector and the noise vector to generate a latent representation; The latent representation is reconstructed by the decoder of the conditional generative model to generate the styling parameter vector, which includes at least one of clothing self-illumination intensity, clothing self-illumination color, clothing material UV roll speed, hair highlight intensity, makeup density, and accessory visibility.

8. The personalized character appearance adaptive adjustment method as described in claim 1, characterized in that, The modeling parameter vector is applied to the player's base character model to adjust the player's base character model in real time, including: Extract the parameter values ​​from the shape parameter vector; By using the shader script of the preset rendering engine, the parameter values ​​are set to the material properties and component properties of the basic character model, so as to adjust the material, lighting effects, makeup intensity and accessory visibility of the basic character model. Based on the adjusted basic character model, the basic character model is rendered through the rendering engine to achieve real-time adjustment of the player's basic character model.

9. A personalized character appearance adaptive adjustment device, characterized in that, The device includes a memory, a processor, and a personalized character appearance adaptive adjustment program stored in the memory and executable on the processor. When the processor executes the personalized character appearance adaptive adjustment program, it implements the personalized character appearance adaptive adjustment method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a personalized character appearance adaptive adjustment program, which, when executed by a processor, implements the personalized character appearance adaptive adjustment method as described in any one of claims 1-7.

Images