Personalized service method of cloud computer based on voice features

Abstract

The invention discloses a personalized service method of a cloud computer based on voice features, and relates to the technical field of cloud computers. The method includes the steps of firstly, collecting and preprocessing voice data in real time through a multi-mode sensor, secondly, extracting acoustic voice features and rhythm features in the voice data through a feature extraction algorithm, and analyzing intonation modes, and thirdly, conducting voice emotion recognition according to the extracted acoustic voice features, rhythm features and intonation modes, 4, according to the voice emotion recognition result, generating a personalized service recommendation result through a personalized recommendation engine by using a mixed recommendation algorithm, and 5, generating personalized service according to the personalized service recommendation result by using a task planner.

Description

Technical Field

[0001] The invention discloses a personalized service method of a cloud computer based on voice features, and relates to the technical field of cloud computers. Background Art

[0002] With the rapid development of cloud computing technology, cloud computing has become increasingly popular as a new computing model. Cloud computing concentrates computing and storage resources in the cloud, allowing users to access cloud services through terminal devices, achieving the separation of computing and storage. This model not only reduces the cost and maintenance of user terminal devices, but also improves resource utilization and flexibility. However, most current cloud computing services are generalized and fail to meet the personalized needs of users in different contexts. When using cloud computing, users desire more intelligent and personalized services based on their voice characteristics and current usage context. However, there is currently no comprehensive method for leveraging the voice characteristics of cloud computing terminals to provide users with highly contextualized and personalized services. Summary of the Invention

[0003] In response to the problems of the existing technology, the present invention provides a personalized service method for cloud computers based on voice features. According to the cloud computing power of the cloud computer terminal, the method uses acoustics, pitch, speaking speed, energy, intonation patterns, voice emotion extraction tools and other means to collect and analyze current voice features in real time, and makes decision recommendations based on the features. It deeply combines voice features with cloud intelligence to provide cloud computer users with highly contextualized and personalized services.

[0004] The specific scheme proposed by the present invention is:

[0005] The present invention provides a personalized service method for a cloud computer based on voice features, comprising:

[0006] Step 1: Use multimodal sensors to collect and preprocess voice data in real time.

[0007] Step 2: Use feature extraction algorithms to extract acoustic speech features and prosodic features from the speech data, and analyze the intonation pattern.

[0008] Step 3: Perform speech emotion recognition based on the extracted acoustic speech features, prosodic features, and intonation patterns.

[0009] Step 4: Based on the speech emotion recognition results, a personalized recommendation engine uses a hybrid recommendation algorithm to generate personalized service recommendation results:

[0010] Perform collaborative filtering: Based on user historical behavior data, the matrix decomposition SVD++ algorithm is used to reduce the RMSE on the NetflixPrize dataset to 0.89.

[0011] Combine the cosine similarity between the speech feature vector and the service label, preset a threshold, and recommend related services to users if the similarity is greater than the threshold.

[0012] Step 5: Use the task planner to generate personalized services based on the personalized service recommendation results.

[0013] Furthermore, in step 1 of the described method for providing personalized services based on voice features on a cloud computer, a 7+1 circular microphone array is used as a multimodal sensor, and the time difference of arrival TDOA algorithm is combined to realize sound source localization. The minimum mean square error MMSE-STSA algorithm based on logarithmic spectrum amplitude estimation is used to suppress music noise while retaining the harmonic structure of speech. Combined with the adaptive beamforming ABF method, the weights of each array element are dynamically adjusted through the linear constrained minimum variance LCMV criterion to achieve a noise suppression effect of -20dB.

[0014] Furthermore, in step 2 of the cloud computer personalized service method based on voice features, the acoustic features are processed using short-time Fourier transform, wherein the window length is set to 25ms, the frame shift is 10ms, the Hamming window is used to reduce spectrum leakage, and the Mel frequency cepstral coefficients are used to extract 40-dimensional features, including first-order and second-order differences.

[0015] For rhythmic features, fundamental frequency F0 detection is used, in which an improved YIN algorithm is used, combined with parabolic interpolation and HMM smoothing processing to improve the accuracy in the range of 50-500Hz.

[0016] And analyze the intonation pattern: use the Hidden Markov Model HMM to model the intonation contour and identify the intonation type.

[0017] Furthermore, in step 3 of the cloud computer personalized service method based on voice features, an emotion classification model is established, using a ResNet-18 convolutional neural network with a 128×128 spectrogram as input, which can identify and output 8 emotion categories, and combined with a bidirectional LSTM network to capture the temporal changes of emotional states.

[0018] Furthermore, in step 4 of the described method for providing personalized services based on voice features on a cloud computer, a real-time feedback mechanism is established, wherein implicit feedback is used to feedback user click and dwell time behaviors, and personalized recommendation engine parameters are updated in real time; explicit feedback is used to provide user ratings and label selections, and the recommendation strategy is optimized through online learning of the FTRL algorithm.

[0019] The present invention also provides a cloud computer personalized service device based on voice features, including a voice acquisition module, a voice feature extraction module, an emotion recognition module, a decision recommendation module, and a service execution module.

[0020] The voice acquisition module uses multimodal sensors to collect and pre-process voice data in real time.

[0021] The speech feature extraction module uses feature extraction algorithms to extract acoustic speech features and prosodic features from speech data, and analyzes intonation patterns.

[0022] The emotion recognition module performs speech emotion recognition based on the extracted acoustic speech features, prosodic features, and intonation patterns.

[0023] The decision-making recommendation module generates personalized service recommendation results based on the speech emotion recognition results through a personalized recommendation engine using a hybrid recommendation algorithm:

[0024] Perform collaborative filtering: Based on user historical behavior data, the matrix decomposition SVD++ algorithm is used to reduce the RMSE on the NetflixPrize dataset to 0.89.

[0025] Combine the cosine similarity between the speech feature vector and the service label, preset a threshold, and recommend related services to users if the similarity is greater than the threshold.

[0026] The service execution module uses the task planner to generate personalized services based on the personalized service recommendation results.

[0027] Furthermore, the voice acquisition module of the cloud computer personalized service device based on voice features adopts a 7+1 circular microphone array as a multimodal sensor, combines the arrival time difference TDOA algorithm to realize sound source localization, and uses the minimum mean square error MMSE-STSA algorithm based on logarithmic spectrum amplitude estimation to suppress music noise while retaining the voice harmonic structure. Combined with the adaptive beamforming ABF method, the weight of each array element is dynamically adjusted through the linear constrained minimum variance LCMV criterion to achieve a noise suppression effect of -20dB.

[0028] Furthermore, the speech feature extraction module of the cloud computer personalized service device based on speech features processes the acoustic features using short-time Fourier transform, wherein the window length is set to 25ms, the frame shift is 10ms, the Hamming window is used to reduce spectrum leakage, and the Mel frequency cepstral coefficient is used to extract 40-dimensional features, including first-order and second-order differences.

[0029] For rhythmic features, fundamental frequency F0 detection is used, in which an improved YIN algorithm is used, combined with parabolic interpolation and HMM smoothing processing to improve the accuracy in the range of 50-500Hz.

[0030] And analyze the intonation pattern: use the Hidden Markov Model HMM to model the intonation contour and identify the intonation type.

[0031] Furthermore, the emotion recognition module of the personalized service device of a cloud computer based on voice features establishes an emotion classification model, uses the ResNet-18 convolutional neural network, and inputs a 128×128 spectrogram, which can recognize and output 8 emotion categories, and combines with a bidirectional LSTM network to capture the temporal changes of emotional states.

[0032] Furthermore, the decision-making recommendation module of the personalized service device of the cloud computer based on voice features establishes a real-time feedback mechanism, wherein implicit feedback is used to feedback user click and stay time behaviors, and the personalized recommendation engine parameters are updated in real time; explicit feedback is used to provide user ratings and label selections, and the recommendation strategy is optimized through online learning of the FTRL algorithm.

[0033] The benefits of the present invention are:

[0034] Using acoustics, pitch, speaking speed, energy, intonation patterns, voice emotion extraction tools and other means, we collect and analyze current voice features in real time, make decision recommendations based on these features, and deeply combine voice features with cloud intelligence to provide cloud computer users with highly contextualized and personalized services. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the application framework of the method of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0037] Example 1

[0038] The present invention provides a personalized service method for a cloud computer based on voice features, comprising:

[0039] Step 1: Use multimodal sensors to collect and preprocess speech data in real time. A 7+1 circular microphone array is used as the multimodal sensor, combined with the Time Difference of Arrival (TDOA) algorithm to localize the sound source. The Minimum Mean Square Error (MMSE)-STSA algorithm, based on logarithmic spectrum amplitude estimation, suppresses music noise while preserving the speech harmonic structure. Combined with the adaptive beamforming (ABF) method, the linearly constrained minimum variance (LCMV) criterion dynamically adjusts the weights of each array element to achieve a noise suppression effect of -20dB.

[0040] Step 2: Use feature extraction algorithms to extract acoustic speech features and prosodic features from the speech data, and analyze the intonation pattern. For acoustic features, short-time Fourier transform is used for processing, with a window length of 25ms and a frame shift of 10ms. A Hamming window is used to reduce spectral leakage, and 40-dimensional features are extracted using Mel-frequency cepstral coefficients, including first-order and second-order differences.

[0041] For rhythmic features, fundamental frequency F0 detection is used, in which an improved YIN algorithm is used, combined with parabolic interpolation and HMM smoothing processing to improve the accuracy in the range of 50-500Hz.

[0042] And analyze the intonation pattern: use the Hidden Markov Model HMM to model the intonation contour and identify the intonation type.

[0043] Step 3: Perform speech emotion recognition based on the extracted acoustic speech features, prosodic features, and intonation patterns. For example, this identifies whether the user's voice conveys emotions such as happiness, sadness, anger, or calmness. This helps more accurately understand the user's intentions and needs, providing an important basis for providing personalized services. A sentiment classification model is built using a ResNet-18 convolutional neural network. Taking a 128×128 spectrogram as input, it can identify and output eight emotion categories. This model is combined with a bidirectional LSTM network to capture the temporal changes in emotional states.

[0044] Step 4: Based on the speech emotion recognition results, a personalized recommendation engine uses a hybrid recommendation algorithm to generate personalized service recommendation results:

[0045] Perform collaborative filtering: Based on user historical behavior data, the matrix decomposition SVD++ algorithm is used to reduce the RMSE on the NetflixPrize dataset to 0.89.

[0046] Combine the cosine similarity between the speech feature vector and the service label, preset a threshold, and recommend related services to users if the similarity is greater than the threshold.

[0047] In step 4, a real-time feedback mechanism is also established, in which implicit feedback is used to feedback user click and dwell time behaviors, and the parameters of the personalized recommendation engine are updated in real time; explicit feedback is used to provide user ratings and label selections, and the recommendation strategy is optimized through online learning of the FTRL algorithm.

[0048] Step 5: Use the task planner to generate personalized services based on the personalized service recommendation results.

[0049] The method can be implemented within a distributed computing framework, such as edge-cloud collaboration for personalized service delivery. Edge nodes can deploy lightweight speech recognition models (such as DeepSpeech 2) for real-time noise reduction and endpoint detection. The cloud server utilizes the Apache Spark distributed computing framework to process large amounts of user behavior data and support 100,000 recommendation requests per second.

[0050] In application scenarios, such as voice office scenario 1:

[0051] The user opens the office software on the cloud computer terminal and starts to write documents by voice input.

[0052] The cloud computer terminal's microphone collects the user's voice signal in real time and analyzes it using tools such as acoustics, pitch, speaking rate, energy, and intonation to extract voice features. For example, it detects that the user speaks quickly, with high pitch, high energy, and an eager tone.

[0053] Analyzing the extracted speech features, we determined that the user's current emotional state was excitement. Combined with the speech content, we discovered that the user was describing an urgent project requirement.

[0054] Based on the collected voice features and emotional information, cloud computing power is used to search the big data model for a service model that matches the current situation. Because the user is in an office setting with urgent project needs and is emotionally excited, services such as quickly generating a project outline and accessing relevant templates and materials are recommended.

[0055] Based on the recommended decision results, a project outline is automatically generated for the user on the cloud computer. Relevant project data and templates are then retrieved from the cloud database, allowing users to quickly write documents. Furthermore, the document's formatting and layout, such as font size and paragraph spacing, are automatically adjusted based on the user's voice style and habits to enhance the user's work experience.

[0056] Voice office scenario 2:

[0057] The user opens an entertainment application on the cloud computer terminal, such as a music player or video platform, and says, "I want to listen to some relaxing music."

[0058] After collecting the user's voice features, we found that the user's speaking speed was moderate, the pitch was steady, the energy was normal, and the intonation pattern was relatively soothing.

[0059] The analysis shows that the user's current emotional state is relaxation.

[0060] Combining voice features and emotional information, the app filters music from a cloud-based music database. Based on the user's music playback history and preferences, as well as current music trends, it recommends a series of relaxing and enjoyable music plays, such as light-hearted pop songs and soothing instrumental music.

[0061] The recommended music list is displayed on the cloud computer's entertainment application interface, and users can directly click to play and enjoy personalized music services. At the same time, the music playback control method is automatically adjusted according to the user's voice command habits, such as voice control to play, pause, and switch songs, to enhance the user's entertainment experience.

[0062] Example 2

[0063] The present invention also provides a cloud computer personalized service device based on voice features, including a voice acquisition module, a voice feature extraction module, an emotion recognition module, a decision recommendation module, and a service execution module.

[0064] The voice acquisition module uses multimodal sensors to collect and pre-process voice data in real time.

[0065] The speech feature extraction module uses feature extraction algorithms to extract acoustic speech features and prosodic features from speech data, and analyzes intonation patterns.

[0066] The emotion recognition module performs speech emotion recognition based on the extracted acoustic speech features, prosodic features, and intonation patterns.

[0067] The decision-making recommendation module generates personalized service recommendation results based on the speech emotion recognition results through a personalized recommendation engine using a hybrid recommendation algorithm:

[0068] Perform collaborative filtering: Based on user historical behavior data, the matrix decomposition SVD++ algorithm is used to reduce the RMSE on the NetflixPrize dataset to 0.89.

[0069] Combine the cosine similarity between the speech feature vector and the service label, preset a threshold, and recommend related services to users if the similarity is greater than the threshold.

[0070] The service execution module uses the task planner to generate personalized services based on the personalized service recommendation results.

[0071] Since the information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0072] Similarly, the device of the present invention uses acoustics, pitch, speaking speed, energy, intonation patterns, voice emotion extraction tools and other means to collect and analyze current voice features in real time, make decision recommendations based on the features, and deeply combine voice features with cloud intelligence to provide cloud computer users with highly contextualized and personalized services.

[0073] It should be noted that not all steps and modules in the above-mentioned processes and device structures are required, and certain steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by certain components in multiple independent devices.

[0074] The above embodiments are merely preferred embodiments for the purpose of fully illustrating the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are within the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the claims.

Claims

1. A personalized service method for cloud computers based on voice features, characterized by include: Step 1: Use multimodal sensors to collect and preprocess voice data in real time. Step 2: Use feature extraction algorithms to extract acoustic speech features and prosodic features from the speech data, and analyze the intonation pattern. Step 3: Perform speech emotion recognition based on the extracted acoustic speech features, prosodic features, and intonation patterns. Step 4: Based on the speech emotion recognition results, a personalized recommendation engine uses a hybrid recommendation algorithm to generate personalized service recommendation results: Perform collaborative filtering: Based on user historical behavior data, the matrix decomposition SVD++ algorithm is used to reduce the RMSE on the Netflix Prize dataset to 0.

89. Combine the cosine similarity between the speech feature vector and the service label, preset a threshold, and recommend related services to users if the similarity is greater than the threshold. Step 5: Use the task planner to generate personalized services based on the personalized service recommendation results.

2. A cloud computer personalized service method based on voice features according to claim 1, characterized in that in step 1, a 7+1 circular microphone array is used as a multimodal sensor, and the arrival time difference TDOA algorithm is combined to realize sound source localization, and the minimum mean square error MMSE-STSA algorithm based on logarithmic spectrum amplitude estimation is used to suppress music noise while retaining the voice harmonic structure. In combination with the adaptive beamforming ABF method, the weight of each array element is dynamically adjusted by the linear constrained minimum variance LCMV criterion to achieve a noise suppression effect of -20dB.

3. The personalized service method based on voice features of a cloud computer according to claim 1, characterized in that In step 2, the acoustic features are processed using short-time Fourier transform, where the window length is set to 25ms, the frame shift is 10ms, the Hamming window is used to reduce spectrum leakage, and the Mel-frequency cepstral coefficients are used to extract 40-dimensional features, including first-order and second-order differences. For rhythmic features, fundamental frequency F0 detection is used, in which an improved YIN algorithm is used, combined with parabolic interpolation and HMM smoothing processing to improve the accuracy in the range of 50-500Hz. And analyze the intonation pattern: use the Hidden Markov Model HMM to model the intonation contour and identify the intonation type.

4. The personalized service method based on voice features of a cloud computer according to claim 1, characterized in that In step 3, a sentiment classification model is established using the ResNet-18 convolutional neural network. The input is a 128×128 spectrogram, which can identify and output 8 emotion categories. It is combined with a bidirectional LSTM network to capture the temporal changes of emotional states.

5. A cloud computer personalized service method based on voice features according to claim 1, Its characteristic is that a real-time feedback mechanism is established in step 4, wherein the user's click and stay time behaviors are fed back through implicit feedback, and the personalized recommendation engine parameters are updated in real time; By providing explicit feedback, user ratings and label selection, the recommendation strategy is optimized through online learning FTRL algorithm.

6. A cloud computer personalized service device based on voice features, characterized by It includes voice acquisition module, voice feature extraction module, emotion recognition module, decision recommendation module and service execution module. The voice acquisition module uses multimodal sensors to collect and pre-process voice data in real time. The speech feature extraction module uses feature extraction algorithms to extract acoustic speech features and prosodic features from speech data, and analyzes intonation patterns. The emotion recognition module performs speech emotion recognition based on the extracted acoustic speech features, prosodic features, and intonation patterns. The decision-making recommendation module generates personalized service recommendation results based on the speech emotion recognition results through a personalized recommendation engine using a hybrid recommendation algorithm: Perform collaborative filtering: Based on user historical behavior data, the matrix decomposition SVD++ algorithm is used to reduce the RMSE on the Netflix Prize dataset to 0.

89. Combine the cosine similarity between the speech feature vector and the service label, preset a threshold, and recommend related services to users if the similarity is greater than the threshold. The service execution module uses the task planner to generate personalized services based on the personalized service recommendation results.

7. A cloud computer personalized service device based on voice features according to claim 6, characterized in that The voice acquisition module uses a 7+1 circular microphone array as a multimodal sensor, combined with the time difference of arrival (TDOA) algorithm to achieve sound source localization, and the minimum mean square error (MMSE-STSA) algorithm based on logarithmic spectrum amplitude estimation to suppress music noise while retaining the harmonic structure of speech. Combined with the adaptive beamforming (ABF) method, the linearly constrained minimum variance (LCMV) criterion dynamically adjusts the weights of each array element to achieve a noise suppression effect of -20dB.

8. The cloud computer personalized service device based on speech features according to claim 6 is characterized in that the speech feature extraction module processes acoustic features using short-time Fourier transform, wherein a window length of 25ms and a frame shift of 10ms are set, a Hamming window is used to reduce spectral leakage, and 40-dimensional features are extracted using Mel-frequency cepstral coefficients, including first-order and second-order differences. For rhythmic features, fundamental frequency F0 detection is used, in which an improved YIN algorithm is used, combined with parabolic interpolation and HMM smoothing processing to improve the accuracy in the range of 50-500Hz. And analyze the intonation pattern: use the Hidden Markov Model HMM to model the intonation contour and identify the intonation type.

9. The cloud computer personalized service device based on voice features according to claim 6 is characterized by: The emotion recognition module establishes an emotion classification model using the ResNet-18 convolutional neural network. With a 128×128 spectrogram as input, it can identify and output eight emotion categories and combine it with a bidirectional LSTM network to capture the temporal changes in emotional states.

10. The personalized service device based on voice features of cloud computers according to claim 1 is characterized in that the decision The recommendation module establishes a real-time feedback mechanism, which uses implicit feedback to feedback user clicks and dwell time behaviors, and updates the personalized recommendation engine parameters in real time; By providing explicit feedback, user ratings and label selection, the recommendation strategy is optimized through online learning FTRL algorithm.

Images