Voice active recommendation system and method based on driving situation

Abstract

The present invention belongs to the field of speech recognition technology, and discloses a voice active recommendation system and method based on driving scenarios; the method includes: collecting vehicle status data, environmental status data, user personal data and user behavior data; using user data as input of a user portrait model to obtain a user portrait label; analyzing the status data and environmental status data to obtain a vehicle safety score and a driving convenience score; inputting the user portrait label, vehicle safety score and driving convenience score into a timbre style recommendation model, and training the timbre style recommendation model in combination with a natural optimization heuristic algorithm; outputting the recommended voice timbre and voice style based on the trained timbre style recommendation model. The present invention comprehensively considers user portraits, as well as driving safety and driving convenience in driving scenarios, and recommends voice timbre and voice style that best meet user preferences; it can enhance the user's driving experience and help safe driving.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and more specifically, to a driving situation-based speech active recommendation system and method. Background Art

[0002] With the in-depth development of car screens in domestic and foreign markets and facing the differentiated needs of different customers, the degree of customization of voice is getting higher and higher, and as the range of car models continues to expand, the overall cost is also increasing; in the development of in-vehicle voice interaction systems, there are some limitations in voice timbre and voice style, which is related to the fact that current smart in-vehicle voice solutions cannot meet the market's rapid iteration needs.

[0003] Existing in-vehicle voice systems often only provide a limited number of standard voice options, such as one or two common male and female voices. These voices cannot meet users' needs for high-quality voice in terms of sound quality and naturalness. For example, the voice may sound mechanical and stiff, lacking emotion and personality, which can easily cause users to have auditory fatigue during long-term use. In addition, in different application scenarios, such as navigation, entertainment recommendations, and vehicle information prompts, the voice style used does not change much. For example, navigation instructions may always be a concise but slightly monotonous style, without adjustment based on factors such as the complexity of road conditions and driving time periods (such as peak hours and non-peak hours); entertainment recommendations may also only use a general style to introduce content, which cannot match the emotional color and characteristics of the entertainment content itself, and cannot well stimulate user interest.

[0004] The Chinese patent application with publication number CN115762503A discloses an in-vehicle voice system, an autonomous learning method, device and medium for in-vehicle voice, including a user interaction front end and a policy management back end, wherein the user interaction front end is used to obtain user behavior portrait data; the user behavior portrait data is sent to the policy management back end; the policy management back end is used to perform model training based on the user behavior portrait data to determine the voice control update strategy, and update the corresponding historical voice control strategy based on the voice control update strategy to obtain the current voice control strategy. This invention solves the problem that the voice control strategy in the existing in-vehicle voice system is preset before leaving the factory, cannot meet the individual needs of different users, and cannot learn autonomously as the user needs change.

[0005] Although the above method can meet most scenarios, research and practical application of the above method and existing technology have found that the above method and existing technology have at least the following defects:

[0006] (1) Simple classification and recommendation based on some basic user portrait tags, but not deep enough exploration of users’ personalized needs.

[0007] (2) The system conducts analysis and learning based on relatively simple user behavior profile data, but does not collect and utilize complex driving situation information sufficiently, which results in the inability to provide users with the most appropriate voice service based on the overall driving situation.

[0008] (3) The recommendation strategy adopted is relatively simple. It only performs model training and recommendation based on existing user behavior profile data, and lacks dynamic analysis and prediction of user needs in different driving scenarios.

[0009] In view of this, the present invention proposes a voice active recommendation system and method based on driving context to solve the above problems. Summary of the invention

[0010] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a voice active recommendation method based on driving situation, comprising the following steps:

[0011] Collect vehicle status data and environmental status data;

[0012] Collect user data, including user personal data and user behavior data;

[0013] Extracting features from user behavior data to obtain behavior feature data, where the behavior feature data includes driving feature data and interaction feature data;

[0014] Use behavioral feature data and user personal data as input to the user portrait model to obtain user portrait labels;

[0015] Analyze status data and environmental status data to obtain vehicle safety scores and driving convenience scores;

[0016] The user portrait label, vehicle safety score and driving convenience score are input into the timbre style recommendation model, and the timbre style recommendation model is trained in combination with the natural optimization inspired algorithm; the recommended voice timbre and voice style are output based on the trained timbre style recommendation model.

[0017] Furthermore, the interactive feature data includes intonation features, speed features and sentiment word score features of the interactive speech data;

[0018] Methods for obtaining intonation features include:

[0019] The collected interactive speech data is sampled and quantized, the interactive speech data is converted into digital signals, and the interactive speech data is divided into short frames. The fundamental frequency of each frame of speech is extracted using the autocorrelation method, the cepstrum method or a method based on a neural network, and the mean of the fundamental frequency is used as the intonation feature;

[0020] Methods for obtaining fast and slow features include:

[0021] Calculate the duration of the interactive speech data and mark it as speech duration; then divide the number of syllables by the speech duration to obtain the result as the speed feature.

[0022] Furthermore, the method for obtaining the sentiment word score feature includes:

[0023] Select an emotional dictionary, use speech recognition technology to convert interactive speech data into speech text, and preprocess the converted speech text, including speech text cleaning and word segmentation; encode each word in the preprocessed speech text according to the pinyin and tone of each word to obtain a word code, match each word code in the preprocessed speech text with the vocabulary code in the emotional dictionary one by one, and obtain the corresponding emotional word in the emotional dictionary;

[0024] Establish corresponding sentiment word category sets for sentiment words of the same category, match the sentiment words in the same sentiment word category set in the preset sentiment word quantization table to obtain the quantization score of each sentiment word, and take the sum of the quantization scores of each sentiment word in the same sentiment word category set as the score of the sentiment word category set; perform weighted summation on the scores of different sentiment word category sets to obtain the sentiment word score feature.

[0025] Furthermore, the training method of the user portrait model includes:

[0026] Collect R groups of training data in advance. The training data includes behavioral feature data, user personal data and user portrait labels;

[0027] Each group of behavioral feature data and user personal data is used as the input of the user portrait model, and the user portrait model uses the user portrait label corresponding to each group of behavioral feature data and user personal data as the output, and uses the actual user portrait label corresponding to each group of behavioral feature data and user personal data as the prediction target; minimizing the sum of prediction errors of all user portrait labels is used as the training target; the user portrait model is trained until the sum of prediction errors reaches convergence and the training is stopped; the user portrait model is a deep neural network model.

[0028] Further, the environmental status data includes surrounding environment data and weather conditions; the surrounding environment data includes road signs and distances to obstacles; the road signs include directional signs and prompt signs;

[0029] The method for obtaining the distance to the obstacle includes:

[0030] Step 101: input the road image collected by the vehicle-mounted camera into a pre-built image recognition model, and the image recognition model outputs whether the road image contains obstacles;

[0031] Step 102: If the road image contains an obstacle, the distance between the vehicle and the obstacle is obtained through the vehicle-mounted radar. If there are multiple distances between the vehicle and the obstacle, the smallest distance between the vehicle and the obstacle is used as the distance between the vehicle and the obstacle.

[0032] Further, the driving characteristic data includes a speed average and a speed standard deviation;

[0033] The method for obtaining the driving characteristic data includes:

[0034] User behavior data includes driving operation data and interactive voice data; driving operation data is The acceleration data from the vehicle starting to the scheduled time;

[0035] Will A speed analysis set is established based on the acceleration data from the start of the vehicle to the predetermined time, and the speed average and standard deviation of the speed analysis set are calculated; the speed average of the speed analysis set is used as the speed average; and the standard deviation of the speed analysis set is used as the speed standard deviation.

[0036] Furthermore, the status data includes vehicle driving data, remaining energy and road information; the vehicle driving data includes the vehicle's current driving speed, acceleration and steering angle; the road information is the slope and curvature data of the vehicle's current location; the status data and environmental status data are input into the safety analysis model to obtain the vehicle safety score.

[0037] Further, the method of obtaining the driving convenience score includes:

[0038] Based on the real-time location information, n navigation paths to the destination are obtained. If n is 1, the remaining time corresponding to the navigation path is obtained. The remaining time is the time required for the real-time location information to reach the destination;

[0039] If n is greater than 1, the n navigation paths include a current navigation path, the remaining time of the current navigation path is marked as the first remaining time, and the current navigation path is the navigation path currently selected by the vehicle navigation system; the time with the smallest remaining time is selected from the n navigation paths and marked as the second remaining time, and the ratio of the second remaining time to the first remaining time is marked as the time characteristic value; if the time characteristic value is greater than or equal to 1, the current navigation path is the optimal path among the n navigation paths; if the time characteristic value is less than 1, the current navigation path is not the optimal path among the n navigation paths;

[0040] The ratio of the mileage that the remaining energy can support to the remaining distance is marked as the distance characteristic value;

[0041] The time characteristic value and the distance characteristic value are weighted and summed to obtain the driving convenience score;

[0042] ;

[0043] In the formula, Score points for driving convenience; and is the preset weight; is the time feature; For the route characteristics.

[0044] Furthermore, the training method of the timbre style recommendation model includes:

[0045] Collect training data in advance, including comprehensive feature data and voice timbre and voice style corresponding to the comprehensive feature data; the comprehensive feature data includes user portrait labels, vehicle safety scores, and driving convenience scores;

[0046] Each group of comprehensive feature data is used as the input of the timbre style recommendation model, and the timbre style recommendation model takes the speech timbre and speech style corresponding to each group of comprehensive feature data as output, and takes the actual speech timbre and speech style corresponding to each group of comprehensive feature data as prediction targets; wherein, minimizing the sum of loss function values ​​of all speech timbre and speech style is used as the training target, and the timbre style recommendation model is trained based on the natural optimization inspired algorithm until the sum of prediction errors reaches convergence and the training is stopped; the timbre style recommendation model is a deep neural network model.

[0047] Furthermore, the loss function value of the timbre style recommendation model is as follows:

[0048] ;

[0049] In the formula, is the loss function value; is the number of training data; The prediction function for the timbre style recommendation model is based on the parameters For Training data Produce prediction output; For the The actual output corresponding to the training data, that is, the actual voice timbre and voice style, For the training data; The parameters of the timbre style recommendation model, namely the weights of the user portrait label, vehicle safety score, and driving convenience score, and the parameters of the timbre style recommendation model The optimization is obtained by training using the cuckoo optimization algorithm.

[0050] Furthermore, the cuckoo optimization algorithm is used to train and optimize the parameters of the timbre style recommendation model. The methods include:

[0051] Step 1: Randomly generate H groups of initial solutions, namely nests, each nest represents a set of parameters of the timbre style recommendation model ,as follows:

[0052] ;

[0053] In the formula, Recommend model output for timbre style; Label the user portrait; Score the vehicle's safety; Score points for driving convenience; Weight of user portrait label; Weighting for vehicle safety score; weighting for driving convenience score;

[0054] Step 2: Iterate and update to generate a new nest. The method of generating a new nest includes:

[0055] ;

[0056] In the formula, For the The parameters of the iterations; For the The parameters of the iterations; is the step size factor, which controls the size of the search step; To obey the random step size of Levy distribution, that is, the random number of distribution, Levy flight helps to jump out of the local optimal solution; is the scale parameter, controlling the shape of the distribution;

[0057] Step 3: Use Calculate the loss function value of the new nest;

[0058] Step 4: If the loss function value of the new nest is less than the loss function value of any nest in group H, then the nest corresponding to the loss function value greater than the new nest in group H is replaced with the new nest with a preset probability;

[0059] When the preset number of iterations is reached, the nest with the smallest loss function value is selected as the parameter of the timbre style recommendation model. .

[0060] The voice active recommendation system based on driving situations includes:

[0061] A first acquisition module, used to collect vehicle status data and environmental status data;

[0062] The second collection module is used to collect user data, which includes user personal data and user behavior data;

[0063] A first analysis module is used to extract features from user behavior data to obtain behavior feature data, where the behavior feature data includes driving feature data and interaction feature data;

[0064] The portrait building module is used to use the behavioral feature data and user personal data as inputs of the user portrait model to obtain the user portrait label;

[0065] A data analysis module is used to analyze the status data and environmental status data to obtain the vehicle safety score and driving convenience score;

[0066] The voice recommendation module is used to input user portrait labels, vehicle safety scores and driving convenience scores into the timbre style recommendation model, and train the timbre style recommendation model in combination with the natural optimization inspired algorithm; based on the trained timbre style recommendation model, the recommended voice timbre and voice style are output.

[0067] The technical effects and advantages of the driving situation-based voice active recommendation system and method of the present invention are as follows:

[0068] The present invention comprehensively considers user personal data and user behavior data to construct a user portrait, fully explores user preferences, deeply explores and accurately matches user's personalized needs, and provides more personalized voice recommendation services; it also collects various driving information, analyzes driving safety and driving convenience in actual driving situations, and can more accurately grasp the actual needs of users in the driving process, and provide more accurate voice recommendation services. It comprehensively considers user portraits, as well as driving safety and driving convenience in driving situations, and trains an optimized timbre and voice style recommendation model to recommend the voice timbre and voice style that best meets user needs and the current driving situation, which can meet the diverse needs of users; and adapts to market development trends, providing effective ideas and practical references for the development of intelligent in-vehicle voice systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a flow chart of the method for active voice recommendation based on driving situations mentioned in Embodiment 1 of the present invention;

[0070] Figure 2 This is a schematic diagram of the user portrait construction method mentioned in Example 1 of the present invention;

[0071] Figure 3 This is a schematic diagram of the generative adversarial training network mentioned in Example 2 of the present invention;

[0072] Figure 4 This is a block diagram of the driving scenario-based voice active recommendation system mentioned in Example 3 of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] Example 1

[0075] See also Figure 1 As shown, this embodiment provides a voice active recommendation method based on driving situations.

[0076] Collect vehicle status data and environmental status data; among them, status data is obtained through the vehicle's built-in sensors, and the status data includes vehicle driving data, remaining energy (remaining energy such as remaining fuel or remaining power) and road information; vehicle driving data includes the vehicle's current driving speed, acceleration, steering angle and other information; specifically collected through sensors on the vehicle (such as speed sensors, acceleration sensors, steering angle sensors, etc.); collected by connecting to the vehicle's internal sensors (such as oil or power sensors, etc.); road information is the vehicle's current slope and curvature data, and the vehicle's real-time location information is obtained through the vehicle's onboard GPS system. The current road slope and curvature data can be directly obtained from the navigation system based on the real-time location information.

[0077] Environmental status data includes surrounding environment data and weather conditions; surrounding environment data includes road signs and the distance to obstacles (such as surrounding buildings, pedestrians and adjacent vehicles, etc.); road signs are recognized through road images collected by the on-board camera to obtain road signs. Road signs include directional signs and prompt signs, such as directional signs including straight signs and right turn signs, etc.; prompt signs include deceleration signs and accident-prone point signs, etc. By collecting these road signs, the dangerous situation of the current road can be judged more accurately, and the auxiliary voice system can make more accurate voice timbre and voice style recommendations.

[0078] Methods for obtaining the distance to obstacles include:

[0079] Step 101: input the road image collected by the vehicle-mounted camera into a pre-built image recognition model, and the image recognition model outputs whether the road image contains obstacles;

[0080] Step 102: If the road image contains an obstacle, the distance between the vehicle and the obstacle is obtained through the vehicle-mounted radar. If there are multiple distances between the vehicle and the obstacle, the smallest distance between the vehicle and the obstacle is used as the distance between the vehicle and the obstacle. By obtaining the distance to the obstacle, the vehicle can sense whether the distance to the obstacle is safe, which is conducive to making better voice timbre and voice style recommendations in the subsequent vehicle safety score calculation. For example, when the vehicle is too close to an adjacent vehicle during driving, a serious and brief voice can be selected for reminder.

[0081] Weather conditions are data received from meteorological services, such as sunny, rainy, foggy, etc. The current weather conditions are collected to facilitate a more accurate analysis of driving safety.

[0082] Collect user data, which includes user personal data and user behavior data. User personal data includes basic information such as age, gender, and occupation. When using the recommendation system for the first time, users can be guided to enter their personal data. User personal data can help to more accurately divide user portrait labels when constructing user portraits in the later stage. For example, users can be divided into young and fashionable groups, business people, or elderly groups based on their personal data. Different user groups have different preferences for the voice timbre and voice style of the voice system, which can help to more accurately recommend the voice timbre and voice style of the voice system. User behavior data includes driving operation data and interactive voice data. Driving operation data is The acceleration data from the start of the vehicle to the scheduled time, where the scheduled time can be set to 3 seconds or other times, can reflect the time of the acceleration behavior; the interactive voice data can reflect the user's driving style; the interactive voice data is the interactive voice data generated by the previous interaction between the user and the vehicle, which is convenient for the subsequent analysis of the user's tone, speed, word preference and sentiment score from the interactive voice data; the driving operation data and interactive voice data are collected to further improve the user portrait, so as to facilitate the subsequent recommendation of voice timbre and voice style that better meets the user's needs based on the user's driving style.

[0083] Extract features from user behavior data to obtain behavior feature data, which includes driving feature data and interaction feature data; driving feature data includes speed average and speed standard deviation; interaction feature data includes intonation features, speed features, and sentiment word score features of interaction voice data;

[0084] Methods for obtaining driving characteristic data include:

[0085] Will A speed analysis set is established based on the acceleration data from the start of the vehicle to the predetermined time, and the speed average and standard deviation of the speed analysis set are calculated; the speed average of the speed analysis set is used as the speed average; and the standard deviation of the speed analysis set is used as the speed standard deviation.

[0086] Methods for obtaining intonation features include:

[0087] The collected interactive speech data is sampled and quantized, converted into digital signals, and divided into short frames. The fundamental frequency of each frame of speech is extracted using the autocorrelation method, the cepstrum method, or a neural network-based method, and the mean of the fundamental frequency is used as the intonation feature.

[0088] Methods for obtaining fast and slow features include:

[0089] Calculate the duration of the interactive speech data and mark it as speech duration; then divide the number of syllables by the speech duration to obtain the result as the speed feature.

[0090] Methods for obtaining sentiment word score features include:

[0091] Select an emotional dictionary, use speech recognition technology to convert interactive speech data into speech text, and preprocess the converted speech text, including speech text cleaning and word segmentation; encode each word in the preprocessed speech text according to the pinyin and tone of each word to obtain a word code, match each word code in the preprocessed speech text with the vocabulary code in the emotional dictionary one by one, and obtain the corresponding emotional word in the emotional dictionary;

[0092] Establish corresponding sentiment word category sets for sentiment words of the same category, match the sentiment words in the same sentiment word category set in the preset sentiment word quantization table to obtain the quantization score of each sentiment word, and take the sum of the quantization scores of each sentiment word in the same sentiment word category set as the score of the sentiment word category set; perform weighted summation on the scores of different sentiment word category sets to obtain the sentiment word score feature.

[0093] Emotional word categories include joy, sadness, anger, etc. The setting method of the emotional word quantization table is as follows:

[0094] For example, emotional words related to joy: happy: 3 points, joyful: 4 points, pleasant: 5 points; emotional words related to sadness: sad: 3 points, heartbroken: 4 points, grief: 6 points; and so on, they are used to quantify emotions and reflect the degree of emotions.

[0095] Use behavioral feature data and user personal data as input to the user portrait model to obtain user portrait labels. Figure 2 .

[0096] The training methods of the user portrait model include:

[0097] Collect R groups of training data in advance. The training data includes behavioral characteristic data, user personal data and user portrait labels.

[0098] Each group of behavioral feature data and user personal data is used as the input of the user portrait model, and the user portrait model uses the user portrait label corresponding to each group of behavioral feature data and user personal data as the output, and uses the actual user portrait label corresponding to each group of behavioral feature data and user personal data as the prediction target; minimizing the sum of prediction errors of all user portrait labels is used as the training target; the user portrait model is trained until the sum of prediction errors reaches convergence and the training is stopped; the user portrait model is a deep neural network model.

[0099] The user portrait model loss function value is the mean square error.

[0100] The status data and environmental status data are analyzed to obtain the vehicle safety score and driving convenience score; the details are as follows:

[0101] The methods used to obtain a vehicle safety score include:

[0102] The status data and environmental status data are input into the safety analysis model to obtain the vehicle safety score.

[0103] The training methods for security analysis models include:

[0104] K groups of safety training data are collected in advance, the safety training data including safety feature data and vehicle safety scores corresponding to the safety feature data; the safety feature data includes state data and environment state data.

[0105] The safety feature data of each group of vehicles is used as the input of the safety analysis model. The safety analysis model uses the vehicle safety score corresponding to the safety feature data of each group of vehicles as the output, and uses the actual vehicle safety score corresponding to the safety feature data of each group of vehicles as the prediction target; minimizing the sum of the prediction errors of all vehicle safety scores is used as the training target; the safety analysis model is trained until the sum of the prediction errors reaches convergence, and the training is stopped; the safety analysis model is a deep neural network model.

[0106] The loss function value of the safety analysis model is the mean square error;

[0107] Prediction error formula: , k is the kth group of security feature data; K is the number of groups of security feature data; is the predicted vehicle safety score corresponding to the kth group of safety feature data, is the actual vehicle safety score corresponding to the kth group of safety feature data.

[0108] The methods for obtaining driving convenience scores include:

[0109] Based on the real-time location information, n navigation paths to the destination are obtained. If n is 1, the remaining time corresponding to the navigation path is obtained. The remaining time is the time required for the real-time location information to reach the destination.

[0110] If n is greater than 1, the n navigation paths include a current navigation path, the remaining time of the current navigation path is marked as the first remaining time, and the current navigation path is the navigation path currently selected by the vehicle navigation system; the time with the smallest remaining time is selected from the n navigation paths, marked as the second remaining time, and the ratio of the second remaining time to the first remaining time is marked as the time feature value; if the time feature value is greater than or equal to 1, the current navigation path is the optimal path among the n navigation paths; if the time feature value is less than 1, the current navigation path is not the optimal path among the n navigation paths, indicating that the current navigation path may have a traffic jam, resulting in a long time consumption, and the user is prone to irritability. The voice timbre and voice style can be adjusted according to the current time feature value to alleviate the user's irritability caused by traffic jams.

[0111] The ratio of the mileage that the remaining energy (such as the remaining fuel or the remaining power) can support to the remaining distance is marked as the distance characteristic value; the remaining energy can be directly obtained through the vehicle system.

[0112] The time characteristic value and the distance characteristic value are weighted and summed to obtain the driving convenience score;

[0113] ;

[0114] In the formula, Score points for driving convenience; and is the preset weight; is the time feature; The driving convenience score is a route feature. If the driving convenience score is larger, the driving convenience is higher, and the user's driving mood may be more pleasant and relaxed. On the contrary, if the driving convenience score is smaller, the driving convenience is lower, and the user's driving interest may be lower. In this case, the voice timbre and voice style can be adjusted to calm, motivational, etc. to alleviate the user's depressed mood.

[0115] By reasonably determining the feature weights and standardizing the features, the time feature value and the distance feature value can be combined more scientifically to calculate the driving convenience. This not only provides drivers with more valuable driving reference information, but also provides a simpler and more intuitive direction for adjusting the voice timbre and voice style.

[0116] The user portrait label, vehicle safety score and driving convenience score are input into the timbre style recommendation model, and the timbre style recommendation model is trained with the natural optimization inspired algorithm; the recommended voice timbre and voice style are output based on the trained timbre style recommendation model, and the recommended voice timbre and voice style are applied to the car voice broadcast system.

[0117] The training method of the timbre style recommendation model includes:

[0118] Collect training data in advance, including comprehensive feature data and voice timbre and voice style corresponding to the comprehensive feature data; the comprehensive feature data includes user portrait labels, vehicle safety scores, and driving convenience scores;

[0119] Each group of comprehensive feature data is used as the input of the timbre style recommendation model, and the timbre style recommendation model takes the speech timbre and speech style corresponding to each group of comprehensive feature data as output, and takes the actual speech timbre and speech style corresponding to each group of comprehensive feature data as prediction targets; wherein, minimizing the sum of loss function values ​​of all speech timbre and speech style is used as the training target, and the timbre style recommendation model is trained based on the natural optimization inspired algorithm until the sum of prediction errors reaches convergence and the training is stopped; the timbre style recommendation model is a deep neural network model.

[0120] The loss function value of the timbre style recommendation model is as follows:

[0121] ;

[0122] In the formula, is the loss function value; is the number of training data; The prediction function for the timbre style recommendation model is based on the parameters For Training data Produce prediction output; For the The actual output corresponding to the training data, that is, the actual voice timbre and voice style, For the training data; The parameters of the timbre style recommendation model, namely the weights of the user portrait label, vehicle safety score, and driving convenience score, need to be optimized through Cuckoo optimization algorithm training.

[0123] Optimizing the parameters of the timbre style recommendation model through Cuckoo Optimization Algorithm Training The methods include:

[0124] Step 1: Randomly generate H groups of initial solutions, namely nests, each nest represents a set of parameters of the timbre style recommendation model .as follows:

[0125] ;

[0126] In the formula, Recommend model output for timbre style; Label the user portrait; Score the vehicle's safety; Score points for driving convenience; Weight of user portrait label; Weighting for vehicle safety score; Weighting for driving convenience score.

[0127] Step 2: Iterate and update to generate a new nest. The method of generating a new nest includes:

[0128] ;

[0129] In the formula, For the The parameters of the iterations; For the The parameters of the iterations; is the step size factor, which controls the size of the search step; To obey the random step size of Levy distribution, that is, the random number of distribution, Levy flight helps to jump out of the local optimal solution; is the scale parameter that controls the shape of the distribution.

[0130] Step 3: Use Calculate the loss function value of the new nest.

[0131] Step 4: If the loss function value of the new nest is smaller than the loss function value of any nest in group H, the nest corresponding to the loss function value greater than the new nest in group H is replaced with the new nest with a preset probability.

[0132] When the preset number of iterations is reached, the nest with the smallest loss function value is selected as the parameter of the timbre style recommendation model. .

[0133] In the complex parameter space of the voice style recommendation model, traditional algorithms may fall into local optimal solutions, while the cuckoo optimization algorithm can explore the entire parameter space more extensively. For example, based on different weights and biases, these parameters interact with each other to form a complex parameter space. The cuckoo optimization algorithm simulates the parasitic behavior of the cuckoo's nest, discards the current solution with a certain probability and generates a new solution, so that there is a greater chance of finding the global optimal parameter combination, which helps to improve the model's recommendation accuracy for different voice timbres and voice styles.

[0134] Example 2

[0135] This example provides another training method for the timbre style recommendation model, including:

[0136] See also Figure 3 The timbre style recommendation model is designed as a generator, and the FCN network is used as a discriminator, thereby generating an adversarial training network to train the timbre style recommendation model. Through the generation of the adversarial training network, the pre-collected voice timbre and voice style data are used as input samples; the generation of the adversarial training network is trained, and the training method of the generation of the adversarial training network includes:

[0137] Collect training data in advance, including comprehensive feature data and voice timbre and voice style corresponding to the comprehensive feature data; the comprehensive feature data includes user portrait labels, vehicle safety scores, and driving convenience scores;

[0138] Each set of comprehensive feature data is used as the input of the timbre style recommendation model. The timbre style recommendation model uses the voice timbre and voice style corresponding to each set of comprehensive feature data as output. The input sample and the voice timbre and voice style generated by the generator are respectively input into the discriminator as input data; the discriminator discriminates the input data and outputs the discrimination result; the discrimination is 1, indicating that the input data is real; the discrimination is 0, indicating that the input data is generated. The training goal is to minimize the sum of the loss function values ​​of all voice timbre and voice style, that is, to optimize the loss functions of the generator and the discriminator at the same time, so that the overall prediction error is continuously reduced.

[0139] The loss function of the generator is the cross entropy loss.

[0140] The cross entropy loss calculation formula is: ;in, For the generator; is the discriminator; is the comprehensive feature data;

[0141] The loss function of the discriminator is cross entropy loss.

[0142] The cross entropy loss calculation formula is: ;in, For the generator; is the discriminator; is the comprehensive feature data; is the sample label;

[0143] According to the calculated loss functions of the generator and discriminator, the back-propagation algorithm is used to update the weight parameters of the generator and discriminator.

[0144] Based on the above training steps, the naturalness and realism of speech synthesis results synthesized with different voice timbres and different voice styles can be improved, thereby enhancing the user experience.

[0145] Example 3

[0146] See also Figure 4 As shown, the driving situation-based voice active recommendation system provided in this embodiment includes:

[0147] A first acquisition module, used to collect vehicle status data and environmental status data;

[0148] The second collection module is used to collect user data, which includes user personal data and user behavior data;

[0149] A first analysis module is used to extract features from user behavior data to obtain behavior feature data, where the behavior feature data includes driving feature data and interaction feature data;

[0150] The portrait building module is used to use the behavioral feature data and user personal data as inputs of the user portrait model to obtain the user portrait label;

[0151] A data analysis module is used to analyze the status data and environmental status data to obtain the vehicle safety score and driving convenience score;

[0152] The voice recommendation module is used to input user portrait labels, vehicle safety scores and driving convenience scores into the timbre style recommendation model, and train the timbre style recommendation model in combination with the natural optimization inspired algorithm; based on the trained timbre style recommendation model, the recommended voice timbre and voice style are output.

[0153] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

[0154] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A voice active recommendation method based on driving situations, characterized in that: The steps include: Collect vehicle status data and environmental status data; Collect user data, including user personal data and user behavior data; Extracting features from user behavior data to obtain behavior feature data, where the behavior feature data includes driving feature data and interaction feature data; Use behavioral feature data and user personal data as input to the user portrait model to obtain user portrait labels; Analyze status data and environmental status data to obtain vehicle safety scores and driving convenience scores; Inputting the user portrait label, vehicle safety score, and driving convenience score into the trained voice style recommendation model, and outputting the recommended voice timbre and voice style based on the trained voice style recommendation model; The driving characteristic data includes a speed average and a speed standard deviation; The method for obtaining the driving characteristic data includes: User behavior data includes driving operation data and interactive voice data; driving operation data is The acceleration data from the vehicle starting to the scheduled time; Will The acceleration data from the start of the vehicle to the predetermined time is used to establish a speed analysis set, and the speed average and standard deviation of the speed analysis set are calculated; the speed average of the speed analysis set is used as the speed average; and the standard deviation of the speed analysis set is used as the speed standard deviation; The state data includes vehicle driving data, remaining energy and road information; the vehicle driving data includes the current driving speed, acceleration and steering angle of the vehicle; the road information is the slope and curvature data of the vehicle; the state data and environmental state data are input into the safety analysis model to obtain the vehicle safety score; The methods for obtaining driving convenience scores include: Based on the real-time location information, n navigation paths to the destination are obtained. If n is 1, the remaining time corresponding to the navigation path is obtained. The remaining time is the time required for the real-time location information to reach the destination; If n is greater than 1, the n navigation paths include a current navigation path, the remaining time of the current navigation path is marked as the first remaining time, and the current navigation path is the navigation path currently selected by the vehicle navigation system; the time with the smallest remaining time is selected from the n navigation paths and marked as the second remaining time, and the ratio of the second remaining time to the first remaining time is marked as the time characteristic value; if the time characteristic value is greater than or equal to 1, the current navigation path is the optimal path among the n navigation paths; if the time characteristic value is less than 1, the current navigation path is not the optimal path among the n navigation paths; The ratio of the mileage that the remaining energy can support to the remaining distance is marked as the distance characteristic value; The time characteristic value and the distance characteristic value are weighted and summed to obtain the driving convenience score; ； In the formula, Score points for driving convenience; and is the preset weight; is the time feature; is the route feature; The training method of the timbre style recommendation model includes: Collect training data in advance, including comprehensive feature data and voice timbre and voice style corresponding to the comprehensive feature data; the comprehensive feature data includes user portrait labels, vehicle safety scores, and driving convenience scores; Each group of comprehensive feature data is used as the input of the timbre style recommendation model, and the timbre style recommendation model takes the speech timbre and speech style corresponding to each group of comprehensive feature data as output, and takes the actual speech timbre and speech style corresponding to each group of comprehensive feature data as prediction targets; wherein, minimizing the sum of loss function values ​​of all speech timbre and speech style is used as the training target, and the timbre style recommendation model is trained based on the natural optimization inspired algorithm until the sum of prediction errors reaches convergence and the training is stopped; the timbre style recommendation model is a deep neural network model.

2. The method for active voice recommendation based on driving situations according to claim 1, characterized in that: The interactive feature data includes intonation features, speed features and sentiment word score features of the interactive speech data; Methods for obtaining intonation features include: The collected interactive speech data is sampled and quantized, the interactive speech data is converted into digital signals, and the interactive speech data is divided into short frames. The fundamental frequency of each frame of speech is extracted using the autocorrelation method, the cepstrum method or a method based on a neural network, and the mean of the fundamental frequency is used as the intonation feature; Methods for obtaining fast and slow features include: Calculate the duration of the interactive speech data and mark it as speech duration; then divide the number of syllables by the speech duration to obtain the result as the speed feature.

3. The method for active voice recommendation based on driving situations according to claim 2, characterized in that: The method for obtaining the sentiment word score feature includes: Select an emotional dictionary, use speech recognition technology to convert interactive speech data into speech text, and preprocess the converted speech text, including speech text cleaning and word segmentation; encode each word in the preprocessed speech text according to the pinyin and tone of each word to obtain a word code, match each word code in the preprocessed speech text with the vocabulary code in the emotional dictionary one by one, and obtain the corresponding emotional word in the emotional dictionary; Establish corresponding sentiment word category sets for sentiment words of the same category, match the sentiment words in the same sentiment word category set in the preset sentiment word quantization table to obtain the quantization score of each sentiment word, and take the sum of the quantization scores of each sentiment word in the same sentiment word category set as the score of the sentiment word category set; perform weighted summation on the scores of different sentiment word category sets to obtain the sentiment word score feature.

4. The method for active voice recommendation based on driving situation according to claim 1, characterized in that: The training method of the user portrait model includes: Collect R groups of training data in advance. The training data includes behavioral feature data, user personal data and user portrait labels; Each group of behavioral feature data and user personal data is used as the input of the user portrait model, and the user portrait model uses the user portrait label corresponding to each group of behavioral feature data and user personal data as the output, and uses the actual user portrait label corresponding to each group of behavioral feature data and user personal data as the prediction target; minimizing the sum of prediction errors of all user portrait labels is used as the training target; the user portrait model is trained until the sum of prediction errors reaches convergence and the training is stopped; the user portrait model is a deep neural network model.

5. The method for active voice recommendation based on driving situation according to claim 1, characterized in that: The environmental status data includes surrounding environment data and weather conditions; the surrounding environment data includes road signs and distances to obstacles; The road signs include directional signs and prompt signs; The method for obtaining the distance to the obstacle includes: Step 101: input the road image collected by the vehicle-mounted camera into a pre-built image recognition model, and the image recognition model outputs whether the road image contains obstacles; Step 102: If the road image contains an obstacle, the distance between the vehicle and the obstacle is obtained through the vehicle-mounted radar. If there are multiple distances between the vehicle and the obstacle, the smallest distance between the vehicle and the obstacle is used as the distance between the vehicle and the obstacle.

6. The method for active voice recommendation based on driving situations according to claim 1, characterized in that: The loss function value of the timbre style recommendation model is as follows: ； In the formula, is the loss function value; is the number of training data; The prediction function for the timbre style recommendation model is based on the parameters For Training data Produce prediction output; For the The actual output corresponding to the training data, that is, the actual voice timbre and voice style, For the training data; The parameters of the timbre style recommendation model, namely the weights of the user portrait label, vehicle safety score, and driving convenience score, and the parameters of the timbre style recommendation model The optimization is obtained by training using the cuckoo optimization algorithm.

7. The method for active voice recommendation based on driving situations according to claim 1, characterized in that: Use the Cuckoo Optimization Algorithm to train and optimize the parameters of the timbre style recommendation model The methods include: Step 1: Randomly generate H groups of initial solutions, namely nests, each nest represents a set of parameters of the timbre style recommendation model ,as follows: ； In the formula, Recommend model output for timbre style; Label the user portrait; Score the vehicle's safety; Score points for driving convenience; Weight of user portrait labels; Weighting for vehicle safety score; weighting for driving convenience score; Step 2: Iterate and update to generate a new nest. The method of generating a new nest includes: ； In the formula, For the The parameters of the iterations; For the The parameters of the iterations; is the step size factor, which controls the size of the search step; is the random step length that obeys the Levy distribution, that is, the random number of the distribution; is the scale parameter, controlling the shape of the distribution; Step 3: Use Calculate the loss function value of the new nest; Step 4: If the loss function value of the new nest is less than the loss function value of any nest in group H, then the nest corresponding to the loss function value greater than the new nest in group H is replaced with the new nest with a preset probability; When the preset number of iterations is reached, the nest with the smallest loss function value is selected as the parameter of the timbre style recommendation model. .

8. The voice active recommendation system based on driving situations is characterized by: The method for implementing the driving situation-based voice active recommendation method according to any one of claims 1 to 7 comprises: A first acquisition module, used to collect vehicle status data and environmental status data; The second collection module is used to collect user data, which includes user personal data and user behavior data; A first analysis module is used to extract features from user behavior data to obtain behavior feature data, where the behavior feature data includes driving feature data and interaction feature data; The portrait building module is used to use the behavioral feature data and user personal data as inputs of the user portrait model to obtain the user portrait label; A data analysis module is used to analyze the status data and environmental status data to obtain the vehicle safety score and driving convenience score; A voice recommendation module, which is used to input the user portrait label, vehicle safety score and driving convenience score into the trained timbre and style recommendation model, and output the recommended voice timbre and voice style based on the trained timbre and style recommendation model; The driving characteristic data includes a speed average and a speed standard deviation; The method for obtaining the driving characteristic data includes: User behavior data includes driving operation data and interactive voice data; driving operation data is The acceleration data from the vehicle starting to the scheduled time; Will The acceleration data from the start of the vehicle to the predetermined time is used to establish a speed analysis set, and the speed average and standard deviation of the speed analysis set are calculated; the speed average of the speed analysis set is used as the speed average; and the standard deviation of the speed analysis set is used as the speed standard deviation; The state data includes vehicle driving data, remaining energy and road information; the vehicle driving data includes the current driving speed, acceleration and steering angle of the vehicle; the road information is the slope and curvature data of the vehicle; the state data and environmental state data are input into the safety analysis model to obtain the vehicle safety score; The methods for obtaining driving convenience scores include: Based on the real-time location information, n navigation paths to the destination are obtained. If n is 1, the remaining time corresponding to the navigation path is obtained. The remaining time is the time required for the real-time location information to reach the destination; If n is greater than 1, the n navigation paths include a current navigation path, the remaining time of the current navigation path is marked as the first remaining time, and the current navigation path is the navigation path currently selected by the vehicle navigation system; the time with the smallest remaining time is selected from the n navigation paths and marked as the second remaining time, and the ratio of the second remaining time to the first remaining time is marked as the time characteristic value; if the time characteristic value is greater than or equal to 1, the current navigation path is the optimal path among the n navigation paths; if the time characteristic value is less than 1, the current navigation path is not the optimal path among the n navigation paths; The ratio of the mileage that the remaining energy can support to the remaining distance is marked as the distance characteristic value; The time characteristic value and the distance characteristic value are weighted and summed to obtain the driving convenience score; ； In the formula, Score points for driving convenience; and is the preset weight; is the time feature; is the route feature; The training method of the timbre style recommendation model includes: Collect training data in advance, including comprehensive feature data and voice timbre and voice style corresponding to the comprehensive feature data; the comprehensive feature data includes user portrait labels, vehicle safety scores, and driving convenience scores; Each group of comprehensive feature data is used as the input of the timbre style recommendation model, and the timbre style recommendation model takes the speech timbre and speech style corresponding to each group of comprehensive feature data as output, and takes the actual speech timbre and speech style corresponding to each group of comprehensive feature data as prediction targets; wherein, minimizing the sum of loss function values ​​of all speech timbre and speech style is used as the training target, and the timbre style recommendation model is trained based on the natural optimization inspired algorithm until the sum of prediction errors reaches convergence and the training is stopped; the timbre style recommendation model is a deep neural network model.

Images