Cold start method, device, electronic device and storage medium based on large model

Abstract

The present application discloses a cold start method, device, electronic device and storage medium based on a large model. The method includes: obtaining user voice command information and vehicle driving information; analyzing the voice command information to obtain user emotional characteristics and user type characteristics; analyzing the vehicle driving information to determine the corresponding driving scenario; obtaining first audio data and second audio data; analyzing the first audio data and the second audio data to obtain content tendency characteristics; obtaining third audio data; and recalling fourth audio data from the third audio data based on the driving scenario, user emotional characteristics, user type characteristics and content tendency characteristics. This method can achieve zero-sample cold start without the need for user historical data and user privacy data, and accurately push personalized audio to users.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent cockpit systems and recommendation algorithms, and in particular to a cold start method, device, electronic device and storage medium based on a large model. Background Art

[0002] With the widespread adoption of intelligent in-vehicle systems, personalized recommendations have become a core technology for improving user experience. However, in-vehicle interaction data between users and audio content is often highly sparse. This is especially true when users log in for the first time or switch between scenarios (e.g., when a new user first uses the app). This lack of sufficient interaction records renders traditional recommendation models ineffective.

[0003] Existing methods primarily rely on collaborative filtering (CF) and graph neural networks (GNNs) for cold start optimization. Hybrid collaborative filtering models, on the one hand, combine user profiles (such as age and gender) with audio content features (such as metadata and tag word frequency) and utilize matrix factorization techniques to mitigate the sparsity of the interaction matrix. However, these models still rely on historical interaction data, making it difficult to achieve "zero-shot" cold starts. This means that when users or content have no historical behavior, the model cannot generate effective recommendations.

[0004] In addition, existing technologies have significant deficiencies in their understanding of semantics. Traditional content features are difficult to capture complex semantic associations, resulting in recommendation results that tend to favor surface statistical features rather than deep semantic matching. In in-vehicle scenarios, the dynamic driving environment and fragmented user behavior further exacerbate data sparsity. Therefore, it is difficult to accurately push content to users during a zero-sample cold start. Summary of the Invention

[0005] In order to solve the technical problems that existing data push methods are unable to solve zero-sample cold start and have insufficient semantic understanding capabilities, the present invention provides a cold start method, device, electronic device and storage medium based on a large model. The method extracts the user's current driving scene, user emotional characteristics, user type characteristics and content tendency characteristics through the user's voice command information and vehicle driving information, and matches them with the third audio data carrying the corresponding scene layer label, user layer label and content layer label, and recalls the fourth audio data that the user may be interested in. Without the need for user historical data and user privacy data, the method achieves zero-sample cold start and accurately pushes personalized audio to the user. At the same time, through multiple layers of hierarchical label levels, support is provided for semantic understanding, so that the pushed audio is more accurate.

[0006] In a first aspect, an embodiment of the present application provides a cold start method based on a large model, the method comprising:

[0007] Acquiring user voice command information and vehicle driving information; the vehicle driving information includes current location information and current time information;

[0008] Analyze the voice command information to obtain user emotion characteristics and user type characteristics;

[0009] Analyzing the vehicle driving information to determine a corresponding driving scenario;

[0010] Acquire first audio data and second audio data; the first audio data is a hot audio associated with the current time information; the second audio data is a hot audio associated with the current position information;

[0011] Analyze the first audio data and the second audio data to obtain content tendency features;

[0012] Acquire third audio data; the third audio data carries corresponding scene layer tags, user layer tags, and content layer tags; the scene layer tags, the user layer tags, and the content layer tags have a hierarchical relationship;

[0013] Based on the driving scene, the user emotion feature, the user type feature and the content tendency feature, fourth audio data is recalled from the third audio data.

[0014] In an optional embodiment, before recalling the fourth audio data from the third audio data based on the driving scene, the user emotion feature, the user type feature, and the content tendency feature, the method further includes:

[0015] Performing content recognition on the third audio data and converting the third audio data into third text data;

[0016] In response to the received first structured prompt word, extracting a plurality of basic tags of the third text data based on a preset large model;

[0017] In response to the received second structured prompt word, the multiple basic tags are constructed into a hierarchical relationship based on a preset large model to obtain the scene layer tag, the user layer tag and the content layer tag of the third audio data.

[0018] In an optional embodiment, the recalling of the fourth audio data from the third audio data based on the driving scene, the user emotion feature, the user type feature, and the content tendency feature includes:

[0019] If the driving scene matches the scene-level label, determining a first similarity based on the user emotion feature, the user type feature, and the user-level label;

[0020] When the first similarity is greater than a preset similarity threshold, determining a second similarity based on the content tendency feature and the content layer label;

[0021] The fourth audio data is recalled from the third audio data based on the second similarity.

[0022] In an optional embodiment, the scene layer labels have a one-to-one information density;

[0023] If the driving scene matches the scene-layer label, determining a first similarity based on the user emotion feature, the user type feature, and the user-layer label includes:

[0024] Determining a safety factor based on the preset large model based on the driving scenario and the information density;

[0025] If the safety factor is greater than a preset safety factor threshold, determining that the driving scene matches the scene layer label;

[0026] The first similarity is determined based on the user emotion feature, the user type feature, and the user layer label.

[0027] In an optional embodiment, analyzing the voice command information to obtain user emotion characteristics and user type characteristics includes:

[0028] Performing intonation analysis on the voice command information to extract the user's emotional characteristics;

[0029] Performing timbre analysis on the voice command information to extract the user type features; the user type features include age features and gender features.

[0030] In an optional embodiment, the vehicle driving information further includes driving state information and driving environment information;

[0031] The analyzing the vehicle driving information to determine the corresponding driving scenario includes:

[0032] If the driving state information and the driving environment information indicate that the current driving state is safe, determining that the information demand scenario is the corresponding driving scenario; or;

[0033] If the driving state information and the driving environment information indicate that emotional guidance is currently required, the emotional demand scenario is determined to be the corresponding driving scenario.

[0034] In an optional embodiment, the method further includes:

[0035] Obtain updated fifth audio data;

[0036] Determining the scene layer tag, the user layer tag, and the content layer tag corresponding to the fifth audio data;

[0037] Acquire multiple zero-sample users; each of the multiple zero-sample users carries the corresponding driving scenario, the user emotion feature, the user type feature, and the content tendency feature;

[0038] Recall a plurality of users to be shared from the plurality of zero-sample users based on the scene layer tag, the user layer tag, and the content layer tag.

[0039] In a second aspect, an embodiment of the present application provides a cold start device based on a large model, the device comprising:

[0040] A first acquisition module is used to acquire user voice command information and vehicle driving information; the vehicle driving information includes current location information and current time information;

[0041] A first analysis module is used to analyze the voice command information to obtain user emotion characteristics and user type characteristics;

[0042] A second analysis module is used to analyze the vehicle driving information and determine a corresponding driving scenario;

[0043] A second acquisition module is configured to acquire first audio data and second audio data; the first audio data is a popular audio data associated with the current time information; the second audio data is a popular audio data associated with the current location information;

[0044] a third analysis module, configured to analyze the first audio data and the second audio data to obtain content tendency features;

[0045] A third acquisition module is configured to acquire third audio data; the third audio data carries corresponding scene layer tags, user layer tags, and content layer tags; the scene layer tags, the user layer tags, and the content layer tags have a hierarchical relationship;

[0046] The first recall module is used to recall the fourth audio data from the third audio data based on the driving scene, the user emotion characteristics, the user type characteristics and the content tendency characteristics.

[0047] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the large model-based cold start method of the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the large model-based cold start method of the first aspect.

[0049] In a fifth aspect, embodiments of the present application provide a computer program product or computer program, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the large model-based cold start method of the first aspect.

[0050] The large model-based cold start method, device, electronic device, and storage medium provided in the embodiments of the present application have the following technical effects:

[0051] Obtain the user's voice command information and vehicle driving information; the vehicle driving information includes current location information and current time information; analyze the voice command information to obtain user emotional characteristics and user type characteristics; analyze the vehicle driving information to determine the corresponding driving scene; obtain first audio data and second audio data; the first audio data is a popular audio associated with the current time information; the second audio data is a popular audio associated with the current location information; analyze the first audio data and the second audio data to obtain content tendency characteristics; obtain third audio data; the third audio data carries corresponding scene layer labels, user layer labels and content layer labels; there is a hierarchical relationship between the scene layer labels, the user layer labels and the content layer labels; based on the driving scene, the user emotional characteristics, the user type characteristics and the content tendency characteristics, recall the fourth audio data in the third audio data. In an embodiment of the present application, the user's current driving scene, user emotional characteristics, user type characteristics and content tendency characteristics are extracted through the user's voice command information and vehicle driving information, and matched with the third audio data carrying the corresponding scene layer label, user layer label and content layer label, and the fourth audio data that the user may be interested in is recalled. Without the need for user historical data and user privacy data, a zero-sample cold start is achieved, and personalized audio is accurately pushed to the user. At the same time, through multiple layers of hierarchical label levels, support is provided for semantic understanding, so that the pushed audio is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0054] Figure 2 This is a schematic diagram of a cold start method based on a large model provided in an embodiment of the present application. Figure 1 ;

[0055] Figure 3 This is a schematic diagram of a cold start method based on a large model provided in an embodiment of the present application. Figure 2 ;

[0056] Figure 4 This is a schematic diagram of a cold start method based on a large model provided in an embodiment of the present application. Figure 3 ;

[0057] Figure 5 This is a schematic structural diagram of a large-scale model-based cold start device provided in an embodiment of the present application;

[0058] Figure 6 This is a hardware structure block diagram of a server of a large model-based cold start method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0061] See also Figure 1 , Figure 1 It is a schematic diagram of an application environment provided in an embodiment of the present application, including an information collection device 101 and a server 102.

[0062] In a possible embodiment, the information collection device 101 is used to collect user voice command information and vehicle driving information, and specifically includes a voice collection device and a vehicle information collection device.

[0063] In an embodiment of the present application, the voice collection device may include a microphone array arranged inside the vehicle. By arranging 4-8 microphones, sound source positioning, noise suppression and echo cancellation can be achieved, thereby improving the clarity of the voice signal.

[0064] In the embodiment of the present application, the vehicle information collection device may include a

[0065] Positioning and navigation sensors, such as the Global Positioning System (GPS) that obtains vehicle position, speed, and heading in real time, and the Inertial Measurement Unit (IMU) that detects vehicle acceleration, angular velocity, and attitude changes through accelerometers and gyroscopes; it can also include vehicle status sensors, such as wheel speed sensors that monitor wheel speed and calculate mileage and vehicle speed, pressure sensors that detect tire pressure and monitor hydraulic system status in real time, and temperature sensors that monitor engine coolant temperature and in-vehicle air conditioning temperature; it can also include driving behavior sensors, such as steering wheel angle sensors that detect steering angle and speed, or pedal travel sensors that collect accelerator / brake pedal position; it can also include cameras installed inside and outside the vehicle to detect images inside and around the vehicle.

[0066] In an embodiment of the present application, the information collection device 101 may further include an interface connected to a local storage device or a cloud storage device, for obtaining audio data from the local storage device or the cloud storage device.

[0067] In a possible embodiment, the user's voice command information and vehicle driving information are obtained; the vehicle driving information includes current location information and current time information; the voice command information is analyzed to obtain user emotional characteristics and user type characteristics; the vehicle driving information is analyzed to determine the corresponding driving scene; first audio data and second audio data are obtained; the first audio data is a popular audio associated with the current time information; the second audio data is a popular audio associated with the current location information; the first audio data and the second audio data are analyzed to obtain content tendency characteristics; third audio data is obtained; the third audio data carries corresponding scene layer labels, user layer labels and content layer labels; there is a hierarchical relationship between the scene layer labels, the user layer labels and the content layer labels; based on the driving scene, the user emotional characteristics, the user type characteristics and the content tendency characteristics, fourth audio data is recalled in the third audio data. In an embodiment of the present application, the user's current driving scene, user emotional characteristics, user type characteristics and content tendency characteristics are extracted through the user's voice command information and vehicle driving information, and matched with the third audio data carrying the corresponding scene layer label, user layer label and content layer label, and the fourth audio data that the user may be interested in is recalled. Without the need for user historical data and user privacy data, a zero-sample cold start is achieved, and personalized audio is accurately pushed to the user.

[0068] The following describes a specific embodiment of a cold start method based on a large model of the present application. Figure 2 This is a schematic diagram of a cold start method based on a large model provided in an embodiment of the present application. Figure 1 , this specification provides method operation steps such as embodiments or flow charts, but may include more or fewer operation steps based on routine or non-creative work. The order of steps listed in the embodiments is only one way of executing the steps among many steps, and does not represent the only execution order. When the actual system or server product is executed, it can be executed in the order shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the method is applied to the server and may include:

[0069] S201: Acquire user's voice command information and vehicle driving information; the vehicle driving information includes current location information and current time information.

[0070] S202: Analyze the voice command information to obtain user emotion characteristics and user type characteristics.

[0071] S203: Analyze the vehicle driving information to determine a corresponding driving scenario.

[0072] S204: Acquire first audio data and second audio data; the first audio data is a popular audio associated with the current time information; the second audio data is a popular audio associated with the current position information.

[0073] S205: Analyze the first audio data and the second audio data to obtain content tendency features.

[0074] S206: Acquire third audio data; the third audio data carries corresponding scene layer tags, user layer tags, and content layer tags; the scene layer tags, the user layer tags, and the content layer tags have a hierarchical relationship.

[0075] S207: Recalling fourth audio data from the third audio data based on the driving scene, the user emotion characteristics, the user type characteristics, and the content tendency characteristics.

[0076] Figure 3 This is a schematic diagram of a cold start method based on a large model provided in an embodiment of the present application. Figure 2 , the method may include:

[0077] S301: Acquire user's voice command information and vehicle driving information.

[0078] In a possible embodiment, the vehicle driving information includes current location information and current time information. Specifically, voice command information is obtained through a microphone, current location information is obtained through a global positioning system set in the vehicle, and current time information is synchronized through a time synchronization module.

[0079] S302: Analyze the voice command information to obtain user emotion characteristics and user type characteristics.

[0080] In a possible embodiment, analyzing the voice command information to obtain user emotion characteristics and user type characteristics includes:

[0081] S3021: Perform intonation analysis on the voice command information to extract the user's emotional characteristics.

[0082] In a possible embodiment, intonation analysis is performed on the received voice instruction information based on a large model.

[0083] First, the acquired voice command information is pre-processed. Specifically, the RNN-Turbo model is used to separate the human voice from background noise (such as the sound of the air conditioner and navigation broadcast in the car).

[0084] Then the continuous speech is divided into short frames of 20-40ms and the time domain / frequency domain features are extracted.

[0085] Further extract acoustic features such as speech rate (SR), Mel-frequency cepstral coefficients (MFCC), volume (RMS Energy), and extract prosodic features such as fundamental frequency (F0) curve and formants.

[0086] Since people will show higher pitch, faster speaking speed, and louder volume when they are anxious, acoustic features can accurately capture the intonation contour, reflect the number of syllables and emotional intensity per unit time, and prosodic features can reflect the pattern of intonation rise and fall and analyze the shape of the vocal tract, which also helps to distinguish emotional states.

[0087] In this embodiment of the present application, a large model is used to determine the user's current emotional characteristics based on acoustic and prosodic features. Specifically, this model may include multiple levels of emotional labels, for example, upper-level secondary classification labels: positive (0.8 confidence level), neutral (0.5), negative (0.3), and lower-level fine-grained labels: anger (0.9), anxiety (0.7), and joy (0.6).

[0088] For example, if the user inputs the voice command "Navigate to the nearest gas station, hurry up!", the output may be: {"emotion": "anxiety", "confidence": 0.85} .

[0089] S3022: Perform timbre analysis on the voice command information to extract the user type features.

[0090] In a possible embodiment, the voice command information is subjected to timbre analysis based on a large model to extract the user type features, where the user type features include age features and gender features.

[0091] First, the voiceprint and timbre features of the voice command information are extracted. This includes fundamental frequency (F0) statistics. The average fundamental frequency for men is between 85-180Hz, while the average fundamental frequency for women is between 165-255Hz. Harmonic structure analysis is also performed. Women have shorter vocal cords and richer high-frequency harmonics. These voiceprint and timbre features are input into the classification model to infer the user's gender. For example, if a voice command is input, the output may be: {"gender": "male", "confidence": 0.92} .

[0092] Next, we extract the formants, combined speech rate and intonation features, and voiceprint aging characteristics of the voice command information and input them into the age segmentation model to infer the user's age characteristics. For example, if we input a voice command information, we can output: {"age range": "30-45 years old", "confidence level": 0.88}.

[0093] Through the above steps, the user's age and gender characteristics can be extracted, which makes it easier to recommend content that is more likely to be of interest to users of different genders and age groups, thereby achieving stronger targeting.

[0094] S303: Analyze the vehicle driving information to determine a corresponding driving scenario.

[0095] In a possible embodiment, the vehicle driving information further includes driving state information and driving environment information.

[0096] In a possible embodiment, the driving status information may include vehicle speed, acceleration (longitudinal / lateral), braking frequency, steering operation, steering angle, lane keeping deviation rate, etc.

[0097] In a possible embodiment, the driving environment information may involve road conditions (congestion index 0-1), weather conditions (rain / snow / fog), time (day or night), surrounding vehicle density (vehicles / 100 meters), etc.

[0098] By combining the above driving status information and driving environment information, it can be comprehensively judged whether the current state is a regular safe driving state or a state that requires emotional guidance.

[0099] In a possible embodiment, analyzing the vehicle driving information to determine the corresponding driving scenario includes:

[0100] In a possible embodiment, if the driving state information and the driving environment information indicate that the current driving state is safe, the information demand scenario is determined to be the corresponding driving scenario.

[0101] In a possible embodiment, if the driving state information and the driving environment information indicate that emotional guidance is currently required, the emotional demand scenario is determined to be the corresponding driving scenario.

[0102] In the embodiment of the present application, the safety of the current driving state means normal driving under stable road conditions. The safety factor can support the listening of audio with a certain amount of information, or in other words, audio with a certain amount of information will not affect the user's normal driving.

[0103] In the embodiment of the present application, the current need for emotional guidance means that in a complex or high-pressure driving environment (such as congestion or emergency situations), the safety factor cannot support the listening of audio with a certain amount of information, or in other words, audio with a certain amount of information will affect the user's normal driving, and the user needs audio with low or no information content, such as white noise and light music, for emotional guidance.

[0104] In one possible embodiment, when all of the following conditions are met, the current driving state is determined to be safe, and the information demand scenario is determined to be the corresponding driving scenario: (1) vehicle speed stability: the current vehicle speed change rate is <5%; (2) driving smoothness: the lateral acceleration standard deviation is <0.3m / s²; (3) the environment is low risk: the road condition index is ≤0.3; (4) the weather is not bad (not rain / snow / fog); and (5) the time period is daytime. Otherwise, it is determined that emotional guidance is currently required, and the emotional demand scenario is determined to be the corresponding driving scenario.

[0105] In another possible embodiment, when any of the following conditions are met, the congestion pressure determines that emotional guidance is currently required, and the emotional demand scenario is determined to be the corresponding driving scenario: (1) the road condition index is greater than 0.7 and the duration is ≥10 minutes; (2) sudden interference: the number of sudden accelerations and decelerations is greater than 3 times / minute and the lane departure rate is greater than 15%; (3) environmental pressure: the weather is rainy, snowy, or foggy; (4) the time is nighttime (18:00-6:00) and the headlights are not turned on. Otherwise, the current driving state is determined to be safe, and the information demand scenario is determined to be the corresponding driving scenario.

[0106] S304: Acquire first audio data and second audio data.

[0107] In a possible embodiment, the first audio data is a hot audio associated with the current time information; and the second audio data is a hot audio associated with the current position information.

[0108] S305: Analyze the first audio data and the second audio data to obtain content tendency features.

[0109] That is to say, the first audio data represents the popular audio of the month, day or time, and the second audio data represents the popular audio of the province, city or region. Then, the big model is used to analyze the popular audio of the first audio data and the second audio data in two dimensions to obtain the content tendency characteristics under the current time and space state.

[0110] Specifically, the first audio data and the second audio data are first merged and deduplicated, and then the content is parsed to extract text features (keywords such as "technology" and "finance"), emotional tendencies (positive, neutral, or negative), rhythm features (music style such as electronic music), and timbre features in the title or description.

[0111] The above features are then input into the large model, prompt words are designed, content tendency features are generated, and a three-level structured prompt word is specifically constructed, including task definition, assigning specific identities, activating relevant domain knowledge, and then defining the constraint condition layer (JSON output format, including relevant data albums, singles, and radio names). Finally, relevant cases are given as a few samples to inject in-vehicle domain knowledge.

[0112] For example, the first audio data input is the current popular content: {XXX,XXX}, and the second audio data is the regional popular content: {XXX playback volume ↑35%, XXX collection rate ↑22%}, and finally some audio features are output as the user's potential listening audio resources.

[0113] S306: Acquire third audio data.

[0114] In a possible embodiment, the third audio data is a local or cloud-based audio resource to be recommended, including various types such as podcasts, audio books, songs, light music, white noise, etc.

[0115] S307: Perform content recognition on the third audio data and convert it into third text data.

[0116] In a possible embodiment, for each third audio data, the third audio data may be pre-processed such as segmentation processing and noise reduction processing to improve audio quality, reduce noise interference, and adapt to a subsequent processing model.

[0117] Then, the pre-processed third audio data is subjected to sound recognition based on the sound recognition model to obtain the initial text information. Specifically, the pre-processed audio information is subjected to automatic speech recognition based on the locally built ASR model, and the initial text information is cached.

[0118] In a possible embodiment, the initial text information may also be cleaned by removing stop words, correcting spelling errors, and normalizing numbers to obtain third text data, optimize text quality, and facilitate subsequent semantic analysis.

[0119] S308: In response to the received first structured prompt word, extract a plurality of basic tags of the third text data based on a preset large model.

[0120] In a possible embodiment, the first structured prompt word is also a three-level structured prompt word, including first task specification information, first constraint condition information and first example information.

[0121] In an embodiment of the present application, the first task specification information, the first constraint information and the first example information constitute a three-level prompt word structure. Through the first task specification information, the model is explicitly required to perform "basic label generation". The number of labels generated (for example, 3), the output format (for example, Json structure) and the output results (label text and label confidence score) are specified through the first constraint information specification. Finally, the first example information is used as a few-shot layer to inject vehicle-borne domain knowledge and provide some sample inputs and outputs.

[0122] For example, the first task specification requires the model to generate basic labels based on text information. The first constraint requires the generated labels to be three, with the output format being JSON. The output includes the label text and the label confidence score. For example, the first example might be: the audio information plays the media message "Fold a thousand paper cranes and tie a red belt," and the output labels are "festive" (0.88), "Spring Festival" (0.64), and "thousand paper cranes" (0.96).

[0123] S309: In response to the received second structured prompt word, the multiple basic tags are constructed into a hierarchical relationship based on a preset large model, and the scene layer tag, the user layer tag and the content layer tag of the third audio data are obtained.

[0124] In a possible embodiment, the second structured prompt word is also a three-level structured prompt word, including second task specification information, second constraint condition information, and second example information.

[0125] The second task specification information explicitly requires the model to "build a hierarchical system for basic tags, including a scenario layer, a user layer, and a content layer, and the layers need to be related." The second constraint information specifies the number of layers (for example, 3), the output format (for example, a Json structure), and the output results (layer text and category names). Finally, the second example information is used as a few-shot layer to specify some desired category names and provide some sample inputs and outputs.

[0126] For example, scene-level labels include: medium-information scene (0.40), user-level labels: “festive” (0.88), and content-level labels: “Spring Festival” (0.64), “thousand paper cranes” (0.96).

[0127] Through the above process, the third audio data carries the corresponding scene layer tag, user layer tag and content layer tag, and there is a hierarchical relationship between the scene layer tag, the user layer tag and the content layer tag.

[0128] S310: Recalling fourth audio data from the third audio data based on the driving scenario, the user emotion characteristics, the user type characteristics, and the content tendency characteristics.

[0129] In a possible embodiment, the recalling of the fourth audio data from the third audio data based on the driving scene, the user emotion feature, the user type feature, and the content tendency feature includes:

[0130] For each third audio data, perform:

[0131] S3101: Determine whether the driving scene matches the scene layer label. If so, execute S3102; if not, execute S3106.

[0132] S3102: Determine a first similarity based on the user emotion feature, the user type feature, and the user layer label.

[0133] In a possible embodiment, the above features are vectorized, and the cosine similarity between the vectors is calculated as the first similarity.

[0134] S3103: Determine whether the first similarity is greater than a preset similarity threshold. If so, execute S3104; if not, execute S3106.

[0135] S3104: Determine a second similarity based on the content tendency feature and the content layer label.

[0136] In a possible embodiment, the above features are vectorized, and the cosine similarity between the vectors is calculated as the second similarity.

[0137] S3105: Recall fourth audio data from the third audio data based on the second similarity.

[0138] In a possible embodiment, the third audio data are sorted based on the second similarity, and the first N required audio data are taken as the fourth audio data.

[0139] S3106: Determine that the third audio data does not meet the requirements.

[0140] The scene-level labels have a one-to-one correspondence with information density. That is, for an audio data set, it has information density. For example, podcasts and audiobooks have the highest information density, and their scene-level labels are high-information scenes (above 0.5). Ordinary songs have medium information density, and their scene-level labels are medium-information scenes (0.2-0.5). Light music or white noise has the lowest information density, and their scene-level labels are medium-information scenes (below 0.2).

[0141] In a possible embodiment, if the driving scene matches the scene-level label, determining a first similarity based on the user emotion feature, the user type feature, and the user-level label includes:

[0142] Based on the driving scenario and the information density, a safety factor is determined based on the preset large model. Specifically, the preset large model is used to simulate the safety factor of audio data received under the driving scenario with the information density, thereby obtaining an accurate value to facilitate subsequent calculations and comparisons.

[0143] Optionally, if the safety factor is greater than a preset safety factor threshold, it is determined that the driving scene matches the scene layer label; otherwise, if the safety factor is less than or equal to the preset safety factor threshold, it is determined that the driving scene does not match the scene layer label.

[0144] The first similarity is determined based on the user emotion feature, the user type feature, and the user layer label.

[0145] Figure 4 This is a schematic diagram of a cold start method based on a large model provided in an embodiment of the present application. Figure 3 , the method may include:

[0146] S401: Acquire updated fifth audio data.

[0147] In a possible embodiment, the fifth audio data is updated audio data, such as a new album, single or broadcast, which needs to be recommended to the user.

[0148] S402: Determine the scene layer tag, the user layer tag, and the content layer tag corresponding to the fifth audio data.

[0149] S403: Acquire multiple zero-sample users.

[0150] In the embodiment of the present application, the multiple zero-sample users respectively carry the corresponding driving scene, the user emotion feature, the user type feature, and the content tendency feature. The specific feature extraction process is as described above and will not be repeated here.

[0151] S404: Recall multiple users to be shared from the multiple zero-sample users based on the scene layer tag, the user layer tag, and the content layer tag.

[0152] Similar to the recall of multiple fourth audio data from multiple third audio data described above, scene matching is also performed first, then the first similarity and the second similarity are calculated, and finally sorted based on the second similarity, and the first N zero-sample users are taken as the users to be shared.

[0153] This application can achieve horizontal audio matching by setting multi-level labels and multi-layer user-side features, effectively improving the accuracy and comprehensiveness of user portraits of zero-sample users, and effectively improving the scenario-based and hierarchical classification of audio resources. It plays an important role in the process of recommending audio data to users and in the process of finding users for audio data.

[0154] The embodiment of the present application also provides a cold start device based on a large model, Figure 5This is a schematic diagram of the structure of a cold start device based on a large model provided in an embodiment of the present application. Figure 5 As shown, the device 500 includes:

[0155] A first acquisition module 510 is configured to acquire user voice command information and vehicle driving information; the vehicle driving information includes current location information and current time information;

[0156] A first analysis module 520 is used to analyze the voice command information to obtain user emotion characteristics and user type characteristics;

[0157] A second analysis module 530 is used to analyze the vehicle driving information and determine a corresponding driving scenario;

[0158] The second acquisition module 540 is configured to acquire first audio data and second audio data; the first audio data is a popular audio data associated with the current time information; the second audio data is a popular audio data associated with the current location information;

[0159] A third analysis module 550 is configured to analyze the first audio data and the second audio data to obtain content tendency features;

[0160] A third acquisition module 560 is configured to acquire third audio data; the third audio data carries corresponding scene layer tags, user layer tags, and content layer tags; the scene layer tags, the user layer tags, and the content layer tags have a hierarchical relationship;

[0161] The first recall module 570 is configured to recall the fourth audio data from the third audio data based on the driving scene, the user emotion feature, the user type feature, and the content tendency feature.

[0162] In an optional embodiment, the method further includes:

[0163] a content conversion module, configured to perform content recognition on the third audio data and convert the third audio data into third text data;

[0164] a first labeling module, configured to extract a plurality of basic labels for the third text data based on a preset large model in response to the received first structured prompt word;

[0165] The second label module is used to respond to the received second structured prompt word, construct a hierarchical relationship between the multiple basic labels based on a preset large model, and obtain the scene layer label, the user layer label and the content layer label of the third audio data.

[0166] In an optional embodiment, the method includes:

[0167] a first determining module, configured to determine a first similarity based on the user emotion feature, the user type feature, and the user layer label if the driving scene matches the scene layer label;

[0168] a second determining module, configured to determine a second similarity based on content tendency features and the content layer label when the first similarity is greater than a preset similarity threshold;

[0169] The second recall module is configured to recall the fourth audio data from the third audio data based on the second similarity.

[0170] In an optional embodiment, the scene layer labels have a one-to-one information density, including:

[0171] a third determination module, configured to determine a safety factor based on the preset large model and the driving scenario and the information density;

[0172] a fourth determining module, configured to determine that the driving scene matches the scene layer label if the safety factor is greater than a preset safety factor threshold;

[0173] A fifth determination module is configured to determine the first similarity based on the user emotion feature, the user type feature, and the user layer label.

[0174] In an optional embodiment, the method includes:

[0175] A first feature extraction module is used to perform intonation analysis on the voice command information to extract the user's emotional features;

[0176] A second feature extraction module is used to perform timbre analysis on the voice command information to extract the user type feature; the user type feature includes age feature and gender feature;

[0177] In an optional embodiment, the vehicle driving information further includes driving state information and driving environment information; including:

[0178] A sixth determining module is configured to determine that the information demand scenario is a corresponding driving scenario if the driving state information and the driving environment information indicate that the current driving state is safe; or;

[0179] The seventh determination module is configured to determine an emotional demand scenario as a corresponding driving scenario if the driving state information and the driving environment information indicate that emotional guidance is currently required.

[0180] In an optional embodiment, the method further includes:

[0181] A third acquisition module, configured to acquire updated fifth audio data;

[0182] an eighth determining module, configured to determine the scene layer label, the user layer label, and the content layer label corresponding to the fifth audio data;

[0183] A fourth acquisition module is configured to acquire a plurality of zero-sample users; the plurality of zero-sample users respectively carry the corresponding driving scenario, the user emotion feature, the user type feature, and the content tendency feature;

[0184] The third recall module is used to recall multiple users to be shared from the multiple zero-sample users based on the scene layer label, the user layer label and the content layer label.

[0185] The device and method embodiments in the embodiments of this application are based on the same application concept.

[0186] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 6 This is a hardware structure diagram of a server of a cold start method based on a large model provided in an embodiment of the present application. Figure 6 As shown, the server 600 may vary significantly depending on its configuration or performance. It may include one or more central processing units (CPUs) 610 (CPUs 610 may include, but are not limited to, processing devices such as microprocessors (MCUs) or programmable logic devices (FPGAs), memory 630 for storing data, and one or more storage media 620 (e.g., one or more mass storage devices) for storing applications 623 or data 622. Memory 630 and storage media 620 may be either transient or persistent storage. The program stored in storage media 620 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, CPU 610 may be configured to communicate with storage media 620 to execute the series of instruction operations stored in storage media 620 on server 600. The server 600 may also include one or more power supplies 660, one or more wired and wireless network interfaces 650, one or more input and output interfaces 640, and / or one or more operating systems 621, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0187] The input / output interface 640 can be used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the server 600. In one embodiment, the input / output interface 640 may include a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the input / output interface 640 may be a radio frequency (RF) module for wireless communication with the Internet.

[0188] It can be understood by those skilled in the art that Figure 6 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 6 More or fewer components than shown, or with Figure 6 Different configurations shown.

[0189] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the above-mentioned data processing method.

[0190] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a server to store at least one instruction, at least one program, code set or instruction set related to a cold start method based on a large model in an embodiment of the method. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the above-mentioned cold start method based on a large model.

[0191] Optionally, in this embodiment, the storage medium may be located in at least one of a plurality of network servers in the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk, among other media capable of storing program code.

[0192] It can be seen from the embodiments of the large model-based cold start method, device, electronic device or storage medium provided by the above-mentioned present application that the user's voice command information and vehicle driving information are obtained in the present application; the vehicle driving information includes current location information and current time information; the voice command information is analyzed to obtain user emotional characteristics and user type characteristics; the vehicle driving information is analyzed to determine the corresponding driving scene; first audio data and second audio data are obtained; the first audio data is a popular audio associated with the current time information; the second audio data is a popular audio associated with the current location information; the first audio data and the second audio data are analyzed to obtain content tendency characteristics; third audio data is obtained; the third audio data carries corresponding scene layer labels, user layer labels and content layer labels; there is a hierarchical relationship between the scene layer labels, the user layer labels and the content layer labels; based on the driving scene, the user emotional characteristics, the user type characteristics and the content tendency characteristics, the fourth audio data is recalled in the third audio data. In an embodiment of the present application, the user's current driving scene, user emotional characteristics, user type characteristics and content tendency characteristics are extracted through the user's voice command information and vehicle driving information, and matched with the third audio data carrying the corresponding scene layer label, user layer label and content layer label, and the fourth audio data that the user may be interested in is recalled. Without the need for user historical data and user privacy data, a zero-sample cold start is achieved, and personalized audio is accurately pushed to the user.

[0193] It should be noted that the order of the embodiments of the present application described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0194] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0195] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0196] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A cold start method based on a large model, characterized in that: include; Acquiring user voice command information and vehicle driving information; the vehicle driving information includes current location information and current time information; Analyze the voice command information to obtain user emotion characteristics and user type characteristics; Analyzing the vehicle driving information to determine a corresponding driving scenario; Acquire first audio data and second audio data; the first audio data is a popular audio associated with the current time information; The second audio data is a popular audio associated with the current position information; Analyze the first audio data and the second audio data to obtain content tendency features; Acquire third audio data; the third audio data carries corresponding scene layer tags, user layer tags, and content layer tags; There is a hierarchical relationship between the scene layer tag, the user layer tag and the content layer tag; Recalling fourth audio data from the third audio data based on the driving scene, the user emotion feature, the user type feature, and the content tendency feature; Before recalling the fourth audio data from the third audio data based on the driving scenario, the user emotion feature, the user type feature, and the content tendency feature, the method further includes: Performing content recognition on the third audio data and converting the third audio data into third text data; In response to the received first structured prompt word, extracting a plurality of basic tags of the third text data based on a preset large model; In response to the received second structured prompt word, constructing a hierarchical relationship among the multiple basic tags based on a preset large model, and obtaining a scene layer tag, a user layer tag, and a content layer tag of the third audio data; The recalling of the fourth audio data from the third audio data based on the driving scenario, the user emotion feature, the user type feature, and the content tendency feature includes: If the driving scene matches the scene-layer label, determining a first similarity based on the user emotion feature, the user type feature, and the user-layer label; the scene-layer label having a one-to-one corresponding information density; When the first similarity is greater than a preset similarity threshold, determining a second similarity based on the content tendency feature and the content layer label; recalling fourth audio data from the third audio data based on the second similarity; If the driving scene matches the scene-layer label, determining a first similarity based on the user emotion feature, the user type feature, and the user-layer label includes: determining a safety factor based on the driving scenario, the information density, and the preset large model; If the safety factor is greater than a preset safety factor threshold, determining that the driving scene matches the scene layer label; The first similarity is determined based on the user emotion feature, the user type feature, and the user layer label.

2. A cold start method based on a large model according to claim 1, characterized in that: The analyzing the voice command information to obtain user emotion characteristics and user type characteristics includes: Performing intonation analysis on the voice command information to extract the user's emotional characteristics; Performing timbre analysis on the voice command information to extract the user type features; the user type features include age features and gender features.

3. A cold start method based on a large model according to claim 1, characterized in that: The vehicle driving information also includes driving state information and driving environment information; The analyzing the vehicle driving information to determine the corresponding driving scenario includes: If the driving state information and the driving environment information indicate that the current driving state is safe, determining that the information demand scenario is the corresponding driving scenario; or; If the driving state information and the driving environment information indicate that emotional guidance is currently required, the emotional demand scenario is determined to be the corresponding driving scenario.

4. A cold start method based on a large model according to claim 1, characterized in that: Also includes: Get updated fifth audio data; Determining the scene layer tag, the user layer tag, and the content layer tag corresponding to the fifth audio data; Acquire multiple zero-sample users; each of the multiple zero-sample users carries the corresponding driving scenario, the user emotion feature, the user type feature, and the content tendency feature; Recall a plurality of users to be shared from the plurality of zero-sample users based on the scene layer tag, the user layer tag, and the content layer tag.

5. A cold start device based on a large model, characterized in that: include: The first acquisition module is used to obtain the user's voice command information and vehicle driving information; The vehicle driving information includes current location information and current time information; A first analysis module is used to analyze the voice command information to obtain user emotion characteristics and user type characteristics; A second analysis module is used to analyze the vehicle driving information and determine a corresponding driving scenario; A second acquisition module is configured to acquire first audio data and second audio data; the first audio data is a popular audio data associated with the current time information; the second audio data is a popular audio data associated with the current location information; a third analysis module, configured to analyze the first audio data and the second audio data to obtain content tendency features; A third acquisition module is used to acquire third audio data; the third audio data carries corresponding scene layer tags, user layer tags and content layer tags; There is a hierarchical relationship between the scene layer tag, the user layer tag and the content layer tag; a first recall module, configured to recall fourth audio data from the third audio data based on the driving scene, the user emotion feature, the user type feature, and the content tendency feature; Before recalling the fourth audio data from the third audio data based on the driving scenario, the user emotion feature, the user type feature, and the content tendency feature, the method further includes: Performing content recognition on the third audio data and converting the third audio data into third text data; In response to the received first structured prompt word, extracting a plurality of basic tags of the third text data based on a preset large model; In response to the received second structured prompt word, constructing a hierarchical relationship among the multiple basic tags based on a preset large model, and obtaining a scene layer tag, a user layer tag, and a content layer tag of the third audio data; The recalling of the fourth audio data from the third audio data based on the driving scenario, the user emotion feature, the user type feature, and the content tendency feature includes: If the driving scene matches the scene-layer label, determining a first similarity based on the user emotion feature, the user type feature, and the user-layer label; the scene-layer label having a one-to-one corresponding information density; When the first similarity is greater than a preset similarity threshold, determining a second similarity based on the content tendency feature and the content layer label; recalling fourth audio data from the third audio data based on the second similarity; If the driving scene matches the scene-layer label, determining a first similarity based on the user emotion feature, the user type feature, and the user-layer label includes: Determining a safety factor based on the preset large model based on the driving scenario and the information density; If the safety factor is greater than a preset safety factor threshold, determining that the driving scene matches the scene layer label; The first similarity is determined based on the user emotion feature, the user type feature, and the user layer label.

6. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the large model-based cold start method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the large model-based cold start method as described in any one of claims 1-4.

Images