Training method of VOC processing large language model, VOC processing method, equipment and medium

Abstract

The invention discloses a training method of a VOC processing large language model, a VOC processing method, equipment and a medium. The method comprises the following steps: screening an initial model from a plurality of candidate large language models; determining a training data set, wherein the training data set comprises a plurality of pieces of user feedback information marked with classification labels; training the initial model by using the training data set until the value of a preset mixed loss function reaches the minimum, and obtaining a trained VOC processing large language model; wherein the trained VOC processing large language model is used for outputting a comment analysis report according to received initial cue words and intermediate cue words, and the initial cue words comprise a collection address of user feedback information, a processing rule for the user feedback information and a format of data returned by the VOC processing large language model; the intermediate cue word comprises a processing rule for user feedback information and a format of data returned by the VOC processing large language model.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a training method, VOC processing method, device, and medium for large language models. Background Technology

[0002] Enterprises' user websites (such as cross-border payment businesses involving multiple foreign banks, service trade businesses involving multiple domestic distribution channels, etc.) need to handle massive amounts of user feedback information from multiple channels such as customer service system inquiries, product homepage proposal submissions, and social media comments. Moreover, these user feedback information have problems such as scattered descriptions, significant differences in regionality and timeliness.

[0003] Large Language Models (LLMs) are artificial intelligence models built on deep learning architectures (with the Transformer at its core), pre-trained on massive amounts of data, and possessing a large parameter scale and strong generalization ability. They can learn general rules and knowledge from data and have cross-task and cross-scenario generalization capabilities. To achieve automated clustering of user issues, statistical analysis of feedback volume, and priority determination, accurately understand user needs and concerns, and provide data support for product optimization, review process iteration, and user experience improvement, it is crucial to leverage the natural language processing capabilities of large language models to efficiently extract core information from massive amounts of user feedback.

[0004] However, current conventional large language models have poor text analysis and counting capabilities, are not sensitive to analysis, statistics, and numbers, and due to the limited size of the context window, they are unable to handle millions of texts. They cannot solve the problem of efficient and automated clustering of millions of user feedback information by standard context window large language models, and it is difficult to establish a strict mapping relationship between large categories and original text. As a result, they cannot accurately count the size of user voices for corresponding issues, and cannot meet the core demand of enterprises to quickly and accurately understand the needs of massive users. Summary of the Invention

[0005] This invention aims to address, to a certain extent, one of the technical problems in related technologies. To this end, this invention provides a training method, VOC processing method, device, and medium for large language models that handle VOC processing. These methods offer advantages such as enhancing the model's text analysis and statistical capabilities, ensuring standardized and consistent comment analysis reports, enabling large language models with standard context windows to effectively process large-scale text, and providing data support for improving user experience.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for training a large language model using VOC processing, comprising: The initial model is selected from multiple candidate large language models; Determine the training dataset, which includes multiple user feedback messages labeled with classification tags; The initial model is trained using the training dataset until the value of the preset mixture loss function reaches its minimum, thus obtaining a trained VOC processing large language model. The trained VOC processing large language model is used to output a comment analysis report based on received initial prompts and intermediate prompts. The initial prompts include the address where user feedback information was collected, the rules for processing user feedback information, and the format of the data returned by the VOC processing large language model. The intermediate prompts include the rules for processing user feedback information and the format of the data returned by the VOC processing large language model.

[0007] Optionally, training the initial model using the training dataset until the value of the preset mixture loss function is minimized to obtain the trained VOC processing large language model includes: The training dataset is split into supervised fine-tuned SFT samples and reinforcement learning RL samples; The initial model is subjected to supervised fine-tuning SFT using the supervised fine-tuning SFT samples to obtain the supervised fine-tuning SFT base model. The reinforcement learning (RL) samples are labeled with the keyword "Ground Truth," which includes a list of reference keywords and the total number of reference keywords. Based on the reinforcement learning RL samples labeled with the keyword "Ground Truth" and the preset hybrid loss function, the parameters of the supervised fine-tuning SFT base model are iteratively adjusted until the value of the preset hybrid loss function reaches its minimum, thus obtaining a trained VOC processing large language model.

[0008] Optionally, the preset hybrid loss function includes: (1); (2); (3); In formulas (1)-(3), This indicates that the parameters of the current model are The value of the preset hybrid loss function at that time. This represents the weight coefficients of the supervised fine-tuning SFT loss function. The weight coefficients represent the weights of the reinforcement learning (RL) loss function. This indicates the total number of tokens output by the current model. Indicates the current model output of the first... One token, This represents the user feedback information input. Indicates parameters The current model for time, This indicates that the parameters of the current model are The autoregressive generation probability at time, This indicates that the current model is generating the first... Step output Time depends on the current model in the first All historical outputs generated before the step, This represents the baseline used to reduce the variance of the policy gradient estimate. Indicates recall rate, Indicates the repetition rate. This represents the balance coefficient between recall and duplicate rate (DupRate). This indicates the number of keywords correctly identified by the current model. This indicates the total number of reference keywords in the keyword "Ground Truth". This indicates the number of repeated keywords in the current model output. This indicates the total number of keywords output by the current model.

[0009] Optionally, the step of selecting an initial model from multiple candidate large language models includes: Obtain multiple user feedback messages for filtering, and determine the keyword identification reference results and keyword classification reference results corresponding to the multiple user feedback messages for filtering; The multiple filtering user feedback information are respectively input into each of the candidate large language models to obtain the keyword recognition prediction results and keyword classification prediction results output by each of the candidate large language models; For any of the candidate large language models, its comprehensive accuracy is determined based on its output keyword recognition prediction results and keyword classification prediction results, as well as the keyword recognition reference results and keyword classification reference results. The model with the highest overall accuracy among multiple candidate large language models is selected as the initial model.

[0010] The training method for a VOC processing large language model provided by this invention, by selecting an initial model from multiple candidate large language models, ensures the basic adaptability of the model, laying the foundation for improving the model's text analysis and statistical capabilities, and effectively addressing the weakness of conventional large language models in terms of analytical and statistical capabilities. By determining a training dataset, which includes multiple user feedback information labeled with classification tags, it closely matches the actual VOC scenario, providing accurate supervision signals for model training. This facilitates the model in establishing a mapping relationship between the original text and the corresponding categories, supporting the statistical analysis of user voice volume. The initial model is trained using the training dataset until the preset mixture loss function reaches its minimum value, resulting in a trained VOC processing large language model. The model balances lightweight design with training accuracy. Furthermore, the trained VOC processing large language model can output comment analysis reports based on received initial and intermediate prompts. The initial prompts include the address where user feedback was collected, the rules for processing user feedback, and the format of the data returned by the VOC processing large language model. The intermediate prompts also include the rules for processing user feedback and the format of the data returned by the VOC processing large language model. This ensures the standardization and uniformity of comment analysis reports while adapting to scenarios where the number of tokens exceeds the model's context window through intermediate prompts. It enables the large language model with a standard context window to effectively process large-scale text, providing data support for improving user experience.

[0011] Furthermore, the present invention also provides a VOC processing method, comprising: The initial prompt words are sent to the trained VOC processing large language model; wherein, the initial prompt words include the collection address of user feedback information, the processing rules for user feedback information, and the format of the data returned by the VOC processing large language model, the VOC processing large language model is obtained by selecting the initial model from multiple candidate large language models, and training the initial model using a training dataset including multiple user feedback information labeled with classification tags until the value of the preset mixture loss function reaches the minimum; If the number of tokens in the comment analysis report returned by the VOC processing large language model exceeds the context window of the VOC processing large language model, intermediate prompt words are sent to the VOC processing large language model until the number of tokens in the comment analysis report returned by the VOC processing large language model does not exceed the context window of the VOC processing large language model; wherein, the intermediate prompt words include the processing rules for user feedback information and the format of the data returned by the VOC processing large language model; Obtain the comment analysis report currently returned by the VOC processing large language model.

[0012] Optionally, the processing rules include core text fragment extraction rules, coarse classification rules, and fine classification rules; The core text fragment extraction rule is used to prompt the VOC processing large language model to extract no more than a first preset number of core text fragments from each user feedback information; The coarse classification rule is used to prompt the VOC processing large language model to divide all the extracted core text fragments into multiple batches according to the first preset sample number length, divide the core text fragments of each batch into a second preset number of temporary categories with the number of words in each temporary category not exceeding the first preset number, and perform semantic summarization on all the temporary categories to obtain a second preset number of coarse categories. The detailed classification rules are used to prompt the VOC processing large language model to divide all core text fragments corresponding to the second preset number of coarse categories into multiple batches according to the second preset number of samples, divide the core text fragments of each batch into a third preset number of detailed categories with the number of words in each detailed category not exceeding the first preset number, and summarize the detailed classification of all batches and return the results to the comment analysis report. The first preset sample number length, the second preset sample number length, the first preset number, the second preset number, and the third preset number are determined according to the VOC processing large language model context window size.

[0013] Optionally, the processing rules also include classification correction rules, which are used to prompt the VOC processing large language model to perform a preset correction action when the current classification result triggers the correction condition. The correction conditions include any of the following or a combination thereof: the current classification result contains a misclassified original sound whose semantics do not match the category; the current classification result contains an unclassified original sound that is not assigned to any category; the current classification result contains a redundant category with a similarity not lower than a preset similarity threshold. The preset correction actions include any of the following or a combination thereof: reclassifying the misclassified original sound to a target category that matches its semantics, reclassifying the unclassified original sound to determine its category, and reclassifying the original sound corresponding to redundant categories to multiple subcategories with semantic similarity lower than a preset similarity threshold.

[0014] Optionally, the prompt words also include dynamic classification adjustment rules, which include rules for automatically adding classifications, rules for automatically eliminating classifications, and rules for classifying and splitting categories. The automatic addition of classification rules is used to prompt the VOC processing large language model to statistically analyze the output ratio of each category in all returned comment analysis reports according to a preset time period, and to add any category to the current sub-category matrix if the output ratio of any category exceeds the first preset ratio threshold and the category does not exist in the current sub-category matrix. The automatic elimination classification rule is used to prompt the VOC processing large language model to statistically analyze the output ratio of each category in all returned comment analysis reports according to a preset time period, and to eliminate the category from the current sub-category matrix if the output ratio of any category is lower than the second preset ratio threshold and the category exists in the current sub-category matrix. The classification splitting rule is used to prompt the VOC processing large language model to statistically analyze the output proportion of each category in all returned comment analysis reports according to a preset time period, and to execute the above VOC processing method when the output proportion of any category exceeds a third preset proportion threshold and that category exists in the current sub-category matrix, wherein the third preset proportion threshold is greater than the first preset proportion threshold.

[0015] This invention first sends initial prompts to a trained VOC processing large language model, including the address for collecting user feedback, the rules for processing user feedback, and the format of the data returned by the VOC processing large language model. If the number of tokens in the comment analysis report returned by the VOC processing large language model exceeds the context window of the VOC processing large language model, then intermediate prompts are sent to the VOC processing large language model, including the rules for processing user feedback and the format of the data returned by the VOC processing large language model, until the number of tokens in the comment analysis report returned by the VOC processing large language model does not exceed the context window of the VOC processing large language model. This allows the VOC processing large language model to dynamically process the acquired user feedback in a hierarchical manner, enabling even a standard context window VOC processing large language model to handle user feedback of any scale. This ensures both the standardization and uniformity of comment analysis reports and adapts to scenarios where the number of tokens exceeds the model's context window through intermediate prompts. It achieves effective processing of large-scale text by a large language model with a standard context window, providing data support for improving user experience.

[0016] Furthermore, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to achieve any of the following: The training method for VOC processing of large language models described in any of the above items; The VOC processing method described in any of the above items.

[0017] Furthermore, the present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, performs any of the following: The training method for VOC processing of large language models described in any of the above items; The VOC processing method described in any of the above items.

[0018] These features and advantages of the present invention will be disclosed in detail in the following specific embodiments and accompanying drawings. The preferred embodiments or means of the present invention will be shown in detail in conjunction with the accompanying drawings, but are not intended to limit the technical solutions of the present invention. In addition, each of these features, elements and components appearing in the following text and drawings is a plurality of, and different symbols or numbers are used for convenience of representation, but all represent parts with the same or similar construction or function. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings: Figure 1 A flowchart illustrating one implementation of the VOC processing large language model training method provided in this invention; Figure 2 A flowchart illustrating another implementation of the training method for a large language model using VOC processing provided in this invention. Figure 3 A flowchart illustrating another implementation of the VOC processing large language model training method provided in this invention; Figure 4 A flowchart illustrating one embodiment of the VOC processing method provided in this invention; Figure 5 This is an example diagram of a user feedback message provided in an embodiment of the present invention; Figure 6 A block diagram illustrating one embodiment of the electronic device provided in this invention; Figure 7 A schematic diagram of a computer-readable medium provided in an embodiment of the present invention; Explanation of reference numerals in the attached figures 101: Processor; 102: Memory 103: I / O interface; 104: Bus. Detailed Implementation

[0020] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described are intended to explain the present invention and should not be construed as limiting the invention.

[0021] The terms "an embodiment," "example," or "trademark" used in this specification refer to a particular feature, structure, or characteristic described in connection with the embodiment itself that may be included in at least one embodiment disclosed in this invention. The phrase "in an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0022] Enterprises' user websites (such as cross-border payment businesses involving multiple foreign banks, service trade businesses involving multiple domestic distribution channels, etc.) need to handle massive amounts of user feedback information from multiple channels such as customer service system inquiries, product homepage proposal submissions, and social media comments. Moreover, these user feedback information have problems such as scattered descriptions, significant differences in regionality and timeliness.

[0023] Large Language Models (LLMs) are artificial intelligence models built on deep learning architectures (with the Transformer at its core), pre-trained on massive amounts of data, and possessing a large parameter scale and strong generalization ability. They can learn general rules and knowledge from data and have cross-task and cross-scenario generalization capabilities. To achieve automated clustering of user issues, statistical analysis of feedback volume, and priority determination, accurately understand user needs and concerns, and provide data support for product optimization, review process iteration, and user experience improvement, it is crucial to leverage the natural language processing capabilities of large language models to efficiently extract core information from massive amounts of user feedback.

[0024] However, current conventional large language models have poor text analysis and counting capabilities, are not sensitive to analysis, statistics, and numbers, and due to the limited size of the context window, they are unable to handle millions of texts. They cannot solve the problem of efficient and automated clustering of millions of user feedback information by standard context window large language models, and it is difficult to establish a strict mapping relationship between large categories and original text. As a result, they cannot accurately count the size of user voices for corresponding issues, and cannot meet the core demand of enterprises to quickly and accurately understand the needs of massive users.

[0025] To address this, the inventors of this invention propose a VOC (Voice of Custom) processing method. First, an initial model that is highly sensitive to analysis, statistics, and numerical data is selected from multiple candidate large language models. This initial model is then trained using a pre-set hybrid loss function, further enhancing its text analysis and statistical capabilities. By constructing initial and intermediate prompt words, the trained model can automatically analyze and process user feedback based on these prompt words and automatically compress the number of output tokens to fit its context window. This enables the model to handle text clustering problems exceeding its context window, thereby achieving efficient and accurate classification of massive amounts of user feedback and statistical analysis of the volume of user voices in each category. This facilitates accurate identification of user needs, leading to optimization of product and process solutions and improved user experience.

[0026] As a first aspect of the present invention, a method for training large language models using VOC processing is provided, such as... Figure 1 As shown, it includes: In step S110, an initial model is selected from multiple candidate large language models; In step S120, a training dataset is determined, which includes multiple user feedback messages labeled with classification tags; In step S130, the initial model is trained using the training dataset until the value of the preset mixture loss function reaches its minimum, thus obtaining a trained VOC processing large language model. The trained VOC processing large language model is used to output a comment analysis report based on the received initial prompts and intermediate prompts. The initial prompts include the address where user feedback information is collected, the rules for processing user feedback information, and the format of the data returned by the VOC processing large language model. The intermediate prompts include the rules for processing user feedback information and the format of the data returned by the VOC processing large language model.

[0027] In this invention, the embodiments do not impose specific limitations on the multiple candidate large language models; any currently available, high-performance, open-source large language model can be selected. The embodiments also do not impose specific limitations on the source of user feedback information; for example, it can come from the homepage of a business website, social media website, or other user websites, as well as from chatbot customer service systems, human customer service systems, suggestion portals, etc. Furthermore, the embodiments do not impose specific limitations on the category tags; for example, multiple categories can be defined according to different dimensions such as product functions and customer opinions, and then category tags can be labeled according to the corresponding categories.

[0028] In this context, it can be understood that the classification label is used to indicate the true category of the corresponding user feedback information, providing a supervision signal for model training. After the model makes category predictions on the training data, it compares the prediction results with the corresponding classification labels. Through this comparison process, it learns the mapping relationship between the features of the training data and the classification labels, thereby gaining the ability to determine the category of user feedback information.

[0029] To further improve training efficiency, during the step of determining the training dataset, invalid original user feedback information can be filtered out. That is, short and invalid information that does not provide effective value can be filtered out. For example, short phrases or interjections such as "human" or "hello" can be filtered out, as can evaluations such as "good" or "too slow" that do not carry specific reasons.

[0030] In the step of determining the training dataset, user feedback information in non-target languages ​​can be translated into user feedback information in the target language so that the model can directly process a single target language, thereby further improving training efficiency.

[0031] The trained VOC processing language model obtained in this embodiment of the invention can be further used for VOC processing. Specifically, initial prompts can be sent to the trained VOC processing language model; wherein the initial prompts include the address for collecting user feedback information, the processing rules for user feedback information, and the format of the data returned by the VOC processing language model; if the number of tokens returned by the VOC processing language model exceeds the context window of the VOC processing language model, intermediate prompts are sent to the VOC processing language model until the number of tokens returned by the VOC processing language model does not exceed the context window of the VOC processing language model; wherein the intermediate prompts include the processing rules for user feedback information and the format of the data returned by the VOC processing language model; and a comment analysis report returned by the VOC processing language model is obtained.

[0032] Understandably, the comment analysis report returned by the VOC processing large language model includes the mapping relationship between the original text, the category corresponding to the original text, and the user volume of the category corresponding to the original text (i.e., the proportion of user feedback information of the corresponding category in the total number of user feedback information, and / or the number of user feedback information of the corresponding category).

[0033] The training method for a large language model for VOC processing provided in this invention, by selecting an initial model from multiple candidate large language models, ensures the basic adaptability of the model, laying the foundation for improving the model's text analysis and statistical capabilities. This method effectively addresses the weakness of conventional large language models in terms of analytical and statistical capabilities. By determining a training dataset, which includes multiple user feedback messages labeled with classification tags, the method closely matches real-world VOC scenarios, providing accurate supervision signals for model training. This facilitates the model in establishing a mapping relationship between the original text and the corresponding categories, supporting the statistical analysis of user feedback volume. The initial model is trained using this training dataset until the preset mixture loss function reaches its minimum value, resulting in a well-trained large language model for VOC processing. The language model balances lightweight design with training accuracy. Furthermore, the trained VOC processing large language model can output comment analysis reports based on received initial and intermediate prompts. The initial prompts include the address where user feedback was collected, the rules for processing user feedback, and the format of the data returned by the VOC processing large language model. The intermediate prompts also include the rules for processing user feedback and the format of the data returned by the VOC processing large language model. This ensures the standardization and uniformity of comment analysis reports while adapting to scenarios where the number of tokens exceeds the model's context window through intermediate prompts. It enables the large language model with a standard context window to effectively process large-scale text, providing data support for improving user experience.

[0034] The inventors of this invention further propose that by combining supervised fine-tuning SFT and reinforcement learning (RL), the model can learn a preset hybrid loss function on the training dataset, thereby further improving the model's text analysis and statistical capabilities. Accordingly, in some embodiments, the initial model is trained using the training dataset until the value of the preset hybrid loss function reaches its minimum, resulting in a trained VOC processing large language model (i.e., the one involved in step S130). Figure 2 As shown, it may include: In step S210, the training dataset is split into supervised fine-tuning SFT samples and reinforcement learning RL samples; In step S220, the initial model is subjected to supervised fine-tuning SFT using the supervised fine-tuning SFT samples to obtain the supervised fine-tuning SFT base model. In step S230, the reinforcement learning RL samples are labeled with the keyword Ground Truth, which includes a list of reference keywords and the total number of reference keywords; In step S240, based on the reinforcement learning RL samples labeled with the keyword Ground Truth and the preset hybrid loss function, the parameters of the supervised fine-tuning SFT base model are iteratively adjusted until the value of the preset hybrid loss function reaches its minimum, thus obtaining the trained VOC processing large language model.

[0035] In this embodiment of the invention, no specific limitation is made on the ratio in which the training dataset is split into supervised fine-tuning SFT samples and reinforcement learning RL samples. For example, the ratio between reinforcement learning RL samples and supervised fine-tuning SFT samples can be 6:4, 7:3, 8:2, etc.

[0036] The inventors of this invention further propose that the initial model can be trained using RLVR (Reinforcement Learning with Verifiable Reward), and the optimization objectives can be set to two: reducing keyword omissions and reducing keyword repetitions, i.e., improving recall. and reduce repetition rate .

[0037] For the reward function part in RLVR training, set This indicates the number of keywords correctly identified by the current model, and is set as follows: This indicates the total number of reference keywords in the Ground Truth keyword set. This indicates the number of repeated keywords in the current model output. This indicates the total number of keywords output by the current model. , Further comprehensive rewards It can be designed as: ,in, Indicates recall rate With repetition rate The balance coefficient between them aims to achieve a comprehensive reward. The bigger the better.

[0038] For the loss function part in RLVR training, in reinforcement learning frameworks such as RLVR or PPO (Proximal Policy Optimization), the loss function is typically: ,in, Represents the reward function, This represents the baseline used to reduce the variance of the policy gradient estimate. This indicates that the model is at the input. Next generation action The probability of outputting a token.

[0039] Therefore, the hybrid loss combining supervised fine-tuning of SFT and reinforcement learning RL can be initially expressed as: ,in, This represents the supervised fine-tuning SFT loss function, which is the standard cross-entropy loss used to ensure the language fluency and basic format of the model. This represents the reinforcement learning (RL) loss function, which is a policy gradient loss based on comprehensive rewards used to optimize specific task metrics. This represents the weight coefficients of the supervised fine-tuning SFT loss function. The weight coefficients represent the weights of the reinforcement learning (RL) loss function. , Used to control the ratio of supervised fine-tuning of the SFT loss function to reinforcement learning (RL) loss function.

[0040] Therefore, the hybrid loss function can be ultimately expressed as: ,in, , .

[0041] Accordingly, in some embodiments, the preset hybrid loss function includes: (1); (2); (3); In formulas (1)-(3), This indicates that the parameters of the current model are The value of the preset hybrid loss function at that time. This represents the weight coefficients of the supervised fine-tuning SFT loss function. The weight coefficients represent the weights of the reinforcement learning (RL) loss function. This indicates the total number of tokens output by the current model. Indicates the current model output of the first... One token, This represents the user feedback information input. Indicates parameters The current model for time, This indicates that the parameters of the current model are The autoregressive generation probability at time, This indicates that the current model is generating the first... Step output Time depends on the current model in the first All historical outputs generated before the step, This represents the baseline used to reduce the variance of the policy gradient estimate. Indicates recall rate, Indicates the repetition rate. This represents the balance coefficient between recall and duplicate rate (DupRate). This indicates the number of keywords correctly identified by the current model. This indicates the total number of reference keywords in the keyword "Ground Truth". This indicates the number of repeated keywords in the current model output. This indicates the total number of keywords output by the current model.

[0042] It is understood that the embodiments of the present invention are for , The value of is not specifically limited. For example... , The values ​​can be 0.4 and 0.6, 0.3 and 0.7, 0.2 and 0.8, etc.

[0043] What is understandable is that This represents the balance coefficient between recall and duplicate rate (DupRate), which is also the penalty coefficient for duplicate rate (DupRate). However, in this embodiment of the invention, for... The value of is not specifically limited and can be determined according to . What is understandable is that This represents the baseline used to reduce the variance of policy gradient estimation, but in this embodiment of the invention... The value of is not specifically limited. For example, it can be the arithmetic mean of the combined rewards of all samples in the training of the current batch_size. Alternatively, a sliding window can be maintained (the window size can be adjusted according to the dataset size, for example, 5 to 10 batches), and the arithmetic mean of the comprehensive reward of all samples within the window can be calculated in real time, which is used as the b value for the current step. The window is dynamically updated with training iterations. Alternatively, a b value can be calculated separately for each category, that is, only the arithmetic mean of the comprehensive reward of samples of that category in the current batch is calculated, which is used as the exclusive b value for training samples of that category.

[0044] By pre-setting the above-mentioned hybrid loss function, the present invention can fine-tune the model based on a specific task (reducing keyword omission and reducing keyword repetition), which can significantly improve the keyword recognition accuracy of the model. For example, after fine-tuning the training of qwen3-30B, the keyword recognition accuracy can be increased from 60% to 90%.

[0045] The inventors of this invention further propose that keyword omission rate and output result repetition rate can be used as core screening criteria to select an initial model from multiple candidate large language models. Correspondingly, in some embodiments, the selection of the initial model from multiple candidate large language models (i.e., the step involved in S110) is as follows: Figure 3 As shown, it may include: In step S310, multiple user feedback messages for filtering are obtained, and the keyword identification reference results and keyword classification reference results corresponding to the multiple user feedback messages for filtering are determined. In step S320, the multiple user feedback information for filtering is input into each of the candidate large language models to obtain the keyword recognition prediction results and keyword classification prediction results output by each of the candidate large language models. In step S330, for any candidate large language model, its comprehensive accuracy is determined based on its output keyword recognition prediction results and keyword classification prediction results, as well as the keyword recognition reference results and keyword classification reference results; In step S340, the model with the highest overall accuracy among multiple candidate large language models is selected as the initial model.

[0046] The keyword identification reference results include multiple keywords used for reference and the number of these keywords; the keyword classification reference results include multiple categories used for reference and the number of these categories; the keyword identification prediction results include multiple keywords predicted by the candidate large language model and the number of these keywords; and the keyword classification reference results include multiple categories predicted by the candidate large language model and the number of these categories.

[0047] It is understood that, for any candidate large language model, the keyword omission rate of the candidate large language model can be calculated based on its output keyword recognition prediction results and keyword recognition reference results, and the repetition rate of the output results of the candidate large language model can be calculated based on its output keyword classification prediction results and keyword classification reference results. In turn, its comprehensive accuracy can be determined based on its keyword omission rate and output result repetition rate.

[0048] In this embodiment of the invention, no specific limitation is made on how to determine the comprehensive accuracy based on the keyword omission rate and the output result repetition rate of the candidate large language model. For example, a weighted statistical algorithm can be used.

[0049] As mentioned above, the trained VOC processing large language model can be further used for VOC processing. Accordingly, as a second aspect of the present invention, a VOC processing method is also provided, such as... Figure 4 As shown, it may include: In step S410, the set initial prompt words are sent to the trained VOC processing large language model; wherein, the initial prompt words include the collection address of user feedback information, the processing rules for user feedback information, and the format of the data returned by the VOC processing large language model, the VOC processing large language model is obtained by selecting the initial model from multiple candidate large language models, and training the initial model using a training dataset including multiple user feedback information labeled with classification tags until the value of the preset mixture loss function reaches its minimum; In step S420, if the number of tokens in the comment analysis report returned by the VOC processing large language model exceeds the context window of the VOC processing large language model, intermediate prompt words are sent to the VOC processing large language model until the number of tokens in the comment analysis report returned by the VOC processing large language model does not exceed the context window of the VOC processing large language model; wherein, the intermediate prompt words include the processing rules for user feedback information and the format of the data returned by the VOC processing large language model; In step S430, the comment analysis report currently returned by the VOC processing large language model is obtained.

[0050] It can be understood that the user feedback information collection address carried in the initial prompt is used by the VOC processing large language model to obtain user feedback information from the user feedback information collection address for processing. The "format of data returned by the VOC processing large language model" carried in the initial prompt and intermediate prompt is used to prompt the VOC processing large language model to return data according to this format. This embodiment of the invention does not make specific limitations on this.

[0051] For example, the following is the format of the data returned by the VOC processing large language model, carried in a certain prompt word: It is understandable that the comment analysis report returned by the VOC processing large language model includes the mapping relationship between the original text, the category corresponding to the original text, and the user volume of the category corresponding to the original text (i.e., the proportion of user feedback information of the corresponding category in the total number of user feedback information, and / or the number of user feedback information of the corresponding category).

[0052] This invention first sends initial prompts to a trained VOC processing large language model, including the collection address of user feedback information, processing rules for user feedback information, and the format of the data returned by the VOC processing large language model. If the number of tokens in the comment analysis report returned by the VOC processing large language model exceeds the context window of the VOC processing large language model, then intermediate prompts are sent to the VOC processing large language model, including processing rules for user feedback information and the format of the data returned by the VOC processing large language model, until the number of tokens in the comment analysis report returned by the VOC processing large language model does not exceed the context window of the VOC processing large language model. This allows the VOC processing large language model to dynamically process the acquired user feedback information in a hierarchical manner, enabling even a standard context window VOC processing large language model to handle user feedback information of any scale. This ensures both the standardization and uniformity of comment analysis reports and adapts to scenarios where the number of tokens exceeds the model's context window through intermediate prompts. It achieves effective processing of large-scale text by a standard context window large language model, which is beneficial for providing data support for improving user experience.

[0053] The inventors of this invention further propose that by setting core text fragment extraction rules in the processing rules, the VOC processing large language model can be guided to extract core text fragments from massive user feedback information, which can initially significantly reduce the total number of input tokens. By setting coarse classification rules to guide the VOC processing large language model to perform coarse classification of core text fragments in batches, the total number of input tokens can be adapted to the model's context window, and the model can have preliminary clustering ability. By setting fine classification rules to guide the VOC processing large language model to perform fine classification of coarse classification samples in batches, the total number of output tokens of the model can also be adapted to the model's context window, and the model can have better clustering ability.

[0054] Accordingly, in some embodiments, the processing rules include core text fragment extraction rules, coarse classification rules, and fine classification rules; The core text fragment extraction rule is used to prompt the VOC processing large language model to extract no more than a first preset number of core text fragments from each user feedback information; The coarse classification rule is used to prompt the VOC processing large language model to divide all the extracted core text fragments into multiple batches according to the first preset sample number length, divide the core text fragments of each batch into a second preset number of temporary categories with the number of words in each temporary category not exceeding the first preset number, and perform semantic summarization on all the temporary categories to obtain a second preset number of coarse categories. The detailed classification rules are used to prompt the VOC processing large language model to divide all core text fragments corresponding to the second preset number of coarse categories into multiple batches according to the second preset number of samples, divide the core text fragments of each batch into a third preset number of detailed categories with the number of words in each detailed category not exceeding the first preset number, and summarize the detailed classification of all batches and return the results to the comment analysis report. The first preset sample number length, the second preset sample number length, the first preset number, the second preset number, and the third preset number are determined according to the VOC processing large language model context window size.

[0055] The following example illustrates the process of VOC processing large language models to execute processing rules. Assume that 10,000 user feedback messages are sampled from the user feedback information collection address, with an average of over 100 characters per message, resulting in 1 million tokens. This far exceeds the context window of a typical large language model. For example, the context window of a typical large language model for two consecutive versions is only 64K and 100,000 respectively. Furthermore, it must be considered that the performance of a large language model deteriorates significantly when the context window exceeds 200,000. Further assuming that the maximum capacity (maxtoken) of the VOC processing large language model's context window is 10,000, the number of tokens input in a single session needs to be controlled within 5K. First, the VOC processing model extracts no more than 10 core text fragments from each of the 10,000 user feedback messages, reducing the total number of input tokens to less than 100,000. Next, the VOC processing model divides the 10,000 core text fragments into 20 batches according to a pre-set sample size of 500. Each batch of core text fragments is then divided into 50 temporary categories, with each temporary category containing no more than 10 words. Semantic summation is performed on all temporary categories to obtain 50 coarse categories. At this point, the total number of output tokens is reduced to less than 50 categories * 20 batches * 10 tokens = 10,000, and the total number of input tokens per batch is also reduced to 500 * 10 tokens. With an en=5000 tokens or less, it is already adapted to the context window of the VOC processing large language model. Furthermore, all core text fragments corresponding to the 50 coarse categories (assuming there are 500 samples) are divided into two batches according to the second preset sample size length batch_size=250. The core text fragments in each batch are divided into 250 fine categories, and the number of words in each fine category does not exceed 10. The fine classification of the two batches is summarized to obtain 500 fine categories. Finally, a comment analysis report with 500*10 tokens=5000 tokens can be output. At this time, the total number of tokens output is also adapted to the context window of the VOC processing large language model, and the VOC processing large language model also has better clustering ability.

[0056] The inventors of this invention further propose that classification correction rules can be set in the prompt words, enabling the model to automatically identify and correct classification problems. Accordingly, in some embodiments, the processing rules further include classification correction rules, which are used to prompt the VOC processing large language model to perform a preset correction action when the current classification result triggers a correction condition. The correction conditions include any of the following or a combination thereof: the current classification result contains a misclassified original sound whose semantics do not match the category; the current classification result contains an unclassified original sound that is not assigned to any category; the current classification result contains a redundant category with a similarity not lower than a preset similarity threshold. The preset correction actions include any of the following or a combination thereof: reclassifying the misclassified original sound to a target category that matches its semantics, reclassifying the unclassified original sound to determine its category, and reclassifying the original sound corresponding to redundant categories to multiple subcategories with semantic similarity lower than a preset similarity threshold.

[0057] It is understandable that in some extreme cases, there may be semantic overlap between categories (i.e., redundant categories with a similarity of no less than a preset similarity threshold). In such cases, splitting the redundant categories into at least two semantically distinct categories can effectively reduce the overlap rate between categories. If a classification result includes the categories "customer service support issues," "slow or delayed service," "communication problems," and "unprofessional or poor service," and these four categories have high semantic similarity, the model can reclassify their corresponding core text segments.

[0058] It is understandable that, in some extreme cases, after merging multiple batches, the same original audio (user feedback information) might belong to multiple categories simultaneously, causing confusion in the model and ultimately leading to the original audio being incorrectly classified into a mismatched category. In such cases, reclassifying the incorrectly classified original audio to a target category that matches its semantics can effectively improve the accuracy of the model's clustering. For example, Figure 5 The example image shows a user feedback message. The original audio was simply categorized as "customer support issue". However, according to the content marked by the black line box in the image, the original audio should belong to the "lack of transparency in payment settings" category. The model can reclassify it to the "lack of transparency in payment settings" category.

[0059] It is understandable that in some extreme cases, after merging multiple batches, the same original sound (user feedback information) may belong to multiple categories at the same time, causing confusion in the model and ultimately the original sound is not classified into any category. In this case, reclassifying the unclassified original sound to determine its category can also effectively improve the accuracy of model clustering.

[0060] The inventors of this invention further propose that dynamic classification adjustment rules can be set in the prompt words, enabling the model to automatically adjust the current sub-category matrix. Accordingly, in some embodiments, the prompt words also include dynamic classification adjustment rules, which include automatic addition of classification rules, automatic elimination of classification rules, and classification splitting rules. The automatic addition of classification rules is used to prompt the VOC processing large language model to statistically analyze the output ratio of each category in all returned comment analysis reports according to a preset time period, and to add any category to the current sub-category matrix if the output ratio of any category exceeds the first preset ratio threshold and the category does not exist in the current sub-category matrix. The automatic elimination classification rule is used to prompt the VOC processing large language model to statistically analyze the output ratio of each category in all returned comment analysis reports according to a preset time period, and to eliminate the category from the current sub-category matrix if the output ratio of any category is lower than the second preset ratio threshold and the category exists in the current sub-category matrix. The classification splitting rule is used to prompt the VOC processing large language model to statistically analyze the output proportion of each category in all returned comment analysis reports according to a preset time period, and to execute the above VOC processing method when the output proportion of any category exceeds a third preset proportion threshold and that category exists in the current sub-category matrix, wherein the third preset proportion threshold is greater than the first preset proportion threshold.

[0061] It can be understood that the current fine category matrix refers to the current set of target fine categories within the model.

[0062] As products iterate, new features emerge, and the original feedback (user feedback information) of certain categories may gradually fade while that of others may become more prominent. This can lead to situations where the existing categories are insufficient to cover the current feedback. In such cases, statistics are compiled according to a preset time period (e.g., monthly, weekly). When the output percentage of a newly emerging category in the review analysis report within the current period exceeds a first preset threshold (e.g., 10%), it can be added to the current sub-category matrix. When the output percentage of an existing category in the review analysis report within the current period is lower than a second preset threshold (e.g., 10%), it can be removed from the current sub-category matrix. When the output percentage of an existing category in the review analysis report within the current period exceeds a third preset threshold (e.g., 30%), this category is too concentrated, and the VOC processing method provided by this invention (i.e., steps S110-S130 above) can be reapplied to break it down and form new categories.

[0063] Furthermore, the inventors of this invention propose that other sub-rules can be set in the processing rules. For example, a sub-rule can be used to prompt the VOC processing large language model to classify some meaningless, invalid, or unclassifiable user feedback information into the "other" category. Another example is a sub-rule used to prompt the VOC processing large language model to keep the number of samples covered by each coarse category relatively uniform. Yet another example is a sub-rule used to prompt the VOC processing large language model to focus on user problems or functional requirements while ignoring the language used by the user and the country where the user is located when analyzing user feedback information.

[0064] The following example illustrates the format of the data returned by the VOC processing large language model.

[0065] The processing results are returned in JSON format, including the original comments and the merged category information; the output format is as follows: {{'comments":[ {{"original_comment":["xxx","uuu"],"merge_comments":"ooo"}}, {{"original_comment":["xxx","bbb"],"merge_comments":"ddd"}}]}}.

[0066] The entire JSON data described above is an object containing an array of `comments`. Each element in `comments` corresponds to a set of "original comments - merged categories". The value of the `original_comments` field is an array representing the original user feedback information being merged ("xxx", "uuu", and "bbb" are placeholders; in practice, actual user feedback information will be filled in). The value of the `merge_comments` field is a string representing the category information (i.e., coarse or fine categories) formed by merging these original user feedback information after clustering ("ooo" and "ddd" are placeholders for category summaries; in practice, actual category results will be filled in). It should be emphasized that the above is only an example illustration of the "format of data returned by the VOC processing large language model" carried in the prompt words in this embodiment of the invention. The invention is not limited to this, and the above only uses original user feedback information of the type of original comments as an example. In reality, the sources and types of original user feedback information are diverse, and the invention is not limited to this.

[0067] As a third aspect of the invention, an electronic device is also provided, such as Figure 6 As shown, it includes: One or more processors 101; Memory 102, having stored one or more computer programs, which, when executed by one or more processors 101, cause the one or more processors 101 to perform any of the following: The first aspect of this invention provides a method for training a large language model using VOC processing; The VOC processing method provided in the second aspect of the present invention.

[0068] The electronic device may also include one or more I / O interfaces 103 connected between the processor 101 and the memory 102, configured to enable information interaction between the processor 101 and the memory 102.

[0069] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read-write interface) is connected between the processor and the memory, enabling information exchange between the processor and the memory, including but not limited to a data bus (Bus).

[0070] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0071] As a fourth aspect of the present invention, such as Figure 7 As shown, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the following: The first aspect of this invention provides a method for training a large language model using VOC processing; The VOC processing method provided in the second aspect of the present invention.

[0072] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. Accordingly, the computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can implement the methods of any of the above embodiments. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0073] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art should understand that the present invention includes, but is not limited to, the contents described in the accompanying drawings and the specific embodiments above. Any modifications that do not depart from the functional and structural principles of the present invention will be included within the scope of the claims.

Claims

1. A method for training a VOC processing large language model, characterized in that, The method comprises the following steps: selecting an initial model from a plurality of candidate large language models; determining a training data set comprising a plurality of user feedback information labeled with classification labels; training the initial model using the training data set until the value of the preset hybrid loss function reaches the minimum to obtain a trained VOC processing large language model; wherein the trained VOC processing large language model is used to output a comment analysis report according to the received initial prompt word and intermediate prompt word, the initial prompt word comprising the collection address of the user feedback information, the processing rule of the user feedback information, and the format of the data returned by the VOC processing large language model, and the intermediate prompt word comprising the processing rule of the user feedback information and the format of the data returned by the VOC processing large language model.

2. The method of claim 1, wherein, The method comprises the following steps: splitting the training data set into supervised fine-tuning SFT samples and reinforcement learning RL samples; supervised fine-tuning SFT of the initial model using the supervised fine-tuning SFT samples to obtain a supervised fine-tuning SFT base model; labeling keywords Ground Truth for the reinforcement learning RL samples, the keywords Ground Truth comprising a reference keyword list and a total number of reference keywords; iteratively adjusting the parameters of the supervised fine-tuning SFT base model according to the reinforcement learning RL samples labeled with keywords Ground Truth and the preset hybrid loss function until the value of the preset hybrid loss function reaches the minimum to obtain a trained VOC processing large language model.

3. The method of claim 2, wherein, The preset hybrid loss function comprises: （1）； （2）； （3）； In formulas (1)-(3), This indicates that the parameters of the current model are The value of the preset mixture loss function at that time. This represents the weight coefficients of the supervised fine-tuning SFT loss function. The weight coefficients represent the weights of the reinforcement learning (RL) loss function. This indicates the total number of tokens output by the current model. Indicates the current model output of the first... One token, This represents the user feedback information input. Indicates parameters The current model for time, This indicates that the parameters of the current model are The autoregressive generation probability at time, This indicates that the current model is generating the first... Step output Time depends on the current model in the first All historical outputs generated before the step, This represents the baseline used to reduce the variance of the policy gradient estimate. Indicates recall rate, Indicates the repetition rate. This represents the balance coefficient between recall and duplicate rate (DupRate). This indicates the number of keywords correctly identified by the current model. This indicates the total number of reference keywords in the keyword "Ground Truth". This indicates the number of repeated keywords in the current model output. This indicates the total number of keywords output by the current model.

4. The method of claim 1, wherein, The method comprises the following steps: obtaining a plurality of screening user feedback information and determining the keyword recognition reference results and keyword classification reference results corresponding to the plurality of screening user feedback information; inputting the plurality of screening user feedback information into each candidate large language model to obtain keyword recognition prediction results and keyword classification prediction results output by each candidate large language model; for any candidate large language model, determining its comprehensive accuracy according to the keyword recognition prediction results and keyword classification prediction results output by it, and the keyword recognition reference results and keyword classification reference results; selecting the model with the highest comprehensive accuracy from the plurality of candidate large language models as the initial model.

5. A VOC treatment method, characterized by, The method comprises the following steps: sending a set initial prompt to the trained VOC processing large language model; wherein the initial prompt comprises a collection address of user feedback information, a processing rule for the user feedback information, and a format of returned data of the VOC processing large language model, the VOC processing large language model is obtained by screening an initial model from a plurality of candidate large language models, and training the initial model by using a training data set comprising a plurality of user feedback information labeled with classification tags until a value of a preset hybrid loss function reaches a minimum value; in a case where a number of tokens in a comment analysis report returned by the VOC processing large language model exceeds a context window of the VOC processing large language model, sending an intermediate prompt to the VOC processing large language model until the number of tokens in the comment analysis report returned by the VOC processing large language model does not exceed the context window of the VOC processing large language model; wherein the intermediate prompt comprises a processing rule for the user feedback information, and the format of the returned data of the VOC processing large language model; obtaining a comment analysis report currently returned by the VOC processing large language model.

6. The method of claim 5, wherein, the processing rule comprises a core text segment extraction rule, a coarse classification rule, and a fine classification rule; the core text segment extraction rule is used to prompt the VOC processing large language model to extract a core text segment of no more than a first preset number of words from each user feedback information; the coarse classification rule is used to prompt the VOC processing large language model to divide all extracted core text segments into a plurality of batches according to a first preset sample number length, divide each batch of core text segments into a second preset number of temporary categories and the number of words in each temporary category does not exceed the first preset number, and perform semantic summarization on all temporary categories to obtain a second preset number of coarse categories; the fine classification rule is used to prompt the VOC processing large language model to divide all core text segments corresponding to the second preset number of coarse categories into a plurality of batches according to a second preset sample number, divide each batch of core text segments into a third preset number of fine categories and the number of words in each fine category does not exceed the first preset number, and summarize the fine classification of all batches to return a comment analysis report; wherein the first preset sample number length, the second preset sample number length, the first preset number, the second preset number, and the third preset number are determined according to the context window size of the VOC processing large language model.

7. The method of claim 5, wherein, the processing rule further comprises a classification correction rule, the classification correction rule is used to prompt the VOC processing large language model to perform a preset correction action in a case where a current classification result triggers a correction condition; the correction condition comprises any or a combination of the following: a classification error original voice with a semantic mismatch in the current classification result, an unclassified original voice not belonging to any category in the current classification result, and a redundant category with a similarity not lower than a preset similarity threshold in the current classification result. The preset deviation correction action includes any or a combination thereof: reclassifying the classified error original sound to a target category matching its semantics, re-executing classification on the unclassified original sound to determine its belonging category, and reclassifying the original sound corresponding to the redundant category to a plurality of subcategories with a semantic similarity lower than a preset similarity threshold.

8. The method of claim 5, wherein, The prompt word further includes a dynamic classification adjustment rule, and the dynamic classification adjustment rule includes an automatic addition classification rule, an automatic elimination classification rule, and a classification splitting rule. The automatic addition classification rule is used to prompt the VOC processing large language model to count the output proportion of each category in all returned comment analysis reports according to a preset time period, and in the case that the output proportion of any category exceeds a first preset proportion threshold and the any category does not exist in the current fine category matrix, the any category is added to the current fine category matrix. The automatic elimination classification rule is used to prompt the VOC processing large language model to count the output proportion of each category in all returned comment analysis reports according to a preset time period, and in the case that the output proportion of any category is lower than a second preset proportion threshold and the any category exists in the current fine category matrix, the category is eliminated from the current fine category matrix. The classification splitting rule is used to prompt the VOC processing large language model to count the output proportion of each category in all returned comment analysis reports according to a preset time period, and in the case that the output proportion of any category exceeds a third preset proportion threshold and the any category exists in the current fine category matrix, the VOC processing method according to claim 5 is executed, wherein the third preset proportion threshold is greater than the first preset proportion threshold.

9. An electronic device, comprising: Comprise: One or more processors; A memory having one or more computer programs stored thereon, when the one or more computer programs are executed by the one or more processors, so that the one or more processors implement any of the following: The training method of the VOC processing large language model according to any one of claims 1 to 4; The VOC processing method according to any one of claims 5 to 8.

10. A computer readable medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement any of the following: The training method of the VOC processing large language model according to any one of claims 1 to 4; The VOC processing method according to any one of claims 5 to 8.

Images