Model optimization method and device, electronic equipment, storage medium and program product

Abstract

The application provides a model optimization method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of data processing. The method comprises the following steps: analyzing any output content output by a large language model to be optimized to obtain a semantic naturalness evaluation value, an expression diversity evaluation value and a generation fluency evaluation value of the output content; determining a reward score of the output content based on the semantic naturalness evaluation value, the expression diversity evaluation value and the generation fluency evaluation value; and iteratively optimizing an output strategy of the large language model to be optimized based on the reward score corresponding to each output content until a preset iteration condition is met, so as to obtain an optimized large language model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a model optimization method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of large language model technology, it has been widely used in content generation, intelligent dialogue, and text creation. However, existing large language models generally suffer from obvious machine-generated traces in their output content during practical applications. Specifically, the generated text exhibits stiff and rigid expressions, monotonous and repetitive sentence structures, a lack of variation in word choice, and redundant and cumbersome logical expressions. These characteristics make the model's output content lack the natural fluency and expressive flexibility of human language, seriously affecting user experience and application effectiveness.

[0003] Therefore, how to effectively improve the naturalness and humanization of the output content of large language models, making it closer to the real expression habits of humans, has become a technical problem that the industry urgently needs to solve. Summary of the Invention

[0004] This invention provides a model optimization method, apparatus, electronic device, storage medium, and program product to solve the problem of how to effectively improve the naturalness and humanization of the output content of large language models in the prior art.

[0005] This invention provides a model optimization method, comprising:

[0006] For any output content of the large language model to be optimized, the output content is analyzed to obtain a semantic naturalness evaluation value, an expressive diversity evaluation value, and a generation fluency evaluation value. The semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained from the discriminator network through a nonlinear mapping function. The expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained from the unique word ratio of N-gram phrases through a segmented mapping function. The generation fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function.

[0007] The reward score for the output content is determined based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value.

[0008] Based on the reward score corresponding to each output content, the output strategy of the large language model to be optimized is iteratively optimized until the preset iteration conditions are met, and the optimized large language model is obtained.

[0009] According to a model optimization method provided by the present invention, the method for determining the semantic naturalness evaluation value includes:

[0010] Construct a discriminator network, which includes: a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator;

[0011] Based on the discriminator network, the output content is analyzed to obtain the sentence repetition score, word stereotype score, and logical redundancy score of the output content.

[0012] Based on the average scores of the sentence repetition score, word stereotypedness score, and logical redundancy score, the machine feature strength score is obtained.

[0013] The machine feature strength score is converted into a semantic naturalness evaluation value through a nonlinear mapping function, which results in a higher semantic naturalness evaluation value as the machine feature strength score decreases.

[0014] According to a model optimization method provided by the present invention, the construction process of the discriminator network includes:

[0015] Collect original human-generated text as positive samples;

[0016] The original human text is rewritten by machine in various ways to generate machine output text with different machine feature strength scores as negative samples. The machine rewriting includes prompt-guided rewriting and cross-language back translation.

[0017] The teacher model was used to label the positive and negative samples from three dimensions: sentence repetition, word stereotypedness, and logical redundancy, resulting in a data sample set labeled with sentence repetition, word stereotypedness, and logical redundancy.

[0018] Based on the sentence repetition annotation data sample set, the word stereotype annotation data sample set, and the logical redundancy annotation data sample set, a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator are trained respectively, and the discriminator network is obtained through ensemble learning.

[0019] According to a model optimization method provided by the present invention, the prompt-guided rewriting includes: rewriting human-original text using at least two prompts with different styles and intensities;

[0020] The cross-language back-translation includes: a process of translating Chinese text into English and then back into Chinese;

[0021] The ratio of positive to negative samples is dynamically adjusted based on the training performance of the discriminator.

[0022] According to a model optimization method provided by the present invention, the method for determining the expression diversity evaluation value includes:

[0023] Calculate the unique word ratio of the N-gram phrases in the output content. The unique word ratio of the N-gram phrases is defined as the ratio of the number of segments composed of N consecutive words that are not repeated to the total number of segments composed of all N consecutive words; where N is a positive integer.

[0024] Text length normalization is performed on the unique word ratios of N-gram phrases with different N values. The unique word ratios of multiple normalized N-gram phrases are then weighted and combined to obtain the comprehensive unique word ratio.

[0025] The comprehensive unique word ratio is converted into an expression diversity assessment value using a segmented mapping function.

[0026] According to a model optimization method provided by the present invention, the step of converting the comprehensive unique word ratio into an expression diversity evaluation value through a segmented mapping function includes:

[0027] When the overall unique word ratio is lower than a preset lower threshold, a negative penalty value is calculated based on the square of the difference between the overall unique word ratio and the preset lower threshold, and used as the expression diversity evaluation value.

[0028] When the overall unique word ratio is between a preset lower threshold and a preset upper threshold, a positive reward value is calculated based on the distance between the overall unique word ratio and a preset target value, and this value is used as the expression diversity evaluation value.

[0029] When the overall unique word ratio is higher than a preset upper threshold, an attenuation suppression value is calculated based on the difference between the overall unique word ratio and the preset upper threshold, and used as the expression diversity evaluation value.

[0030] The preset lower threshold, preset upper threshold, and preset target value are determined by statistical analysis of the diversity distribution of human text in the target domain.

[0031] According to a model optimization method provided by the present invention, the method for determining the generation of fluency evaluation values ​​includes:

[0032] Based on the large language model to be optimized, the predicted perplexity of each word position is determined during the generation of the output content, forming a perplexity sequence of the output content;

[0033] The standard deviation of the perplexity sequence is used as the perplexity fluctuation value, and a fluency evaluation value is determined by a linear penalty function based on the deviation between the perplexity fluctuation value and the target fluctuation value.

[0034] The target fluctuation value is obtained by performing perplexity analysis on a large-scale human text corpus.

[0035] According to a model optimization method provided by the present invention, a reward score for the output content is determined based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value, including:

[0036] The semantic naturalness evaluation value, expressive diversity evaluation value, and generation fluency evaluation value are weighted and summed to determine the reward score of the output content;

[0037] The weight of the semantic naturalness evaluation value is greater than the weight of the expression diversity evaluation value, and the weight of the expression diversity evaluation value is greater than the weight of the generation fluency evaluation value.

[0038] The present invention also provides a model optimization device, comprising the following modules:

[0039] The analysis module is used to analyze any output content of the large language model to be optimized, and obtain a semantic naturalness evaluation value, an expressive diversity evaluation value, and a generation fluency evaluation value for the output content. The semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained from the discriminator network through a nonlinear mapping function. The expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained from the unique word ratio of N-gram phrases through a segmented mapping function. The generation fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function.

[0040] An evaluation module is used to determine the reward score of the output content based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value.

[0041] The optimization module is used to iteratively optimize the output strategy of the large language model to be optimized based on the reward score corresponding to each output content, until the preset iteration conditions are met, and the optimized large language model is obtained.

[0042] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0043] Construct a discriminator network, which includes: a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator;

[0044] Based on the discriminator network, the output content is analyzed to obtain the sentence repetition score, word stereotype score, and logical redundancy score of the output content.

[0045] Based on the average scores of the sentence repetition score, word stereotypedness score, and logical redundancy score, the machine feature strength score is obtained.

[0046] The machine feature strength score is converted into a semantic naturalness evaluation value through a nonlinear mapping function, which results in a higher semantic naturalness evaluation value as the machine feature strength score decreases.

[0047] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0048] Collect original human-generated text as positive samples;

[0049] The original human text is rewritten by machine in various ways to generate machine output text with different machine feature strength scores as negative samples. The machine rewriting includes prompt-guided rewriting and cross-language back translation.

[0050] The teacher model was used to label the positive and negative samples from three dimensions: sentence repetition, word stereotypedness, and logical redundancy, resulting in a data sample set labeled with sentence repetition, word stereotypedness, and logical redundancy.

[0051] Based on the sentence repetition annotation data sample set, the word stereotype annotation data sample set, and the logical redundancy annotation data sample set, a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator are trained respectively, and the discriminator network is obtained through ensemble learning.

[0052] According to the model optimization apparatus provided by the present invention, the prompt-guided rewriting includes: rewriting human-original text using at least two prompts with different styles and intensities;

[0053] The cross-language back-translation includes: a process of translating Chinese text into English and then back into Chinese;

[0054] The ratio of positive to negative samples is dynamically adjusted based on the training performance of the discriminator.

[0055] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0056] Calculate the unique word ratio of the N-gram phrases in the output content. The unique word ratio of the N-gram phrases is defined as the ratio of the number of segments composed of N consecutive words that are not repeated to the total number of segments composed of all N consecutive words; where N is a positive integer.

[0057] Text length normalization is performed on the unique word ratios of N-gram phrases with different N values. The unique word ratios of multiple normalized N-gram phrases are then weighted and combined to obtain the comprehensive unique word ratio.

[0058] The comprehensive unique word ratio is converted into an expression diversity assessment value using a segmented mapping function.

[0059] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0060] When the overall unique word ratio is lower than a preset lower threshold, a negative penalty value is calculated based on the square of the difference between the overall unique word ratio and the preset lower threshold, and used as the expression diversity evaluation value.

[0061] When the overall unique word ratio is between a preset lower threshold and a preset upper threshold, a positive reward value is calculated based on the distance between the overall unique word ratio and a preset target value, and this value is used as the expression diversity evaluation value.

[0062] When the overall unique word ratio is higher than a preset upper threshold, an attenuation suppression value is calculated based on the difference between the overall unique word ratio and the preset upper threshold, and used as the expression diversity evaluation value.

[0063] The preset lower threshold, preset upper threshold, and preset target value are determined by statistical analysis of the diversity distribution of human text in the target domain.

[0064] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0065] Based on the large language model to be optimized, the predicted perplexity of each word position is determined during the generation of the output content, forming a perplexity sequence of the output content;

[0066] The standard deviation of the perplexity sequence is used as the perplexity fluctuation value, and a fluency evaluation value is determined by a linear penalty function based on the deviation between the perplexity fluctuation value and the target fluctuation value.

[0067] The target fluctuation value is obtained by performing perplexity analysis on a large-scale human text corpus.

[0068] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0069] The semantic naturalness evaluation value, expressive diversity evaluation value, and generation fluency evaluation value are weighted and summed to determine the reward score of the output content;

[0070] The weight of the semantic naturalness evaluation value is greater than the weight of the expression diversity evaluation value, and the weight of the expression diversity evaluation value is greater than the weight of the generation fluency evaluation value.

[0071] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the model optimization method as described above.

[0072] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model optimization method as described above.

[0073] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the model optimization method as described above.

[0074] The model optimization method, apparatus, electronic device, storage medium, and program products provided by this invention achieve automated and refined iterative optimization of the output strategy of large language models by constructing a multi-dimensional analysis that includes semantic naturalness, expressive diversity, and generation fluency. Without relying on large-scale manually labeled data, a closed-loop mechanism of quantitative evaluation and reward feedback drives the model to autonomously learn and more closely approximate human expression paradigms, thus overcoming the limitations of traditional supervised fine-tuning, such as high cost, weak generalization ability, and unstable feedback engineering effects. It fundamentally improves the generation behavior of large language models, effectively suppressing problems such as stiffness, rigidity, and repetition in output content, enabling the optimized large language model to stably generate high-quality text with greater naturalness, diversity, and fluency, significantly improving its practical application effect and user experience. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0076] Figure 1 This is a flowchart illustrating the model optimization method provided by the present invention;

[0077] Figure 2 A schematic diagram of the model optimization device provided by the present invention;

[0078] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0080] Figure 1 This is a flowchart illustrating the model optimization method provided by the present invention, as shown below. Figure 1 As shown, the method includes the following:

[0081] Step 110: For any output content of the large language model to be optimized, analyze the output content to obtain the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value of the output content; wherein, the semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained by the discriminator network through a nonlinear mapping function; the expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained by the unique word ratio of N-gram phrases through a segmented mapping function; the generation fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function;

[0082] In this application, the large language model to be optimized refers to a pre-trained language model that needs to improve its output quality. The large language model to be optimized can be various models based on the Transformer architecture, such as the BERT series models, T5 models, etc., or it can be a language model with other architectures.

[0083] The model optimization method described in this application does not involve adjusting or changing the network structure of the large language model to be optimized. The large language model to be optimized maintains the same network architecture (such as the number of layers, hidden layer dimensions, number of attention heads, etc.) before and after optimization. The optimization process in this application uses a reinforcement learning algorithm to fine-tune the model parameters through reward signals, enabling the model to learn a better output strategy, thereby improving the naturalness, diversity, and fluency of the output content. In other words, the technical solution of this application is a parameter optimization method based on reward feedback, rather than a model structure design method. The large language model to be optimized already has basic text generation capabilities before optimization, but its output content exhibits obvious machine-generated characteristics.

[0084] In this application, the output content refers to the text sequence generated by the large language model to be optimized based on input prompts or context. The output content can be a single sentence, paragraph, or complete article, and its length can range from tens of characters to thousands of characters.

[0085] The output can take the form of various text generation tasks, such as dialogue responses, article creation, summary generation, and translation results.

[0086] In this application, a composite reward function can be used to analyze the output content. The composite reward function serves to quantitatively evaluate the quality of the output content from multiple dimensions.

[0087] The composite reward function is a multi-dimensional evaluation function that can comprehensively evaluate the output from different perspectives. The multi-dimensional evaluation function contains three independent yet complementary evaluation dimensions, each corresponding to a sub-function that calculates its respective evaluation value.

[0088] In this embodiment, the composite reward function includes at least three core evaluation dimensions:

[0089] Semantic naturalness assessment aims to measure whether the output content closely resembles natural human expression in terms of style, sentence structure, and vocabulary, and whether there are obvious traces of machine generation. The semantic naturalness assessment value is obtained by transforming the machine feature strength score obtained from the discriminator network through a nonlinear mapping function.

[0090] Specifically, the discriminator network is used to identify machine-generated features in the output content and output a machine feature strength score. The higher the score, the more obvious the machine features are. The machine feature strength score is converted into a semantic naturalness evaluation value through a non-linear mapping function, so that the lower the machine feature strength score, the higher the semantic naturalness evaluation value.

[0091] Expression diversity assessment aims to measure the richness of output content in terms of vocabulary and syntactic structure, avoiding the use of monotonous and repetitive expressions. The expression diversity assessment value is derived from the comprehensive unique word ratio obtained based on the unique word ratio of N-gram phrases, which is then transformed through a segmented mapping function.

[0092] Specifically, the unique word ratio of N-gram phrases is used to measure the richness of vocabulary and phrases in the output content. The unique word ratio of N-gram phrases with different N values ​​is normalized and weighted to obtain the comprehensive unique word ratio. The comprehensive unique word ratio is converted into an expression diversity evaluation value through a segmented mapping function. Excessively low or high diversity is penalized, while moderate diversity is rewarded.

[0093] The generation fluency assessment aims to measure the stability of the output content during the generation process, avoiding logical jumps, inconsistencies, or awkward expressions. The generation fluency assessment value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function.

[0094] Specifically, the perplexity sequence reflects the prediction uncertainty of the model when generating each word, and the perplexity fluctuation value is the standard deviation of the sequence, which is used to measure the stability of the generation process. The deviation between the perplexity fluctuation value and the target fluctuation value is converted into a generation fluency evaluation value through a linear penalty function. The smaller the deviation, the higher the evaluation value.

[0095] This composite reward function provides a quantified evaluation value for any piece of text output by the model, based on the three dimensions mentioned above: semantic naturalness evaluation value, expressive diversity evaluation value, and generative fluency evaluation value. These evaluation values ​​can be numerical, scores, or levels. Then, based on the semantic naturalness evaluation value, expressive diversity evaluation value, and generative fluency evaluation value, the reward score for the output content is determined.

[0096] More specifically, the composite reward function is as follows:

[0097] ;

[0098] in, It is the semantic naturalness evaluation value; It is an assessment value for the diversity of expression; This generates a smoothness evaluation value; w1, w2, and w3 are the corresponding weight coefficients.

[0099] Step 120: Determine the reward score for the output content based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value;

[0100] In this application, after obtaining the evaluation values ​​from the three dimensions, they need to be combined to form a unified reward score. The reward score is a scalar value used to quantify the overall quality of the output content. This score can be positive or negative, with positive values ​​representing rewards and negative values ​​representing penalties.

[0101] The methods for determining reward scores include, but are not limited to: linear combination, nonlinear combination, dynamic weighting, etc.

[0102] In a more specific embodiment, the three evaluation values ​​can be linearly weighted and summed. In practical applications, the combination of each evaluation value can be adjusted according to different application scenarios and optimization objectives.

[0103] Step 130: Based on the reward score corresponding to each output content, iteratively optimize the output strategy of the large language model to be optimized until the preset iteration conditions are met, and obtain the optimized large language model.

[0104] In this application, the iterative optimization process can be implemented using a reinforcement learning algorithm, such as the GRPO (Group Relative Policy Optimization) algorithm. In each iteration, the model generates a batch of output content, calculates the corresponding reward score, and then updates the model parameters based on the reward signal.

[0105] Preset iteration conditions can include reaching the maximum number of iterations, the reward score converging to a stable value, or the model performance metrics reaching a preset threshold. Iteration stops when any one of these conditions is met, and the optimized large language model is output.

[0106] Specifically, within the reinforcement learning framework, the large language model to be optimized is treated as a policy network, and its output policy is determined by the model parameters; the reward score serves as an environmental feedback signal, used to evaluate the quality of the current output content.

[0107] In each iteration, the policy gradient is calculated based on the reward score. The policy gradient indicates the direction of parameter adjustment, which increases the generation probability of high reward outputs and decreases the generation probability of low reward outputs.

[0108] The model parameters are updated using gradient ascent. After multiple iterations, the model gradually learns an output strategy that can stably generate high-reward content, that is, while maintaining semantic accuracy, the output content has higher naturalness, diversity and fluency.

[0109] In this application, through an iterative process and feedback from reward scores, the model's output strategy can be adjusted to tend to generate content that yields higher rewards.

[0110] In this application, the optimized large language model refers to a language model whose output quality is significantly improved after optimization by this method. The content generated by this model has higher naturalness and human-like characteristics while maintaining semantic accuracy.

[0111] This application constructs a multi-dimensional composite reward function that incorporates semantic naturalness, expressive diversity, and generative fluency, thereby achieving automated and refined iterative optimization of the output strategy of a large language model. Without relying on large-scale manually labeled data, a closed-loop mechanism of quantitative evaluation and reward feedback drives the model to autonomously learn expression paradigms closer to humans, overcoming the limitations of traditional supervised fine-tuning, such as high cost, weak generalization ability, and unstable feedback engineering effects. It fundamentally improves the generative behavior of large language models, effectively suppressing problems such as stiffness, rigidity, and repetition in output content. The optimized large language model can stably generate high-quality text with greater naturalness, diversity, and fluency, significantly improving its practical application effects and user experience.

[0112] Optionally, the method for determining the semantic naturalness evaluation value includes:

[0113] Construct a discriminator network, which includes: a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator;

[0114] Based on the discriminator network, the output content is analyzed to obtain the sentence repetition score, word stereotype score, and logical redundancy score of the output content.

[0115] Based on the average scores of the sentence repetition score, word stereotypedness score, and logical redundancy score, the machine feature strength score is obtained.

[0116] The machine feature strength score is converted into a semantic naturalness evaluation value through a nonlinear mapping function, which results in a higher semantic naturalness evaluation value as the machine feature strength score decreases.

[0117] In this application, the discriminator network is a neural network system used to identify and evaluate machine features in text.

[0118] The discriminator network consists of three independent discriminators, forming a multi-dimensional evaluation system. Each discriminator is a binary or multi-class neural network model, which can use pre-trained language models such as BERT or RoBERTa as the basic architecture, or use lightweight models such as BERT-tiny to improve computational efficiency.

[0119] In a preferred embodiment, the BERT-tiny model is used, whose network structure includes 4 Transformer encoder layers, each with a hidden layer dimension of 128, and a multi-head attention mechanism with 2 attention heads.

[0120] The discriminator network comprises three sub-discriminators: a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator. The three sub-discriminators share the aforementioned basic network architecture and each has an independent classification output layer.

[0121] Sentence repetition detectors are used to identify problems such as repetitive sentence structures and repetitive use of specific sentence patterns in text. For example, frequent use of templated structures such as "firstly...secondly...again...".

[0122] The word stereotype discriminator is used to identify problems in text, such as word poverty, overuse of certain high-frequency but meaningless words, or use of "formalistic" words that are not appropriate for the context. For example, the text may use stereotyped conjunctions such as "it is worth noting" and "in conclusion" throughout.

[0123] Logical redundancy discriminators are used to identify problems such as information repetition, tautology, and lack of effective information increment in text. For example, repeatedly saying the same thing in different ways leads to verbose content.

[0124] In this application, the output to be evaluated is input into three discriminators, and each discriminator outputs a score. The score range can be set from 1 to 9, where 1 point indicates that the machine feature in that dimension is the weakest (closest to human expression), and 9 points indicates that the machine feature is the strongest (typical machine-generated feature).

[0125] Discriminant analysis can be performed in parallel, with three discriminators processing the same output simultaneously, improving processing efficiency. The score can be calculated using either the softmax probability value of the last layer of the discriminator or a calibrated confidence score.

[0126] The machine feature strength score is a comprehensive reflection of the scores across three dimensions. The simplest method is to calculate the arithmetic mean, i.e., Machine Feature Strength Score = (Sentence Repetition Score + Word Stereotype Score + Logical Redundancy Score) / 3; a weighted average can also be used, assigning different weights to different dimensions based on their importance, with the score ranging from 1 to 9.

[0127] The nonlinear mapping function is used to convert machine feature strength scores into reward signals, specifically, The semantic naturalness evaluation value is:

[0128] ;

[0129] Where S is the discriminator score; when the machine feature strength score is low... A positive reward is given when the machine feature strength score is high. This is a negative reward.

[0130] The nonlinear mapping function ensures that when the machine feature strength score is low (close to 1), the semantic naturalness evaluation value is positive (reward); when the machine feature strength score is high (close to 9), the semantic naturalness evaluation value is negative (penalty). The nonlinear power function form can amplify the differences, making the optimization process more sensitive.

[0131] In this application, by constructing a network consisting of three specialized discriminators, the abstract concept of semantic naturalness is decomposed into three quantifiable and concrete dimensions: sentence structure, word choice, and logic. This makes the evaluation of machine feature strength more accurate and targeted. This refined evaluation provides a clearer direction for model optimization.

[0132] Optionally, the construction process of the discriminator network includes:

[0133] Collect original human-generated text as positive samples;

[0134] The original human text is rewritten by machine in various ways to generate machine output text with different machine feature strength scores as negative samples. The machine rewriting includes prompt-guided rewriting and cross-language back translation.

[0135] The teacher model was used to label the positive and negative samples from three dimensions: sentence repetition, word stereotypedness, and logical redundancy, resulting in a data sample set labeled with sentence repetition, word stereotypedness, and logical redundancy.

[0136] Based on the sentence repetition annotation data sample set, the word stereotype annotation data sample set, and the logical redundancy annotation data sample set, a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator are trained respectively, and the discriminator network is obtained through ensemble learning.

[0137] In this application, "human-created text" refers to authentic text content created by human authors, without any trace of machine generation or rewriting. Sources include, but are not limited to: news websites, blog platforms, social media, academic papers, and literary works.

[0138] Machine rewriting refers to the use of automated methods to rewrite original human text, giving it varying degrees of machine characteristics.

[0139] Prompt-guided rewriting uses different prompts to guide a large language model in rewriting the original text. Cross-language back-translation is the process of first translating the text into another language and then translating it back into the original language.

[0140] In this application, multiple corresponding negative samples with varying degrees of machine output features can be generated for each positive sample through machine rewriting and prompting.

[0141] More specifically, since manual three-dimensional annotation of massive samples is extremely costly and consistency is difficult to guarantee, this application uses a more capable and larger-scale language model as the teacher model.

[0142] By designing specific annotation instructions (Prompt), the teacher model is guided to score each text in the dataset on three dimensions (e.g., 1-9 points) of sentence repetition, word stereotypedness, and logical redundancy.

[0143] In this way, a large-scale training data with fine-grained labels was obtained at a relatively low cost through the teacher model. This data was then organized into three sets: a data set labeled with sentence repetition, a data set labeled with word stereotypes, and a data set labeled with logical redundancy.

[0144] The format of the labeled data sample set is: (text content, dimension score). Each dimension forms an independent training set, which is used to train the corresponding discriminator.

[0145] Then, three independent discriminator models are trained on the three labeled datasets mentioned above. For example, a sentence repetition labeled dataset is used to train a sentence repetition discriminator, enabling it to predict sentence repetition scores based on the text. To ensure computational efficiency, the discriminators themselves can use lightweight model architectures. After training, the three discriminators are combined through ensemble learning. The simplest way is to average or weighted average their outputs to form the final discriminator network.

[0146] In this application, by combining multiple machine rewriting techniques and automatic annotation by teacher models, the bottleneck of traditional methods relying on manual annotation is overcome, enabling the rapid construction of large-scale, high-quality, and multi-dimensional training datasets, thereby training a high-performance semantic naturalness discriminator network.

[0147] Optionally, the cue-guided rewriting includes: rewriting human-original text using at least two cue words of different styles and intensities;

[0148] The cross-language back-translation includes: a process of translating Chinese text into English and then back into Chinese;

[0149] The ratio of positive to negative samples is dynamically adjusted based on the training performance of the discriminator.

[0150] In this application, in order to generate diverse negative samples, a single cue word cannot be used. This scheme uses at least two cue words with different intensities and orientations.

[0151] In an alternative embodiment, a very detailed set of cue words with strict rules is designed, requiring the model to significantly restructure sentences and improve academic rigor while maintaining core information. Such cue words can easily guide the model to produce logically clear but somewhat rigid and impersonal text.

[0152] Example: Please rewrite the academic text below in a professional-level manner, while meeting the following requirements: information accuracy, retention of all factual statements, key data and proper nouns;

[0153] Reconstructing expression involves restructuring sentence patterns and adjusting wording.

[0154] Strengthen academic rigor, improve the precision of expression, and eliminate colloquialisms;

[0155] The logic continues, maintaining the original paragraph structure and argument logic.

[0156] In another alternative embodiment, a relatively loose cue word is designed, requiring the model to rewrite in a colloquial, slightly casual tone. This cue word may guide the model to produce fluent but not rigorous text, or text with a certain fixed colloquial pattern.

[0157] Example: Please rewrite the text below, requiring that: all facts, data, and technical terms be retained; rewrite it in a conversational and slightly casual tone, like a real human being; and maintain the original paragraph divisions and logical order.

[0158] Different levels of prompt words will produce different degrees of machine learning characteristics. Professional-level rewriting usually produces strong machine learning characteristics, while colloquial rewriting produces moderate machine learning characteristics.

[0159] In this application, cross-language back-translation includes: a process of translating Chinese text into English and then back into Chinese.

[0160] The specific steps of cross-language back-translation are: using a translation model to translate the original Chinese text into English; and then translating the English translation back into Chinese. The resulting back-translated text usually loses the natural expression of the original text and produces obvious translational awkwardness.

[0161] Back-translation results in semantic preservation but altered expression, particularly culturally specific expressions such as idioms and proverbs, which become stiff. This is perfect for generating negative samples with machine-generated characteristics.

[0162] In a more specific embodiment, the initial ratio of positive to negative samples can be set to 1:1. During training, the ratio is dynamically adjusted by monitoring performance metrics on the validation set (such as accuracy, F1 score, etc.).

[0163] If the discriminator is weak in recognizing a certain type of sample, the proportion of that type of sample should be increased. Typical adjustment strategies include: increasing slightly rewritten samples when the discriminator performs poorly in recognizing mild machine features; and increasing the proportion of positive samples to balance the data distribution when overfitting occurs. By using various rewriting methods and dynamic proportion adjustments, high-quality and diverse training datasets can be constructed, improving the discriminator's generalization ability.

[0164] In this application, specific and diverse negative sample generation techniques greatly enrich the diversity of training data. Simultaneously, the introduction of a dynamic data ratio adjustment mechanism makes the discriminator training process more intelligent and efficient, enabling it to specifically compensate for its own shortcomings and ultimately achieve more robust and accurate discrimination capabilities.

[0165] Optionally, the method for determining the expression diversity assessment value includes:

[0166] Calculate the unique word ratio of the N-gram phrases in the output content. The unique word ratio of the N-gram phrases is defined as the ratio of the number of segments composed of N consecutive words that are not repeated to the total number of segments composed of all N consecutive words; where N is a positive integer.

[0167] Text length normalization is performed on the unique word ratios of N-gram phrases with different N values. The unique word ratios of multiple normalized N-gram phrases are then weighted and combined to obtain the comprehensive unique word ratio.

[0168] The comprehensive unique word ratio is converted into an expression diversity assessment value using a segmented mapping function.

[0169] In this application, N-grams (N-gram) A word (or phrase) is a fundamental concept in natural language processing, referring to a sequence of N consecutive lexical units in text. For example, when N=1, it is a single word; when N=2, it is a phrase consisting of two adjacent words. bigram ); and so on, the ratio of unique words in N-gram phrases Distinct-N The calculation formula is:

[0170] ;

[0171] Among them, molecules This indicates that the generated text contains no duplicates. N-gram Quantity, denominator Indicates all of the generated text N-gram The total number of segments.

[0172] The ratio of unique words in N-gram phrases reflects the diversity of textual expression; a higher ratio indicates less repetition and richer expression.

[0173] For example: Text: "I like to eat apples, I like programming";

[0174] When N=1, there are nine 1-grams: ["I", "like", "eat", "apple", ",", "I", "like", "program", "program"]; and seven unique 1-grams: ["I", "like", "eat", "apple", ",", "program", "program"]. Distinct-1 = 7 / 9 = 0.778.

[0175] When N=2, there are eight 2-grams: ["I like", "I like to eat", "Eat apples", "Apples,", ", I", "I like", "I like to code", "Programming"]; there are seven non-repeating 2-grams: ["I like", "I like to eat", "Eat apples", "Apples,", ", I", "I like to code", "Programming"]; Distinct-2 = 7 / 8 = 0.875;

[0176] The AI-like quality generated by the model is specifically reflected in the ratio of unique N-gram words as follows:

[0177] Repetitive vocabulary and sentence structure: Frequent use of logical connectors such as "firstly," "secondly," and "however" creates stereotyped structures, which is reflected in a low Distinct-1 score.

[0178] Lack of emotion and detail: Rigid phrase combination patterns, such as fixed collocations like "based on the above analysis", are reflected in low Distinct-2 scores;

[0179] More specifically, text length normalization is a necessary preprocessing step, as short texts tend to have a higher unique word ratio due to data sparsity. The normalization formula is:

[0180] ;

[0181] Here, length is the total number of words in the text. The logarithmic function can reasonably adjust the influence of the length factor.

[0182] For example, “I like to eat apples, I like programming”, Distinct-2norm=0.875 / log2(12)=0.244.

[0183] The final N-gram unique word ratio used in this application is a weighted average of Distinct-1 and Distinct-2, with Distinct-ave = 0.7. Distinct-1+0.3 Distinct-2 has a value range of (0,1).

[0184] In this application, after determining the N-gram unique word ratio, the comprehensive unique word ratio is further mapped using a segmented mapping function. The segmented mapping function employs different mapping strategies based on different ranges of the comprehensive unique word ratio to achieve a refined reward and punishment mechanism. This function maps ratio values ​​within the range [0,1] to reward or punishment signals.

[0185] In this application embodiment, a diversity quantification method based on N-gram statistics and length normalization is proposed, which not only considers the diversity at the word level but also takes into account the diversity at the phrase level. A comprehensive diversity index is formed by weighted combination, which lays the foundation for subsequent refined reward mapping.

[0186] Optionally, converting the comprehensive unique word ratio into an expression diversity assessment value through a segmented mapping function includes:

[0187] When the overall unique word ratio is lower than a preset lower threshold, a negative penalty value is calculated based on the square of the difference between the overall unique word ratio and the preset lower threshold, and used as the expression diversity evaluation value.

[0188] When the overall unique word ratio is between a preset lower threshold and a preset upper threshold, a positive reward value is calculated based on the distance between the overall unique word ratio and a preset target value, and this value is used as the expression diversity evaluation value.

[0189] When the overall unique word ratio is higher than a preset upper threshold, an attenuation suppression value is calculated based on the difference between the overall unique word ratio and the preset upper threshold, and used as the expression diversity evaluation value.

[0190] The preset lower threshold, preset upper threshold, and preset target value are determined by statistical analysis of the diversity distribution of human text in the target domain.

[0191] In the embodiments of this application, the piecewise mapping function is further described in detail.

[0192] First, when the overall unique word ratio is lower than a preset lower threshold, a negative penalty value is calculated based on the square of the difference between the overall unique word ratio and the preset lower threshold, and used as the expression diversity evaluation value.

[0193] This range is the low diversity penalty zone. The preset lower bound threshold is a reasonable lower limit of diversity derived from statistical analysis of a large amount of original human text, for example, it could be 0.4.

[0194] When the overall unique word ratio of the model's output text falls below this value, it indicates that the expression is very monotonous and repetitive. In this case, the system will give a negative reward, i.e., a penalty.

[0195] The severity of the penalty is related to the degree of deviation from the lower threshold, and the square of the difference is used, meaning that the further the deviation, the more drastically the penalty increases quadratically. For example, the formula for calculating a negative penalty is: R distinct =-3(d-0.4) 2 。

[0196] A preset upper threshold, such as 0.7, is a reasonable upper limit for the diversity of human texts. Texts falling within this range are considered to have healthy and ideal diversity.

[0197] The reward value is designed so that it peaks when the overall unique word ratio approaches a preset target value. As the ratio approaches the two boundaries of the interval, the reward value gradually decreases. This can be achieved using a normal distribution function (Gaussian function) or a similar bell-shaped curve function; for example, the positive reward value can adopt a normal distribution form. R distinct = e^(-30(d - 0.55) 2 The value is -0.32; it reaches a maximum of 0.32 at d=0.55 and decreases towards both sides. This design encourages the model to generate text with moderate diversity, avoiding excessive monotony or divergence.

[0198] When the overall unique word ratio is higher than the preset upper threshold, the attenuation suppression value is calculated based on the difference between the overall unique word ratio and the preset upper threshold, and used as the expression diversity assessment value.

[0199] When d > 0.7, the text may exhibit excessive diversity, such as semantic incoherence and confusing expression. The attenuation suppression value is calculated as follows: R distinct = -0.6 + 1.2 / (1 + e^((d-0.7) / 0.1)); This piecewise mapping function slowly decreases when d just exceeds 0.7, and rapidly approaches the lower limit of -0.6 as d increases, thus achieving a soft constraint.

[0200] More specifically:

[0201] ;

[0202] in, R distinct To express the diversity assessment value, d is... N-gram Unique word ratio Distinct-ave According to statistics, the unique word ratio in human text is between 0.4 and 0.7. Therefore, this case sets the reward threshold at 0.4 and 0.7. In the low diversity penalty zone (d<0.4), the penalty increases twice when d deviates from 0.4. In the target reward zone (0.4≤d≤0.7), the reward follows a normal distribution with the midpoint of 0.55 as the peak. In the high diversity penalty zone (d>0.7), the reward decays rapidly after exceeding the threshold.

[0203] Optionally, the method for determining the fluency evaluation value includes:

[0204] Based on the large language model to be optimized, the predicted perplexity of each word position is determined during the generation of the output content, forming a perplexity sequence of the output content;

[0205] The standard deviation of the perplexity sequence is used as the perplexity fluctuation value, and a fluency evaluation value is determined by a linear penalty function based on the deviation between the perplexity fluctuation value and the target fluctuation value.

[0206] The target fluctuation value is obtained by performing perplexity analysis on a large-scale human text corpus.

[0207] In this application, prediction perplexity (PPL) is a classic performance metric in the field of language modeling, used to measure the uncertainty of a model in predicting the next lexical unit.

[0208] When generating a sentence, the model predicts the probability distribution of the next word at each position. If the model predicts a high probability for the actual generated next word, the perplexity at that position is low, and vice versa.

[0209] The perplexity can be calculated in real time during model inference or post-processed after generation. Real-time calculation can reduce additional computational overhead.

[0210] For a sentence consisting of N words, The perplexity is calculated as follows:

[0211] ;

[0212] in, It is a language model based on the given sequence of words. Under the condition of predicting the current word The probability, It is the number of words in the sentence.

[0213] A perplexity sequence is a sequence of perplexity values ​​arranged in the order in which lexical units are generated. This sequence reflects the degree of perplexity or uncertainty of the model when generating each lexical unit.

[0214] The perplexity fluctuation value is the degree of change in perplexity when the model outputs word by word, and its expression is:

[0215] ;

[0216] in, Let N+1 be the perplexity. μ Let be the average perplexity over the first N steps.

[0217] More specifically, the perplexity fluctuation value is the aforementioned standard deviation. Excessively high volatility values ​​indicate that the model generation is unstable and may exhibit sudden semantic jumps; excessively low volatility values ​​may indicate that the model has fallen into a mechanical repetition pattern.

[0218] In this application, the target fluctuation value is also derived through perplexity analysis of a large-scale, high-quality human text corpus. For example, statistics show that the perplexity fluctuation value of human text is typically around 0.15. Therefore, 0.15 can be set as the target fluctuation value.

[0219] More specifically, the linear penalty function transforms the deviation between the perplexity fluctuation value and the target fluctuation value into a reward signal. The reward is highest when the actual fluctuation value is close to the target fluctuation value; the greater the deviation, the lower the reward. Linear penalty means that the reward value decreases linearly with the absolute value of the deviation.

[0220] For example ;

[0221] in, The perplexity fluctuation value is the perplexity fluctuation value of machine output. According to statistics, the perplexity fluctuation value of human text is 0.15. The target perplexity fluctuation value is 0.15. When it is far from 0.15, it is penalized, and when it is close to 0.15, it is rewarded.

[0222] In this application, fluency is evaluated from the perspective of dynamic stability of the generation process by analyzing the volatility of the perplexity sequence. This enables more effective identification of text that appears fluent but is actually awkward or inconsistent in style, guiding the model to generate content with a more natural and coherent rhythm.

[0223] Optionally, based on the semantic naturalness assessment value, expressive diversity assessment value, and generation fluency assessment value, a reward score for the output content is determined, including:

[0224] The semantic naturalness evaluation value, expressive diversity evaluation value, and generation fluency evaluation value are weighted and summed to determine the reward score of the output content;

[0225] The weight of the semantic naturalness evaluation value is greater than the weight of the expression diversity evaluation value, and the weight of the expression diversity evaluation value is greater than the weight of the generation fluency evaluation value.

[0226] In this application, weighted summation is the most direct and effective comprehensive method, and the formula for calculating the reward score is as follows: ;

[0227] in, It is the semantic naturalness evaluation value; It is an assessment value for the diversity of expression; This generates a smoothness evaluation value; w1, w2, and w3 are the corresponding weight coefficients.

[0228] In this application, the weight of the semantic naturalness evaluation value is greater than the weight of the expression diversity evaluation value, and the weight of the expression diversity evaluation value is greater than the weight of the generation fluency evaluation value.

[0229] The specific weight settings can be: w1=0.5 (semantic naturalness weight); w2=0.3 (expressive diversity weight); w3=0.2 (generation fluency weight).

[0230] In this application, semantic naturalness is the most important (weight 0.5) because reducing machine features and improving naturalness are the core objectives of this method; expressive diversity is the second most important (weight 0.3) because avoiding repetition and monotony is an important aspect of improving text quality; and generation fluency is a supplement (weight 0.2) to ensure the basic fluency and readability of the output.

[0231] In this application, by setting a clear weight hierarchy, the model can distinguish between primary and secondary issues during the optimization process, prioritizing the resolution of the most critical output mechanization problem, followed by improving the richness of expression, while also considering the stability of the generation process. This weighted summation method ensures that the final reward signal is both comprehensive and focused, greatly improving the accuracy and efficiency of the optimization direction.

[0232] Figure 2 This is a schematic diagram of the model optimization device provided by the present invention, as shown below. Figure 2 As shown, it includes:

[0233] The analysis module 210 is used to analyze any output content of the large language model to be optimized, and obtain the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value of the output content; wherein, the semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained by the discriminator network through a nonlinear mapping function; the expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained by the unique word ratio of N-gram phrases through a segmented mapping function; and the generation fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function.

[0234] The evaluation module 220 is used to determine the reward score of the output content based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value;

[0235] The optimization module 230 is used to iteratively optimize the output strategy of the large language model to be optimized based on the reward score corresponding to each output content until the preset iteration conditions are met, so as to obtain the optimized large language model.

[0236] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0237] Construct a discriminator network, which includes: a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator;

[0238] Based on the discriminator network, the output content is analyzed to obtain the sentence repetition score, word stereotype score, and logical redundancy score of the output content.

[0239] Based on the average scores of the sentence repetition score, word stereotypedness score, and logical redundancy score, the machine feature strength score is obtained.

[0240] The machine feature strength score is converted into a semantic naturalness evaluation value through a nonlinear mapping function, which results in a higher semantic naturalness evaluation value as the machine feature strength score decreases.

[0241] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0242] Collect original human-generated text as positive samples;

[0243] The original human text is rewritten by machine in various ways to generate machine output text with different machine feature strength scores as negative samples. The machine rewriting includes prompt-guided rewriting and cross-language back translation.

[0244] The teacher model was used to label the positive and negative samples from three dimensions: sentence repetition, word stereotypedness, and logical redundancy, resulting in a data sample set labeled with sentence repetition, word stereotypedness, and logical redundancy.

[0245] Based on the sentence repetition annotation data sample set, the word stereotype annotation data sample set, and the logical redundancy annotation data sample set, a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator are trained respectively, and the discriminator network is obtained through ensemble learning.

[0246] According to the model optimization apparatus provided by the present invention, the prompt-guided rewriting includes: rewriting human-original text using at least two prompts with different styles and intensities;

[0247] The cross-language back-translation includes: a process of translating Chinese text into English and then back into Chinese;

[0248] The ratio of positive to negative samples is dynamically adjusted based on the training performance of the discriminator.

[0249] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0250] Calculate the unique word ratio of the N-gram phrases in the output content. The unique word ratio of the N-gram phrases is defined as the ratio of the number of segments composed of N consecutive words that are not repeated to the total number of segments composed of all N consecutive words; where N is a positive integer.

[0251] Text length normalization is performed on the unique word ratios of N-gram phrases with different N values. The unique word ratios of multiple normalized N-gram phrases are then weighted and combined to obtain the comprehensive unique word ratio.

[0252] The comprehensive unique word ratio is converted into an expression diversity assessment value using a segmented mapping function.

[0253] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0254] When the overall unique word ratio is lower than a preset lower threshold, a negative penalty value is calculated based on the square of the difference between the overall unique word ratio and the preset lower threshold, and used as the expression diversity evaluation value.

[0255] When the overall unique word ratio is between a preset lower threshold and a preset upper threshold, a positive reward value is calculated based on the distance between the overall unique word ratio and a preset target value, and this value is used as the expression diversity evaluation value.

[0256] When the overall unique word ratio is higher than a preset upper threshold, an attenuation suppression value is calculated based on the difference between the overall unique word ratio and the preset upper threshold, and used as the expression diversity evaluation value.

[0257] The preset lower threshold, preset upper threshold, and preset target value are determined by statistical analysis of the diversity distribution of human text in the target domain.

[0258] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0259] Based on the large language model to be optimized, the predicted perplexity of each word position is determined during the generation of the output content, forming a perplexity sequence of the output content;

[0260] The standard deviation of the perplexity sequence is used as the perplexity fluctuation value, and a fluency evaluation value is determined by a linear penalty function based on the deviation between the perplexity fluctuation value and the target fluctuation value.

[0261] The target fluctuation value is obtained by performing perplexity analysis on a large-scale human text corpus.

[0262] According to the model optimization apparatus provided by the present invention, the apparatus is further used for:

[0263] The semantic naturalness evaluation value, expressive diversity evaluation value, and generation fluency evaluation value are weighted and summed to determine the reward score of the output content;

[0264] The weight of the semantic naturalness evaluation value is greater than the weight of the expression diversity evaluation value, and the weight of the expression diversity evaluation value is greater than the weight of the generation fluency evaluation value.

[0265] This application constructs a multi-dimensional composite reward function that incorporates semantic naturalness, expressive diversity, and generative fluency, thereby achieving automated and refined iterative optimization of the output strategy of a large language model. Without relying on large-scale manually labeled data, a closed-loop mechanism of quantitative evaluation and reward feedback drives the model to autonomously learn expression paradigms closer to humans, overcoming the limitations of traditional supervised fine-tuning, such as high cost, weak generalization ability, and unstable feedback engineering effects. It fundamentally improves the generative behavior of large language models, effectively suppressing problems such as stiffness, rigidity, and repetition in output content. The optimized large language model can stably generate high-quality text with greater naturalness, diversity, and fluency, significantly improving its practical application effects and user experience.

[0266] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a model optimization method. This method includes: for any output content of the large language model to be optimized, analyzing the output content to obtain a semantic naturalness evaluation value, an expressive diversity evaluation value, and a generative fluency evaluation value; wherein the semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained from the discriminator network through a nonlinear mapping function; the expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained from the unique word ratio of N-gram phrases through a segmented mapping function; and the generative fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function.

[0267] The reward score for the output content is determined based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value.

[0268] Based on the reward score corresponding to each output content, the output strategy of the large language model to be optimized is iteratively optimized until the preset iteration conditions are met, and the optimized large language model is obtained.

[0269] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0270] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the model optimization method provided by the above methods. The method includes: for any output content of the output of the large language model to be optimized, analyzing the output content to obtain a semantic naturalness evaluation value, an expressive diversity evaluation value, and a generation fluency evaluation value of the output content; wherein, the semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained by the discriminator network through a nonlinear mapping function; the expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained by the unique word ratio of N-gram phrases through a segmented mapping function; and the generation fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function.

[0271] The reward score for the output content is determined based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value.

[0272] Based on the reward score corresponding to each output content, the output strategy of the large language model to be optimized is iteratively optimized until the preset iteration conditions are met, and the optimized large language model is obtained.

[0273] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the model optimization method provided by the above methods. This method includes: for any output content of the output of the large language model to be optimized, analyzing the output content to obtain a semantic naturalness evaluation value, an expressive diversity evaluation value, and a generative fluency evaluation value of the output content; wherein the semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained from the discriminator network through a nonlinear mapping function; the expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained from the unique word ratio of N-gram phrases through a segmented mapping function; and the generative fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function.

[0274] The reward score for the output content is determined based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value.

[0275] Based on the reward score corresponding to each output content, the output strategy of the large language model to be optimized is iteratively optimized until the preset iteration conditions are met, and the optimized large language model is obtained.

[0276] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0277] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0278] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model optimization method, characterized by, include: For any output content of the large language model to be optimized, the output content is analyzed to obtain a semantic naturalness evaluation value, an expressive diversity evaluation value, and a generation fluency evaluation value. The output content is text content. The semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained from the discriminator network through a nonlinear mapping function. The expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained from the unique word ratio of N-gram phrases through a segmented mapping function. The generation fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function. The reward score for the output content is determined based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value. Based on the reward score corresponding to each output content, the output strategy of the large language model to be optimized is iteratively optimized until the preset iteration conditions are met, and the optimized large language model is obtained.

2. The model optimization method of claim 1, wherein, The method for determining the semantic naturalness evaluation value includes: Construct a discriminator network, which includes: a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator; Based on the discriminator network, the output content is analyzed to obtain the sentence repetition score, word stereotype score, and logical redundancy score of the output content. Based on the average scores of the sentence repetition score, word stereotypedness score, and logical redundancy score, the machine feature strength score is obtained. The machine feature strength score is converted into a semantic naturalness evaluation value through a nonlinear mapping function, which results in a higher semantic naturalness evaluation value as the machine feature strength score decreases.

3. The model optimization method of claim 2, wherein, The construction process of the discriminator network includes: Collect original human-generated text as positive samples; The original human text is rewritten by machine in various ways to generate machine output text with different machine feature strength scores as negative samples. The machine rewriting includes prompt-guided rewriting and cross-language back translation. The teacher model was used to label the positive and negative samples from three dimensions: sentence repetition, word stereotypedness, and logical redundancy, resulting in a data sample set labeled with sentence repetition, word stereotypedness, and logical redundancy. Based on the sentence repetition annotation data sample set, the word stereotype annotation data sample set, and the logical redundancy annotation data sample set, a sentence repetition discriminator, a word stereotype discriminator, and a logical redundancy discriminator are trained respectively, and the discriminator network is obtained through ensemble learning.

4. The model optimization method of claim 3, wherein, The cue-guided rewriting includes: rewriting human-original text using cue words of at least two different styles and intensities; The cross-language back-translation includes: a process of translating Chinese text into English and then back into Chinese; The ratio of positive to negative samples is dynamically adjusted based on the training performance of the discriminator.

5. The model optimization method of claim 1, wherein, The method for determining the expression diversity assessment value includes: Calculate the unique word ratio of the N-gram phrases in the output content. The unique word ratio of the N-gram phrases is defined as the ratio of the number of segments composed of N consecutive words that are not repeated to the total number of segments composed of all N consecutive words; where N is a positive integer. Text length normalization is performed on the unique word ratios of N-gram phrases with different N values. The unique word ratios of multiple normalized N-gram phrases are then weighted and combined to obtain the comprehensive unique word ratio. The comprehensive unique word ratio is converted into an expression diversity assessment value using a segmented mapping function.

6. The model optimization method of claim 5, wherein, The process of converting the comprehensive unique word ratio into an expression diversity assessment value through a segmented mapping function includes: When the overall unique word ratio is lower than a preset lower threshold, a negative penalty value is calculated based on the square of the difference between the overall unique word ratio and the preset lower threshold, and used as the expression diversity evaluation value. When the overall unique word ratio is between a preset lower threshold and a preset upper threshold, a positive reward value is calculated based on the distance between the overall unique word ratio and a preset target value, and this value is used as the expression diversity evaluation value. When the overall unique word ratio is higher than a preset upper threshold, an attenuation suppression value is calculated based on the difference between the overall unique word ratio and the preset upper threshold, and used as the expression diversity evaluation value. The preset lower threshold, preset upper threshold, and preset target value are determined by statistical analysis of the diversity distribution of human text in the target domain.

7. The model optimization method of claim 1, wherein, The method for determining the generated fluency evaluation value includes: Based on the large language model to be optimized, the predicted perplexity of each word position is determined during the generation of the output content, forming a perplexity sequence of the output content; The standard deviation of the perplexity sequence is used as the perplexity fluctuation value, and a fluency evaluation value is determined by a linear penalty function based on the deviation between the perplexity fluctuation value and the target fluctuation value. The target fluctuation value is obtained by performing perplexity analysis on a large-scale human text corpus.

8. The model optimization method of claim 1, wherein, Based on the semantic naturalness assessment value, expressive diversity assessment value, and generation fluency assessment value, the reward score for the output content is determined, including: The semantic naturalness evaluation value, expressive diversity evaluation value, and generation fluency evaluation value are weighted and summed to determine the reward score of the output content; The weight of the semantic naturalness evaluation value is greater than the weight of the expression diversity evaluation value, and the weight of the expression diversity evaluation value is greater than the weight of the generation fluency evaluation value.

9. A model optimization apparatus characterized by comprising: include: The analysis module is used to analyze any output content of the large language model to be optimized, and obtain a semantic naturalness evaluation value, an expressive diversity evaluation value, and a generation fluency evaluation value for the output content. The output content is text content. The semantic naturalness evaluation value is obtained by transforming the machine feature strength score obtained from the discriminator network through a nonlinear mapping function. The expressive diversity evaluation value is obtained by transforming the comprehensive unique word ratio obtained from the unique word ratio of N-gram phrases through a segmented mapping function. The generation fluency evaluation value is obtained by transforming the perplexity fluctuation value of the perplexity sequence through a linear penalty function. An evaluation module is used to determine the reward score of the output content based on the semantic naturalness evaluation value, the expressive diversity evaluation value, and the generation fluency evaluation value. The optimization module is used to iteratively optimize the output strategy of the large language model to be optimized based on the reward score corresponding to each output content, until the preset iteration conditions are met, and the optimized large language model is obtained.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the model optimization method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the model optimization method as described in any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the model optimization method as described in any one of claims 1 to 8.

Images