Tobacco licensing business text block dynamic segmentation method based on proximal policy optimization

Abstract

The present application relates to the technical field of natural language processing, and particularly provides a tobacco licensing business text block dynamic segmentation method based on proximal policy optimization, which adopts a text block dynamic segmentation model to perform text segmentation processing on a tobacco licensing business text to obtain a text segmentation result; a training process of the text block dynamic segmentation model comprises a supervised fine-tuning warm start stage and a proximal policy optimization stage; in the proximal policy optimization stage, a pre-trained text segmentation quality evaluation model is taken as a reward function and combined into a proximal policy optimization algorithm to optimize the tobacco licensing business text segmentation result. The method can make the proximal policy optimization algorithm work on a language task, and significantly improve the semantic integrity and retrieval precision of the tobacco licensing business text.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and more particularly to a tobacco licensing business text block dynamic segmentation method based on proximal policy optimization. BACKGROUND

[0002] In the field of tobacco monopoly administrative licensing services, with the continuous improvement of service intelligence level, there is an increasingly urgent need to provide efficient and accurate business consulting services for licensing users through intelligent question and answer systems. However, the business text in this field is usually complex in structure, has many professional terms, and has various semantic expression forms, which leads to significant challenges in information understanding and accurate retrieval for the system. The current intelligent question and answer systems are mostly based on the Retrieval-Augmented Generation (RAG) architecture, but still face many difficulties in real applications, such as incomplete recall information, fragmented semantic context, redundant or missing answer content, and other problems, which seriously affect the system's accurate response ability to user questions.

[0003] To alleviate the above problems, researchers have introduced text chunk division strategies in the text preprocessing stage, aiming to divide the original business documents into several semantically complete segments for subsequent indexing and retrieval. Current mainstream methods include static cutting strategies based on fixed length, punctuation marks, and paragraph structure, as well as some semantic cutting models based on deep learning. Static cutting methods cannot balance semantic completeness and granularity control. Cutting strategies based on deep models lack adjustability and global optimization mechanisms.

[0004] In addition, some research has attempted to introduce reinforcement learning methods to determine cutting positions through policy learning, thereby maximizing downstream task performance. If the existing language model uses proximal policy optimization (PPO), the following difficulties are faced: (1) The output head of the existing language model is LM Head, which outputs a word sequence probability distribution. The output of the text segmentation task should be a sequence label or a binary classification signal of the cutting position, which are incompatible in terms of task form and output space. (2) The reward of traditional proximal policy optimization training is the immediate feedback signal from the task environment, but there is no objective reward signal that can be automatically calculated in the text cutting task. There is no way to obtain an explicit numerical reward, so it is difficult to accurately reflect the real impact of cutting effect on retrieval performance.

[0005] Therefore, in order to improve the availability and retrieval accuracy of tobacco licensing business text in the intelligent question and answer system, a tobacco licensing business text block dynamic segmentation method is urgently needed to enable the model to dynamically decide the cutting position based on reading the complete business text, intelligently adjust the chunk granularity, and ultimately effectively improve the performance of the question and answer system in the actual business scenario. SUMMARY

[0006] In order to overcome the shortcomings and deficiencies in the prior art, the purpose of the present application is to provide a tobacco licensing business text block dynamic segmentation method based on proximal policy optimization; the method uses a reinforcement learning framework based on proximal policy optimization algorithm, uses a trained text cutting quality evaluation model to generate a semantic level reward, so that the proximal policy optimization algorithm can work on language tasks, and significantly improve the semantic integrity and retrieval accuracy of tobacco licensing business text.

[0007] In order to achieve the above purpose, the present application is implemented by the following technical scheme: a tobacco licensing business text block dynamic segmentation method based on proximal policy optimization, which uses a text block dynamic segmentation model to perform text segmentation processing on tobacco licensing business text to obtain a text cutting result;

[0008] The training process of the text block dynamic segmentation model includes: a supervised fine-tuning warm start phase for learning general text cutting, and a proximal policy optimization phase for reinforcement learning fine-tuning for tobacco licensing business text; the proximal policy optimization phase uses a pre-trained text cutting quality evaluation model as a reward function, which is combined into the proximal policy optimization algorithm to optimize the tobacco licensing business text cutting result.

[0009] Preferably, the text cutting quality evaluation model includes a Tokenizer word segmentation module one, a scoring Transformer layer, and a scoring head;

[0010] The input of the text cutting quality evaluation model is a string structure;

[0011] In the proximal policy optimization phase of the text block dynamic segmentation model training process, the input text cutting quality evaluation model string structure refers to: the text block dynamic segmentation model obtains a text cutting result in one text segmentation processing, the text cutting result includes a plurality of text blocks generated according to cutting points; all the generated text blocks form a string structure;

[0012] String structure x i Spliced to the scoring prompt word template, then encoded into Token sequence one by Tokenizer word segmentation module one, and then input into the scoring Transformer layer to complete context modeling and extract the hidden state of the first Token; the scoring head extracts the hidden state of the first Tokenh x i ) ， mapping to prediction score : wherein W1 and b1 are learnable parameters respectively; the prediction score is mapped to a specified score interval to obtain a final score.

[0013] Preferably, in the pre-training of the text cutting quality evaluation model, the sample data set construction method is: generating different text cutting results of the tobacco certification business text through various strategies to form a sample data set.

[0014] Preferably, the pre-training method of the text cutting quality evaluation model is:

[0015] dividing the sample data set into a training data set and a validation data set;

[0016] encoding each text block of each text cutting result of the training data set into a chunk sequence, connecting through a delimiter to form a string structure, splicing the string structure to a score prompt word template, and then processing through a tokenizer module one, a scoring transformer layer and a scoring head to obtain a prediction score;

[0017] taking mean square error loss function L MSE as the optimization objective, and using Adam optimizer to update parameters;

[0018] After each round of training, the performance of the text cutting quality evaluation model is evaluated on the validation data set; the evaluation indexes include average mean square error and Spearman rank correlation coefficient.

[0019] Preferably, the text block dynamic segmentation model comprises: tokenizer module two, cutting transformer layer and multi-classification output head.

[0020] The input of the text block dynamic segmentation model is the tobacco certification business text to be segmented; first, the tobacco certification business text is divided into sentences through a sentence divider, and then encoded into Token sequence two through tokenizer module two.

[0021] ​In the Token sequence two, the sentence absolute position encoding and the paragraph relative bias information are added; then input to the cut Transformer layer to complete the context modeling and extract the hidden state of the first Token; the multi-classification output head is a linear layer with the shape of hidden_size x N, wherein hidden_size is the dimension of the output vector of the cut Transformer layer, N is the number of candidate cutting points, which acts on the hidden state of the first Token to generate a prediction vector with the length of N ; wherein, , i = 1, 2, …, N, represents the probability that the i-th candidate cutting point is selected as the final cutting point; the maximum probability corresponding to the candidate cutting point is selected as the final cutting point.

[0022] Preferably, the prediction vector is:

[0023] ;

[0024] wherein, is a linear weight, d is the dimension of the hidden state vector, is a bias term, h [BOS] is the hidden state of the first Token, and Softmax is an activation function.

[0025] Preferably, a dynamic sliding window mechanism is used to process the input tobacco licensing business text: a document block window containing L_w sentences is set; the tobacco licensing business text in the document block window is input into the text block dynamic segmentation model to process the text segmentation of the tobacco licensing business text; whenever a cutting point is predicted according to the text cutting result obtained from the text block dynamic segmentation model, the starting position of the document block window is updated to the cutting point to continue processing the subsequent input tobacco licensing business text.

[0026] Preferably, in the proximal policy optimization phase of the text block dynamic segmentation model training process, the pre-trained text cutting quality evaluation model is used as the reward function, which means that the tobacco licensing business text is input into the policy network, the policy network outputs the text cutting result, the output obtained by inputting the text cutting result into the pre-trained text cutting quality evaluation model is taken as the reward value, which is substituted into the calculation of the time difference error, and then the generalized advantage estimation is calculated and the policy network is updated by using the policy gradient method.

[0027] A readable storage medium, wherein the storage medium stores a computer program, the computer program causes the processor to execute the tobacco licensing business text block dynamic segmentation method based on proximal policy optimization when the processor executes the computer program.

[0028] A computer device comprises a processor and a memory for storing a program executable by the processor, and the processor implements the method for dynamically segmenting a tobacco licensing business text block based on a proximal policy optimization when executing the program stored in the memory.

[0029] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0030] 1. The present application adopts a reinforcement learning framework based on a proximal policy optimization algorithm, dynamically learns a text cutting strategy, and introduces a semantic task without explicit rewards into the reinforcement learning framework, generates a semantic-level reward by using a trained text cutting quality evaluation model, so that the proximal policy optimization algorithm can work on language tasks, significantly improves the semantic integrity and retrieval accuracy of the tobacco licensing business text, and gradually discovers which cutting points are beneficial to subsequent retrieval and question and answer generation through continuous optimization of the cutting strategy, thereby enhancing the response accuracy of the system;

[0031] 2. The present application introduces a text cutting quality evaluation model based on a large language model, which can effectively evaluate the generated text cutting results in multiple dimensions; through fine scoring of the text cutting results, a reward signal in the reinforcement learning process is provided, so that the text cutting strategy is more in line with the actual application requirements, and the overall performance of the question and answer system is improved;

[0032] 3. The text block dynamic segmentation model proposed by the present application is fine-tuned through a large language model architecture, so that the cutting point can be accurately to the sentence level, thereby realizing more fine segmentation; this makes the complex text in the tobacco licensing business field be cut more naturally and more in line with the semantic logic, avoiding the information loss or context fragmentation problem caused by the coarse-grained cutting in the traditional method;

[0033] 4. The sample data set construction method proposed by the present application, combined with the actual text in the tobacco monopoly administrative license field, can provide rich training samples for the reinforcement learning model, and ensure that the generated cutting results meet the actual business requirements;

[0034] 5. The present application introduces a dynamic sliding window mechanism, further improves the processing capacity of long text, and reduces redundant calculation. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a schematic diagram of the present application based on a proximal policy optimization tobacco licensing business text block dynamic segmentation method;

[0036] Figure 2 is a structural schematic diagram of the text block dynamic segmentation model of the present application;

[0037] Figure 3 is a structural schematic diagram of the text cutting quality evaluation model of the present application;

[0038] Figure 4 is a flowchart of the dynamic sliding window mechanism of the present application. DETAILED DESCRIPTION

[0039] The application will be further described in conjunction with the drawings and specific embodiments.

[0040] Embodiment One

[0041] The tobacco licensing business text block dynamic segmentation method based on proximal policy optimization of the embodiment aims to improve the retrieval accuracy and answer quality of the intelligent question and answer system in the field of tobacco monopoly administrative licensing during long document processing, thereby enhancing the structured understanding ability and question and answer response effect of licensing policy knowledge. The present application focuses on the characteristics of long document length, dense semantics and complex structure of tobacco licensing business documents, and proposes a reinforcement learning driven tobacco licensing business text block dynamic segmentation method. The method is to use a text block dynamic segmentation model to process the text segmentation of tobacco licensing business text to obtain a text cutting result. The training process of the text block dynamic segmentation model includes: a supervised fine-tuning warm start stage for learning general text cutting, and a proximal policy optimization stage for reinforcement learning fine-tuning of tobacco licensing business text. The proximal policy optimization stage uses a pre-trained text cutting quality evaluation model as a reward function, which is combined into a proximal policy optimization algorithm to optimize the tobacco licensing business text cutting result, as shown in Figure 1 .

[0042] The text block dynamic segmentation model aims to predict the most appropriate cutting position in the tobacco licensing business text, realize fine-grained text cutting at the sentence level, and thus improve the accuracy of subsequent information retrieval and question and answer generation. The text block dynamic segmentation model is based on large language models such as LLaMA3-70B-Instruct, and through structural adjustment, the output head is replaced by a sentence-level sequence labeling classifier adapted to the text segmentation task. The model input is the text to be processed, and the output is the label result of whether each sentence of the text should be a cutting point.

[0043] As Figure 2As shown, the text block dynamic segmentation model includes: Tokenizer segmentation module two, cutting Transformer layer and multi-classification output head. The text block dynamic segmentation model is based on the autoregressive large language model architecture (LLaMA3.3-70BInstruct) and is customized and adjusted to adapt to the modeling needs of the sentence-level document segmentation task. The model aims to "predict the most suitable cutting point from the whole text", and modifies the original structure for language generation to enable accurate prediction of semantic breakpoints in long documents. In terms of structure design, the model retains the Transformer layer main body of the original LLM, but replaces the original language modeling output layer (LM Head) for text generation with a multi-classification output head, which has a shape of Linear(hidden_size×N), where hidden_size is the dimension of the cutting Transformer layer output vector, and N is the number of candidate cutting points. The multi-classification output head only acts on the hidden state of the first Token.

[0044] The input of the text block dynamic segmentation model is the tobacco certification business text to be segmented; first, the tobacco certification business text is divided into sentences by the sentence segmenter, and then encoded into Token sequence two by Tokenizer segmentation module two; to enhance the model's ability to model sentence semantics and structure, sentence absolute position encoding and paragraph relative bias information are added to Token sequence two, so that the model can capture cross-sentence context dependencies and potential structural boundary signals; then the sequence of the whole context is input into the cutting Transformer layer to complete context modeling and extract the hidden state of the first Token; the multi-classification output head acts on the hidden state of the first Token to generate a prediction vector of length N ; wherein, i=1,2,…,N, represents the probability that the i-th candidate cutting point is selected as the final cutting point; the prediction vector is:

[0045] ;

[0046] wherein, is a linear weight, d is the dimension of the hidden state vector, is a bias term, h [BOS] is the hidden state of the first Token, Softmax is an activation function used to normalize the scores of all candidate cutting points; the maximum probability corresponding to the candidate cutting point is selected as the final cutting point.

[0047] To achieve accurate recognition of text segmentation points, this invention adopts a two-stage joint training strategy of "supervised fine-tuning + reinforcement learning", which aims to balance the model's basic judgment ability and strategy optimization ability, thereby improving the robustness and generalization of the final generated results in real-world scenarios.

[0048] The training process of the dynamic text block segmentation model includes: a supervised fine-tuning warm-start phase for learning general text segments, and a proximal policy optimization phase for fine-tuning based on tobacco license business texts using reinforcement learning.

[0049] In the first stage of training the dynamic text block segmentation model, a supervised fine-tuning warm-start strategy is introduced. This strategy is initially trained using explicitly labeled paragraph boundary data, enabling the model to acquire basic global cut point selection capabilities and providing stable policy initialization for the subsequent reinforcement learning stage. The supervised fine-tuning warm-start stage formalizes the text segmentation problem as a multi-classification task.

[0050] The goal of the dynamic text block segmentation model is to select the most suitable position from N potential cutting points for paragraph segmentation; the training data uses the publicly available Chinese document segmentation dataset Wiki-zh. The model adopts a structure based on a pre-trained large language model (such as LLaMA3).

[0051] The optimization objective is to minimize the cross-entropy loss L between the predicted vector and the true cut point label.

[0052] ;

[0053] Where N is the number of candidate cut points; y i ∈{0,1} indicates whether the i-th candidate cut point is a true cut point. y i =0 indicates that it is not a true cut point. y i =1 indicates that it is a real cutting point (only one position is 1, and the rest are 0). For the prediction vector The i-th component is the probability predicted for the i-th candidate cut point; by backpropagation and using the AdamW optimizer to iteratively update the model parameters, the model gradually learns to identify the optimal cut position from the overall sentence sequence.

[0054] To alleviate the problem of scarce cut point samples, a candidate point enhancement strategy is further introduced, which increases the sampling ratio of real cut points in the training data (e.g., maintaining it at 30%), to avoid the model being overly biased towards non-cut regions in the early stages of training, thereby improving the robustness and generalization of the judgment ability.

[0055] In the proximal policy optimization stage after the supervised fine-tuning warm-up stage, the present application introduces the Proximal Policy Optimization (PPO) algorithm in reinforcement learning to optimize the text segmentation strategy and improve the adaptability and accuracy of the model. The Proximal Policy Optimization algorithm is a deep reinforcement learning algorithm based on a policy-value network architecture, which includes two main components: a policy network and a value network, used to learn the optimal policy and evaluate the value of the policy, respectively. In the text segmentation task of the present application, the policy network is responsible for selecting the text segmentation action (i.e., determining whether to cut after the current sentence) based on the current state, while the value network is responsible for evaluating the expected return of each action in the current state. The Proximal Policy Optimization algorithm optimizes the policy of the policy network, enabling it to more accurately predict the cutting point in the text segmentation task.

[0056] In the proximal policy optimization stage, the present application introduces a pre-trained text segmentation quality evaluation model as a reward function, which is combined into the Proximal Policy Optimization algorithm to optimize the text segmentation results of the tobacco licensing business. The text segmentation quality evaluation model aims to objectively and stably score the quality of the text segmentation results generated by the text block dynamic segmentation model, thereby providing an effective reward signal for the Proximal Policy Optimization. The text segmentation quality evaluation model is based on a large language model architecture, which inputs the complete segmentation results and outputs a single score value, reflecting the comprehensive quality level of the current segmentation scheme in terms of semantic continuity, keyword coverage, concept integrity, etc. Through this model, the text block dynamic segmentation model can be effectively driven to iteratively optimize, generating more reasonable and semantically natural text segmentation results, and improving the adaptability and information carrying efficiency of the text blocks of the tobacco licensing business in the question-answering system.

[0057] The text segmentation quality evaluation model is constructed based on a large language model (such as LLaMA3.3-70B instruct). The text segmentation quality evaluation model includes a Tokenizer tokenization module, a scoring Transformer layer, and a scoring head. The scoring Transformer layer refers to the Transformer layer of the pre-trained language model. To achieve overall scoring of the text segmentation results, the text segmentation quality evaluation model structurally replaces the output head (LM Head) of the original large language model with a scoring head, which is a hidden_size x 1 linear layer, where hidden_size is the dimension of the output vector of the scoring Transformer layer, which is the same as the dimension of the output vector of the cutting Transformer layer.

[0058] The input of the text cutting quality evaluation model is a string structure; in the proximal policy optimization stage of the text block dynamic segmentation model training process, the input text cutting quality evaluation model string structure refers to the text cutting result obtained by the text block dynamic segmentation model in one text segmentation process; the text cutting result includes a plurality of text blocks generated according to cutting points; all the generated text blocks form a JSON string structure;

[0059] String structure x i Spliced to the scoring prompt word template, and then encoded into a Token sequence by the Tokenizer word segmentation module, and then input into the scoring Transformer layer to complete context modeling and extract the hidden state of the first Token; the scoring head maps the hidden state of the first Token h ( x i ) to a predicted score : where W1 and b1 are learnable parameters; the score is mapped to [0, 1] or a specified score interval by Sigmoid to obtain the final score, to support regression loss or proximal policy optimization reward normalization processing. The structure of the text cutting quality evaluation model and the design of its scoring prompt word template are shown in the accompanying Figure 3 drawings.

[0060] In the pre-training of the text cutting quality evaluation model, the sample data set aims to depict the performance differences of different text cutting strategies in actual question and answer applications, construct scoring samples consistent with human semantic preferences, provide clear supervision signals for the model, and further optimize its recognition and discrimination ability for high-quality text cutting schemes. The constructed sample data set is centered around tobacco monopoly administrative license business related texts; mainly from policy and regulation documents such as “Tobacco Monopoly License Management Measures” and “Tobacco Monopoly Law of the People’s Republic of China”, as well as official website public release of government affairs materials such as license guide, business guide, frequently asked questions, etc. These texts have the characteristics of concentrated theme, clear terminology, and clear structure, and can fully cover the core links and important policy information in the tobacco licensing process, providing an ideal basis for constructing cutting evaluation samples.

[0061] The sample data set construction method is: generate different text cutting results for each tobacco licensing business text (long text) through multiple strategies to form a sample data set; the strategies include: uniform cutting based on a fixed window, rule-based sentence segmentation cutting, semantic segmentation guided by a prompt word through a large language model and predicted cutting by a reinforcement learning strategy model.

[0062] To evaluate the quality of different cutting schemes, the application adopts manual scoring to mark the quality of each sample in the sample data set. Manual scoring mainly considers the following four aspects: attention to semantic integrity, investigation of concept splitting degree, evaluation of structural rationality and coherence between blocks, and evaluation of consistency and appropriateness of block size. The finally constructed sample data set is organized and stored in a structured format, each sample containing original text content, multiple candidate cutting schemes and their corresponding manual scores. The sample data set can fully reflect human preference for text semantic structure division, provide high-quality supervision signal for text cutting quality evaluation model, and then guide the text block dynamic segmentation model to generate text cutting schemes that meet the semantic rules and retrieval requirements in the reinforcement learning stage.

[0063] The pre-training method of the text cutting quality evaluation model is:

[0064] The sample data set is divided into a training data set and a validation data set;

[0065] Each text block of each text cutting result of the training data set is encoded as a chunk sequence, connected by a delimiter to form a string structure, and the string structure is spliced into a scoring prompt word template. Then, it is processed by the Tokenizer word segmentation module I, the scoring Transformer layer and the scoring head to obtain the predicted score .

[0066] To optimize the consistency of the predicted score and the manual score, the mean square error loss function L MSE is used as the optimization target, and the Adam optimizer is used for parameter update:

[0067] .

[0068] wherein, r i is the manual score; M is the number of samples participating in the calculation; the initial learning rate is set to 1x10 -5 , batchsize is set to 16, and linear warm-up and cosine decay strategies are introduced to control the dynamic decay of the learning rate.

[0069] After each round of training, the performance of the text segmentation quality evaluation model is evaluated on the validation dataset; evaluation indicators include mean squared error (MSE) and Spearman rank correlation coefficient; the mean squared error is used to measure the absolute prediction error, and the Spearman rank correlation coefficient is used to measure the consistency of the model in ranking samples, which is suitable for the ranking requirements of the advantage function in the downstream proximal policy optimization phase; when the mean squared error no longer decreases by more than a set number of rounds (such as 3 rounds), stop training to prevent overfitting.

[0070] The proximal policy optimization phase of the text block dynamic segmentation model training process includes the following steps:

[0071] S1, set the text block dynamic segmentation model after the supervised fine-tuning warm-up phase as a policy-value network architecture, including a policy network and a value network; the policy network and the value network share a cut Transformer layer;

[0072] S2, input the tobacco certificate business text s t to the policy network, and the policy network outputs a text segmentation result a t ; the text segmentation result a t is input into the pre-trained text segmentation quality evaluation model to obtain a reward value R t ; then calculate the time difference error δ t , which represents the difference between the prediction and the reward:

[0073] δ t = R t +γV( s t+1 , w t+1 )- V( s t , w t );

[0074] wherein V( ) is the value network, s t 、 s t+1 is t 、 t+ 1 period input to the policy network tobacco certificate business text, w t 、 w t+1 is t 、 t+1 the parameters of the value network, and g is the discount factor of the reward, generally between 0 and 1;

[0075] S3, using the timing difference error δ t , calculate the generalized advantage estimate;

[0076] S4, update the policy network using the policy gradient method, optimize the policy by maximizing the objective function, and introduce the importance sampling ratio and the clipping mechanism to avoid excessive policy updates;

[0077] S5, minimize the mean square error between the value network prediction and the actual value, and update the value network;

[0078] S6, after each round of training, evaluate the performance of the text block dynamic segmentation model using the validation set, calculate the average mean square error MSE and the Spearman rank correlation coefficient, and measure the accuracy and ranking consistency.

[0079] The dynamic sliding window mechanism is used to process the input tobacco licensing business text, ensuring context coverage and cutting point diversity. The dynamic sliding window mechanism processing diagram is shown in Figure 4 A document block window containing L_w sentences is set; the tobacco licensing business text in the document block window is input into the text block dynamic segmentation model for text segmentation processing; whenever a cutting point is predicted according to the text cutting result obtained by the text block dynamic segmentation model, the starting position of the document block window is updated to the cutting point to continue processing the subsequent input tobacco licensing business text.

[0080] Embodiment two

[0081] The embodiment is a readable storage medium, wherein the readable storage medium stores a computer program, and the computer program causes the processor to execute the tobacco licensing business text block dynamic segmentation method based on proximal policy optimization according to embodiment one when executed by the processor.

[0082] Embodiment three

[0083] The embodiment is a computer device, comprising a processor and a memory for storing a processor executable program, and the processor executes the program stored in the memory to realize the tobacco licensing business text block dynamic segmentation method based on proximal policy optimization according to embodiment one.

[0084] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement methods, and all shall be included in the protection scope of the present application.

Claims

1. A tobacco licensing business text block dynamic segmentation method based on proximal policy optimization, characterized by: The text block dynamic segmentation model is used for text segmentation processing on tobacco certification business text to obtain a text segmentation result. The training process of the text block dynamic segmentation model includes a supervised fine-tuning warm start phase for learning general text segmentation and a proximal policy optimization phase for reinforcement learning fine-tuning of tobacco certification business text; in the proximal policy optimization phase, a pre-trained text segmentation quality evaluation model is used as a reward function and combined into a proximal policy optimization algorithm to optimize the tobacco certification business text segmentation result. The text block dynamic segmentation model includes a Tokenizer word segmentation module two, a cutting Transformer layer, and a multi-classification output head. The input of the text block dynamic segmentation model is the tobacco certification business text to be segmented; the tobacco certification business text is first divided into sentences by a sentence divider, and then encoded into a Token sequence two by the Tokenizer word segmentation module two. In the Token sequence two, the sentence absolute position encoding and paragraph relative bias information are added; then input to the cut Transformer layer to complete the context modeling and extract the hidden state of the first Token; the multi-classification output head is a linear layer with the shape of hidden_size x N, where hidden_size is the dimension of the output vector of the cut Transformer layer, N is the number of candidate cutting points, and the linear layer acts on the hidden state of the first Token to generate a prediction vector with the length of N ; wherein , i = 1, 2, …, N, represents the probability that the ith candidate cutting point is selected as the final cutting point; the maximum probability corresponding candidate cutting point is selected as the final cutting point. In the proximal policy optimization phase of the text block dynamic segmentation model training process, the pre-trained text segmentation quality evaluation model is used as a reward function, which means that the tobacco certification business text is input into the policy network, the policy network outputs the text segmentation result, the text segmentation result is input into the pre-trained text segmentation quality evaluation model to obtain the output as the reward value, which is substituted into the calculation of the time difference error, and then the generalized advantage estimation is calculated and the policy network is updated by the policy gradient method.

2. The method for dynamic segmentation of tobacco licensing business text block based on near-end policy optimization according to claim 1, characterized in that: The text segmentation quality evaluation model includes a Tokenizer word segmentation module one, a scoring Transformer layer, and a scoring head. The input of the text segmentation quality evaluation model is a string structure. In the proximal policy optimization phase of the text block dynamic segmentation model training process, the input text segmentation quality evaluation model string structure means that the text block dynamic segmentation model obtains a text segmentation result in one text segmentation processing, and the text segmentation result includes several text blocks generated according to the cutting points; all the generated text blocks form a string structure. String structure x i Concatenate to the scoring prompt word template, then encoded as Token sequence via Tokenizer module, and input to the scoring Transformer layer to complete context modeling and extract the hidden state of the first Token; the scoring head maps the hidden state of the first Token to the predicted score h ( x i ) ， to the specified score interval to obtain the final score. : where W1 and b1 are learnable parameters, respectively; the predicted score is mapped to the specified score interval to obtain the final score.

3. The method of claim 2, wherein the method is a method of dynamic segmentation of a tobacco licensing business text block based on proximal policy optimization, characterized by: In the pre-training of the text segmentation quality evaluation model, the sample data set construction method is to generate different text segmentation results of the tobacco certification business text by multiple strategies to form a sample data set.

4. The method of claim 3, wherein the method is a method of dynamic segmentation of a tobacco licensing business text block based on proximal policy optimization, characterized in that: The pre-training method of the text segmentation quality evaluation model is: Divide the sample data set into a training data set and a validation data set; Encode each text block of each text segmentation result in the training data set into a chunk sequence, connect them by a delimiter to form a string structure, splice the string structure to a scoring prompt word template, and then process it through the Tokenizer word segmentation module one, the scoring Transformer layer, and the scoring head to obtain a predicted score; with mean square error loss function L MSE For optimization target, parameter update is performed by Adam optimizer. After each round of training, evaluate the performance of the text segmentation quality evaluation model on the validation data set; the evaluation indicators include the average mean square error and the Spearman rank correlation coefficient.

5. The method for dynamic segmentation of tobacco license application text blocks based on near-end strategy optimization according to claim 1, characterized in that: The prediction vector is: ； where, is a linear weight, d is the dimension of the hidden state vector, is a bias term, h [BOS] is the hidden state of the first Token, and Softmax is an activation function.

6. The method for dynamic segmentation of tobacco license application text blocks based on near-end strategy optimization according to claim 1, characterized in that: The dynamic sliding window mechanism is adopted to process the input tobacco licensing business text: a document block window containing L_w sentences is set; the tobacco licensing business text in the document block window is input into the text block dynamic segmentation model to perform text segmentation processing on the tobacco licensing business text; whenever a cutting point is predicted according to the text cutting result obtained by the text block dynamic segmentation model, the starting position of the document block window is updated to the cutting point to continue processing the subsequent input tobacco licensing business text.

7. A readable storage medium, characterized by, The storage medium stores a computer program, and the computer program causes the processor to execute the near-end policy optimization-based tobacco licensing business text block dynamic segmentation method in any one of claims 1-6 when executed by the processor.

8. A computer device comprising a processor and a memory for storing a processor executable program, characterized in that, The processor executes the program stored in the memory, and the near-end policy optimization-based tobacco licensing business text block dynamic segmentation method in any one of claims 1-6 is implemented.

Images