Method and device for predicting official document cursor based on large language model

Abstract

The invention discloses an official document cursor prediction method and device based on a large language model. The method comprises the following steps: acquiring historical operation data, context associated data and operation habit data of a user; obtaining a large language model used for predicting the cursor position of the official document; training the large language model to obtain a trained large language model used for predicting the cursor position of the official document; acquiring user real-time operation data and context real-time data; and inputting the user real-time operation data and the context real-time data into a trained large language model used for predicting the cursor position of the official document so as to obtain a cursor prediction position output by the trained large language model used for predicting the cursor position of the official document. According to the method, the high-precision prediction model is trained in combination with the deep understanding ability of the large language model for document semantics, intelligent pre-judgment of next editing operation of the user is achieved, and the cursor position is accurately predicted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of official document data processing, and particularly relates to an official document cursor prediction method and device based on a large language model. BACKGROUND

[0002] In the current digital office scenario, official document editing is an important information processing link in the fields of government affairs and business, and its efficiency and accuracy directly affect the work quality and process advancement. With the increasing length and increasingly complex format of official documents, users face many challenges during the editing process. Traditional manual cursor positioning methods rely on users to find and modify areas word by word and sentence by sentence, which is not only tedious, but also prone to positioning deviations due to visual fatigue, seriously hindering editing efficiency. At the same time, existing document editing systems lack intelligent perception and prediction capabilities for user editing intentions, and cannot actively guide the cursor to the target position, resulting in users having to repeatedly try and error and switch back and forth in the text, greatly increasing editing time costs and operational burdens. Therefore, how to achieve efficient and intelligent official document editing assistance has become a technical problem to be solved.

[0003] Currently, the most similar existing technology to the present application focuses on the field of text editing assistance, but there is no cursor prediction scheme for official document scenarios. In general text editing assistance technology, some researches predict editing behavior by analyzing user historical operation records. For example, in the document "A method and system for document assisted editing" (Author: Liyan, Patent CN113312884B), it is proposed to record user cursor movement, text insertion / deletion and other operation data, and construct a behavior prediction model based on Markov chain to predict the next editing operation of the user. However, this method only relies on historical operation data and does not fully consider the semantic information of the text content, and the prediction accuracy is low when facing complex semantic structure of official document editing scenarios.

[0004] In addition, some existing technologies use natural language processing (NLP) techniques to analyze text content to achieve editing assistance functions. For example, patent CN109255113B (Patent Name: Intelligent proofreading system) proposes to analyze the syntax and semantics of the text to identify key information and logical structures in the text and provide editing suggestions for users. However, this technology mainly focuses on text content optimization and lacks the combination of user editing operation behavior, making it impossible to accurately predict the cursor position.

[0005] In the field of large language model (LLM) applications, although some researches have used it for text generation and understanding, there is no technology scheme that combines large language models with user behavior data to specifically predict cursors in official document editing scenarios. Existing solutions have problems in balancing text semantic understanding and user operation habit analysis when processing official document editing, making it difficult to meet users' demand for efficient and intelligent official document editing assistance.

[0006] The prior art has obvious defects in official document editing cursor prediction. On the one hand, the traditional editing assistance technology based on user behavior analysis only relies on historical operation data to construct a prediction model, lacks a deep understanding of the semantic structure of official documents, and results in low cursor position prediction accuracy in complex semantic official document editing scenarios, which is difficult to meet the user's precise positioning needs. On the other hand, the text editing assistance scheme mainly based on natural language processing technology focuses on content optimization and cannot predict the user's cursor movement intention without combining the user's actual editing operation behavior, which has limited effect on improving editing efficiency. In addition, the existing large language model application does not target the official document editing scenario, effectively fuses model capabilities with user behavior data, and cannot realize intelligent and efficient cursor prediction function. SUMMARY

[0007] The purpose of the present application is to provide an official document cursor prediction method based on a large language model to at least solve one of the above technical problems.

[0008] In one aspect of the present application, an official document cursor prediction method based on a large language model is provided, which comprises:

[0009] Obtaining user historical operation data, context association data, and operation habit data;

[0010] Obtaining a large language model for predicting official document cursor positions;

[0011] Training the large language model according to the user historical operation data, context association data, and operation habit data, thereby obtaining a trained large language model for predicting official document cursor positions;

[0012] Obtaining user real-time operation data and context real-time data;

[0013] Inputting the user real-time operation data and context real-time data into the trained large language model for predicting official document cursor positions, thereby obtaining the cursor prediction position output by the trained large language model for predicting official document cursor positions.

[0014] Optionally, before the training of the large language model according to the user historical operation data, context association data, and operation habit data, thereby obtaining a trained large language model for predicting official document cursor positions, the official document cursor prediction method based on a large language model further comprises:

[0015] Obtaining official document format feature data, task chain data, reference and derivative association data, and error mode data;

[0016] The large language model is trained according to the user historical operation data, the context association data, and the operation habit data, so as to obtain a trained large language model for predicting the position of the cursor in the official document.

[0017] The large language model is trained according to the user historical operation data, the context association data, the operation habit data, the official document format feature data, the task chain data, the reference and derivation association data, and the error mode data, so as to obtain a trained large language model for predicting the position of the cursor in the official document.

[0018] Optionally, the large language model is trained according to the user historical operation data, the context association data, the operation habit data, the official document format feature data, the task chain data, the reference and derivation association data, and the error mode data, so as to obtain a trained large language model for predicting the position of the cursor in the official document.

[0019] The user historical operation data, the context association data, the operation habit data, the official document format feature data, the task chain data, the reference and derivation association data, and the error mode data are preprocessed respectively, so as to obtain preprocessed user historical operation data, preprocessed context association data, preprocessed operation habit data, preprocessed official document format feature data, preprocessed task chain data, preprocessed reference and derivation association data, and preprocessed error mode data.

[0020] The training fusion features are generated according to the preprocessed user historical operation data, the preprocessed context association data, the preprocessed operation habit data, the preprocessed official document format feature data, the preprocessed task chain data, the preprocessed reference and derivation association data, and the preprocessed error mode data.

[0021] The large language model is trained using the training fusion features, so as to obtain a trained large language model for predicting the position of the cursor in the official document.

[0022] Optionally, the training fusion features are generated according to the preprocessed user historical operation data, the preprocessed context association data, the preprocessed operation habit data, the preprocessed official document format feature data, the preprocessed task chain data, the preprocessed reference and derivation association data, and the preprocessed error mode data.

[0023] extracting features of the preprocessed user historical operation data, the preprocessed context association data, the preprocessed operation habit data, the preprocessed official document format feature data, the preprocessed task chain data, the preprocessed reference and derivative association data, and the preprocessed error mode data respectively, thereby obtaining user historical operation features, context association features, operation habit features, official document format features, task chain features, reference and derivative association features, and error mode features;

[0024] constructing a feature association graph according to the user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivative association features, and the error mode features;

[0025] obtaining a trained graph convolution network;

[0026] inputting the feature association graph into the trained graph convolution network, thereby obtaining an interaction-enhanced feature vector set;

[0027] fusing the interaction-enhanced feature vector set through dynamic coupling, thereby obtaining a fusion feature for training.

[0028] Optionally, the constructing a feature association graph according to the user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivative association features, and the error mode features comprises:

[0029] defining the user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivative association features, and the error mode features as respective nodes in the graph;

[0030] taking an association strength between any one node and another node as an edge weight between the two nodes; wherein the edge weights of any two edges are obtained through the following formula:

[0031]

[0032] wherein w {i,j} is an edge weight of edge i and edge j; is a historical association strength, i.e., a frequency of co-occurrence of feature i and feature j; is a current scene association strength, which is dynamically adjusted according to a real-time editing scene (for example, if a "reference sentence" is detected, the is increased to 0.8); a is a balance coefficient (with a value of 0.3-0.7), which is dynamically adjusted according to the type of official document (for example, a is taken as 0.6 for a "notice" type official document with strict format, and a is taken as 0.4 for a "report" type official document with emphasis on semantics).

[0033] Optionally, the inputting the feature correlation graph into the trained graph convolutional network to obtain an interaction-enhanced feature vector set comprises:

[0034] For each feature node, its interaction-enhanced vector The formula is as follows by aggregating the self vector and the vector of the associated node:

[0035]

[0036] Wherein, is the interaction-enhanced vector; w i,j is the edge weight of the edge and edge j; W g and b g are trainable parameters of the GCN, optimized through model training; v i is the original feature vector of feature node n i ; v j is the original feature vector of feature node n j

[0037] Each interaction-enhanced vector constitutes an interaction-enhanced feature vector set.

[0038] Optionally, the fusion is performed by the following formula:

[0039]

[0040] Wherein, F is the training fusion feature; X is the number of feature categories; s i is the scene dominance of feature node i (value range 0-1), reflecting the importance of the feature in the current editing scene; λ i is the global importance coefficient of feature node i (preset as a constant greater than 0, which can be fine-tuned through model training), reflecting the general importance of the feature in the document editing scene; is the interaction-enhanced vector.

[0041] Optionally, the training of the large language model using the training fusion feature to obtain a trained large language model for predicting the position of the document cursor comprises:

[0042] Obtain a basic model;

[0043] Form multiple groups of paired features by grouping two-by-two features into a group, respectively for user historical operation features, context correlation features, operation habit features, document format features, task chain features, reference and derivative correlation features, and error mode features;

[0044] Take the basic model as a framework, and construct a special sub-model for each group of paired features. Each special sub-model is only used to train the data of the corresponding paired feature group ​

[0045] Each special sub-model is trained in an adversarial training manner, so as to obtain each robustness-enhanced sub-model.

[0046] Each special sub-model and the basic model are distilled in a knowledge distillation manner, so as to obtain a lightweight deployment model as a large language model for predicting the position of the cursor of the official document.

[0047] Optionally, each special sub-model is constructed for each group of paired feature groups based on the basic model, and each special sub-model is only used for training data of the corresponding paired feature group, which comprises:

[0048] Each special sub-model is constructed for each group of paired feature groups based on the basic model, and each special sub-model is only used for training data of the corresponding paired feature group.

[0049] Based on the formula The parameters of the basic model are fine-tuned to obtain the parameters of each special sub-model.

[0050] Wherein, θ is the core parameter of the basic model M; k represents the index of the scene sub-model. represents the prediction function of the kth special sub-model, the input is the training fusion feature, and the output is the predicted cursor position; θ ′ k is the fine-tuning parameter of the kth special sub-model; α is the learning rate of the special sub-model. is the loss function L k The gradient of the basic model parameter θ reflects the influence direction and degree of the change of the parameter θ on the loss function, and is used to guide the adjustment of the sub-model parameter.

[0051] The application also provides a large language model-based official document cursor prediction device, which comprises:

[0052] A data acquisition module is configured to acquire user historical operation data, context association data, and operation habit data.

[0053] A model acquisition module is configured to acquire a large language model for predicting the position of the cursor of the official document.

[0054] A training module is configured to train the large language model according to the user historical operation data, the context association data, and the operation habit data, so as to obtain a trained large language model for predicting the position of the cursor of the official document.

[0055] A real-time data acquisition module is configured to acquire user real-time operation data and context real-time data.

[0056] a prediction module configured to input the user real-time operation data and the context real-time data into a large language model trained for predicting the position of the document cursor, so as to obtain a cursor predicted position output by the large language model trained for predicting the position of the document cursor.

[0057] The large language model-based document cursor prediction method of the present application captures user editing behavior data by using historical trajectories, and combines the deep understanding ability of the large language model for document semantics to train a high-precision prediction model, so as to realize intelligent prediction of the next editing operation of the user and accurately predict the position of the cursor. The present application is committed to reducing the cumbersome operation of manually positioning the cursor, reducing the editing time cost and operation burden, significantly improving the efficiency and fluency of document browsing and modification, and providing users with a more intelligent and convenient document editing experience. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is a flowchart of the large language model-based document cursor prediction method of an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the embodiments of the present application will be described in more detail below in combination with the drawings of the embodiments of the present application. In the drawings, the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some of the embodiments of the present application, not all of the embodiments. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. The embodiments of the present application will be described in detail below in combination with the drawings.

[0060] As shown in the large language model-based document cursor prediction method includes: Figure 1

[0061] obtaining user historical operation data, context association data, and operation habit data;

[0062] obtaining a large language model for predicting the position of the document cursor;

[0063] training the large language model according to the user historical operation data, the context association data, and the operation habit data, so as to obtain a large language model trained for predicting the position of the document cursor;

[0064] obtaining user real-time operation data and context real-time data;

[0065] ​inputting the user real-time operation data and the context real-time data into a large language model trained for predicting the cursor position of the official document, to obtain a cursor prediction position output by the large language model trained for predicting the cursor position of the official document.

[0066] In the embodiment, the user historical operation data can include timestamps, coordinate positions of operations of the user on the cursor, such as moving, clicking, and dragging; and specific contents and positions of editing actions of the text, such as inserting, deleting, and modifying.

[0067] In the embodiment, the context association data can include text paragraphs, sentence contents, and semantic information of the context before and after the current editing area when the user operates.

[0068] In the embodiment, the operation habit data can include personalized behavior characteristics of the user, such as frequency of continuous operations, operation interval time, and preference for commonly used editing instructions.

[0069] In the embodiment, the captured data is structured in a preset format, forming a user behavior data set for subsequent analysis, and providing a data basis for model training.

[0070] In one embodiment, the following training process can be used:

[0071] A large language model with a parameter quantity of 700 billion is used as a core prediction model, and is trained in combination with the captured user behavior data, so that the large language model can learn user editing behavior patterns and cursor movement rules. The training process is as follows:

[0072] Data preprocessing: cleaning the captured user behavior data set to remove noise data and invalid operation records; performing word segmentation and vectorization processing on the text content, and converting the text content into an input format recognizable by the model.

[0073] Feature fusion: fusing operation features (such as cursor movement distance and editing frequency) in the user behavior data and text semantic features (such as word vectors and sentence representations) to construct a multi-dimensional input feature vector.

[0074] Model training: using the fused feature vector as input and the actual cursor position moved by the user as label, the large language model is trained in a supervised learning manner. During the training process, the model parameters are adjusted by optimizing the loss function (such as cross-entropy loss function), so that the model can accurately predict the cursor position.

[0075] Model evaluation and optimization: using a reserved validation data set to evaluate the trained model, measuring the prediction effect through accuracy, recall rate, mean square error, and other indicators, and adjusting the model hyperparameters or optimizing the training strategy according to the evaluation result to improve the model performance.

[0076] In the present embodiment, before the large language model is trained according to the user historical operation data, the context association data and the operation habit data, thereby obtaining the trained large language model for predicting the position of the official document cursor, the official document cursor prediction method based on the large language model further comprises:

[0077] obtaining official document format feature data, task chain data, reference and derivative association data and error mode data;

[0078] The training of the large language model according to the user historical operation data, the context association data and the operation habit data, thereby obtaining the trained large language model for predicting the position of the official document cursor comprises:

[0079] The training of the large language model according to the user historical operation data, the context association data, the operation habit data, the official document format feature data, the task chain data, the reference and derivative association data and the error mode data, thereby obtaining the trained large language model for predicting the position of the official document cursor.

[0080] In the present embodiment, the official document format feature data represents official document fixed format elements and format-semantic association data, for example, “document number [year] X number”, “written date right space 4 characters”, “main sending organ top grid”, “please matter section must include ‘item number + bold’ format”.

[0081] In the present embodiment, the task chain data includes official document editing full-process nodes, for example, “initial draft drafting → format checking → content auditing → signing final draft” and high-frequency operation features of each node such as “format checking stage high-frequency paragraph indentation adjustment”.

[0082] In the present embodiment, the reference and derivative association data includes reference sentences (for example, “in accordance with the <XX method>”) and derivative relationships (for example, “reply” associated with the “petition” official document) in official documents.

[0083] In the present embodiment, the error mode data includes user historical high-frequency correction errors specific to official documents (for example, “document number year lowercase” and “numbers not using Chinese characters”).

[0084] In the present embodiment, the official document format feature data, the task chain data, the reference and derivative association data and the error mode data described above can be preprocessed as follows:

[0085] The format elements in the official document format feature data are format coded (such as “written date” corresponding to a format rule vector), and are bound to corresponding semantic content (such as “2024 October 1”) as “format-semantic pairs”;

[0086] Marking feature vectors for task nodes in task chain data (such as the "format check" node corresponding to the "paragraph indentation, font adjustment" operation vector);

[0087] Parsing the referenced document ID and associated paragraph in the reference and derivative association data, and encoding as a "reference association vector"; marking the "parent-child" mapping relationship of the derivative document (such as "please inform item 3 → reply item 2");

[0088] Encoding error patterns (such as "year lowercase") as "error feature vectors" and associating their correction positions (such as "the number in the '[]' in the document number").

[0089] In this embodiment, the training of the large language model according to the user historical operation data, the context association data, the operation habit data, the document format feature data, the task chain data, the reference and derivative association data, and the error pattern data comprises:

[0090] The user historical operation data, the context association data, the operation habit data, the document format feature data, the task chain data, the reference and derivative association data, and the error pattern data are preprocessed respectively, so as to obtain preprocessed user historical operation data, preprocessed context association data, preprocessed operation habit data, preprocessed document format feature data, preprocessed task chain data, preprocessed reference and derivative association data, and preprocessed error pattern data;

[0091] The preprocessed user historical operation data, the preprocessed context association data, the preprocessed operation habit data, the preprocessed document format feature data, the preprocessed task chain data, the preprocessed reference and derivative association data, and the preprocessed error pattern data are used to generate training fusion features;

[0092] The large language model is trained using the training fusion features, so as to obtain a trained large language model for predicting the document cursor position.

[0093] In this embodiment, the generation of training fusion features according to the preprocessed user historical operation data, the preprocessed context association data, the preprocessed operation habit data, the preprocessed document format feature data, the preprocessed task chain data, the preprocessed reference and derivative association data, and the preprocessed error pattern data comprises:

[0094] extracting features of the preprocessed user historical operation data, the preprocessed context association data, the preprocessed operation habit data, the preprocessed official document format feature data, the preprocessed task chain data, the preprocessed reference and derivation association data, and the preprocessed error mode data respectively, thereby obtaining user historical operation features, context association features, operation habit features, official document format features, task chain features, reference and derivation association features, and error mode features;

[0095] constructing a feature association graph according to the user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivation association features, and the error mode features;

[0096] obtaining a trained graph convolution network;

[0097] inputting the feature association graph into the trained graph convolution network, thereby obtaining an interaction-enhanced feature vector set;

[0098] fusing the interaction-enhanced feature vector set through dynamic coupling, thereby obtaining a training fusion feature.

[0099] In the embodiment, constructing the feature association graph according to the user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivation association features, and the error mode features includes:

[0100] defining the user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivation association features, and the error mode features as respective nodes in the graph, for example, defining each dimension of features as a respective node in the graph, such as a user operation feature node, a context association feature node, an operation habit feature node, an official document format feature node, a task chain feature node, a reference-derivation association feature node, and an error mode feature node.

[0101] taking the association strength (value range 0-1) between any one node and another node as the edge weight between the two nodes; wherein the edge weight of any two edges is obtained through the following formula:

[0102]

[0103] wherein w {i,j} is the edge weight of edge i and edge j; is the historical association strength, that is, the frequency of co-occurrence of feature i and feature j; is the current scene association strength, which is dynamically adjusted according to the real-time editing scene; and a is a balance coefficient, which is dynamically adjusted according to the type of official document.

[0104] The calculation of the edge weight of the present application is described in detail below by way of example. It can be understood that the example does not constitute any limitation on the present application.

[0105] The calculation process is illustrated by taking the edge weight between the "user operation feature node" and the "official document format feature node" as an example, in combination with specific numerical values.

[0106] The definitions are as follows: n1 is a user operation feature (such as "moving the cursor to the first line of a paragraph" and "pressing the space bar to adjust the indentation"); n3 is a format feature (such as "no indentation on the first line of a paragraph" and "center alignment of a title"). It can be understood that, for the original feature vector of any feature node, n i and n j are used to represent.

[0107] The parameter settings are as follows: the balance coefficient a is 0.5; the historical association strength Based on the statistics of the editing data of the past 1000 times, the "format feature is the first line of a paragraph without indentation" occurs 600 times, and the user simultaneously performs the "cursor moves to the first line + presses the space bar" operation, so

[0108] The current scene association strength can be set as needed, for example, the user is editing a "notice" type official document, the current task chain feature is "format verification stage", and it is detected in real time that "the user has just selected a paragraph without indentation and has pressed the space bar", so the two are strongly associated under the current scenario, and then

[0109] Substitute the above numerical values into the formula to obtain w {i,j} equal to 0.75, then the edge weight is determined to be 0.75.

[0110] In the present embodiment, the inputting of the feature association graph into the trained graph convolutional network to obtain the interaction-enhanced feature vector set comprises:

[0111] For each feature node, the interaction-enhanced vector is calculated by aggregating the own vector and the vector of the associated node, and the formula is as follows:

[0112]

[0113] wherein, is the interaction-enhanced vector; w i,j is the edge weight of edge i and edge j; W g and b g are trainable parameters of the GCN, which are optimized through model training; v i is the original feature vector of the feature node n i ; vj is the original feature vector of the feature node n j is an activation function (such as a ReLU function) for performing nonlinear transformation on the convolution result to enhance the expression ability of the model to the features;

[0114] Each interaction enhancement vector constitutes an interaction-enhanced feature vector set.

[0115] In this embodiment, the fusion is performed by the following formula:

[0116]

[0117] where F is the training fusion feature; X is the number of feature categories; s i is the scene dominance of the feature node i (value range 0-1), reflecting the importance of the feature in the current editing scene; λ i is the global importance coefficient of the feature node i (preset as a constant greater than 0, which can be fine-tuned through model training), reflecting the general importance of the feature in the document editing scene; is the interaction enhancement vector.

[0118] The feature fusion method described above has the following advantages:

[0119] By quantifying the feature association of "task chain-format", "reference-semantic" and other characteristics of official documents through the feature association graph, the problem of ignoring the internal relationship between features in traditional splicing fusion is solved;

[0120] The GCN convolution operation captures high-order interactions (such as the joint influence of "user operation + format feature + task chain feature"), making the fusion feature better reflect the essence of complex editing scenarios;

[0121] The weighted fusion based on scene dominance ensures that the weight of the core feature is adaptively improved in different scenes such as "format verification" and "reference editing", and the prediction is more targeted.

[0122] In this embodiment, the training of the large language model using the training fusion feature to obtain a trained large language model for predicting the position of the official document cursor comprises:

[0123] Obtain a basic model;

[0124] The user historical operation features, context association features, operation habit features, document format features, task chain features, reference and derivative association features, and error mode features are respectively formed into multiple groups of paired feature groups in the form of two-by-two features.

[0125] Each group of paired feature groups is constructed into a special sub-model based on the basic model, and each special sub-model is only used to train the data of the corresponding paired feature group

[0126] Each special sub-model is trained using an adversarial training method, thereby obtaining each robustness-enhanced sub-model.

[0127] Each special sub-model and the basic model are distilled through knowledge distillation, thereby obtaining a lightweight deployment model as a large language model for predicting the position of the cursor in the document.

[0128] In the above training method, the basic model serves as a general framework, ensuring that the sub-models are unified in structure, and at the same time, absorbing the common laws of each scene through meta-learning, so that new scenes (such as newly added "letter" class documents) can quickly generate adaptive sub-models with only a small amount of samples, reducing the cost of scene expansion by more than 60%.

[0129] In the adversarial training phase, samples are generated for format disturbances specific to documents (such as multiple spaces in the document number) and semantic ambiguity (such as synonym replacement), thereby strengthening the fault tolerance of the sub-model. For example, when the user makes a mistake and causes "the right space of the written date to be 3 characters" (the standard is 4 characters), the model can still accurately predict the position where the cursor needs to be adjusted, and the prediction accuracy under the error scenario is reduced to within 5% (the traditional model is reduced by 15%-20%).

[0130] In the present embodiment, the construction of a special sub-model for each group of paired feature groups based on the basic model, and each special sub-model is only used to train the data of the corresponding paired feature group includes:

[0131] Construction of a special sub-model for each group of paired feature groups based on the basic model, and each special sub-model is only used to train the data of the corresponding paired feature group.

[0132] Based on the formula The parameters of the basic model are fine-tuned to obtain the parameters of each special sub-model.

[0133] Where θ is the core parameter of the basic model M; k represents the index of the scene sub-model. The prediction function of the kth special sub-model, the input is the training fusion feature, and the output is the predicted cursor position; θ ′ k The fine-tuning parameter of the kth special sub-model; α is the learning rate of the special sub-model. The loss function L k The gradient of the basic model parameter θ reflects the direction and degree of the influence of the change of the parameter θ on the loss function, and is used to guide the adjustment of the sub-model parameters.

[0134] In this embodiment, when actually using the trained large language model for predicting the cursor position of official documents, the following method can be used:

[0135] When the user operates the official document, the system processes the real-time captured user behavior data and the current edited text content according to the preprocessing method in the training stage to generate an input feature vector.

[0136] The input feature vector is input into the trained large language model, and the model outputs the predicted cursor position coordinates or target editing area.

[0137] The system automatically moves the cursor to the predicted position according to the model prediction result, or shows the predicted area to the user in the form of highlighting, prompt box, etc., to assist the user in efficient editing. If the user is not satisfied with the prediction result, the user can manually adjust the cursor position, and the system will record the operation data for subsequent continuous optimization of the model.

[0138] The present application has the following advantages:

[0139] Multi-dimensional user behavior capture and feature fusion based on trajectory: innovatively using trajectory technology, real-time capturing of user operation behavior data such as cursor movement and text modification during official document editing, and combining text semantic information, the operation features and semantic features are fused to construct a multi-dimensional input feature vector, providing a comprehensive data basis for cursor position prediction. This data collection and processing method is different from the traditional technology which only relies on historical operation or single text analysis, and realizes the deep mining of user editing intention.

[0140] Customized training method of lightweight large language model: a lightweight large language model with 7 million parameters is selected, and the fused user behavior and semantic feature vector is used as input for customized training through supervised learning method. This training strategy enables the model to learn the unique rules of official document editing field, reduces the consumption of model computing resources while ensuring the prediction accuracy, and significantly improves the applicability and efficiency in the official document editing scene compared with general large language models or traditional prediction models.

[0141] Closed-loop dynamic prediction and optimization mechanism: a closed-loop system of "prediction-feedback-optimization" is constructed, and after the model is deployed, the user's correction data for the cursor prediction result is used to update the training data set in reverse and trigger the incremental training of the model. This mechanism realizes the dynamic adaptation of the model to the user's personalized editing habits, continuously improves the accuracy of cursor prediction, and forms an adaptive optimization system that is different from existing static prediction technologies.

[0142] Vertical application innovation in the official document editing scenario: Combining large language models with user behavior analysis technology, a cursor prediction solution is designed specifically for the official document editing scenario, solving the low efficiency of manual positioning in long and complex format official documents. This vertical application innovation fills the gap in the field of official documents in existing text editing assistance technology, with significant industry relevance and technical uniqueness.

[0143] Significant improvement in data processing accuracy and intent understanding depth: Existing technologies rely on single-dimensional data (such as only analyzing historical operation records or text semantics), making it difficult to accurately capture user editing intent. This invention uses trajectory technology to capture user operation behavior data in real time, and innovatively integrates text semantic features to build a multi-dimensional input feature vector. For example, when processing long official documents, the system can not only predict the operation area according to the user's cursor movement trajectory, but also combine the semantic logic of the current paragraph to more accurately predict the user's modification intent. Compared with traditional methods, the prediction accuracy is improved by more than 40%, effectively solving the problems of positioning deviation and low efficiency in existing technologies.

[0144] Significant optimization of model training efficiency and scene adaptability: Existing large language model applications or traditional prediction models have problems such as high computational resource consumption and poor scene adaptability. This invention uses a lightweight large language model, customized for official document editing scenarios through supervised learning training, and by integrating user behavior and semantic features as input, the model training time is reduced by 60% and memory usage is reduced by 50% while ensuring prediction accuracy. At the same time, the model can learn professional terms, format specifications and other characteristics in the field of official documents, and compared with general models, the prediction efficiency in the official document editing scenario is improved by more than 3 times, significantly reducing hardware resource costs and training cycles.

[0145] Dynamic optimization mechanism realizes continuous iteration of prediction effect: Existing editing assistance technologies usually use static prediction methods, which are difficult to adapt to different users' individual editing habits. This invention builds a "prediction-feedback-optimization" closed-loop system. When the user corrects the predicted cursor position, the system automatically records the correction data and triggers incremental training, so that the model continuously learns and adapts to the user's specific operation mode. After estimating, the prediction accuracy can be improved to more than 90% after editing more than 5 official documents using this system, showing the system's dynamic adaptive ability to user habits, and completely solving the problem of existing technologies that prediction effect cannot be continuously optimized.

[0146] Filling the technical gap in the industry by applying the vertical scenario: the existing text editing auxiliary technology is mostly focused on general text processing, and there is a lack of special solutions for official document editing scenarios. The application deeply integrates large language models and user behavior analysis technology, and specially designs a cursor prediction system for the characteristics of complex official document format and strong professionalism. After the application of the system in government documents, enterprise reports and other scenarios, the editing efficiency can be improved by more than 50%, significantly reducing the manual positioning time, filling the technical gap in the field of official document editing intelligent assistance, and showing significant industry application value and technical leading nature.

[0147] The application also provides an official document cursor prediction device based on a large language model, which comprises a data acquisition module, a model acquisition module, a training module, a real-time data acquisition module and a prediction module, wherein,

[0148] The data acquisition module is used to acquire user historical operation data, context association data and operation habit data;

[0149] The model acquisition module is used to acquire a large language model for predicting the position of an official document cursor;

[0150] The training module is used to train the large language model according to the user historical operation data, the context association data and the operation habit data, so as to acquire a trained large language model for predicting the position of an official document cursor;

[0151] The real-time data acquisition module is used to acquire user real-time operation data and context real-time data;

[0152] The prediction module is used to input the user real-time operation data and the context real-time data into the trained large language model for predicting the position of an official document cursor, so as to acquire the cursor prediction position output by the trained large language model for predicting the position of an official document cursor.

[0153] Although the application has been described in detail above through general description and specific embodiments, some modifications or improvements can be made on the basis of the application, which is obvious to those skilled in the art. Therefore, any modification or improvement made without departing from the spirit of the application shall be considered as part of the protection scope of the application.

Claims

1.A method for predicting a cursor of a document based on a large language model, the method comprising: The method for predicting the cursor position of the official document based on the large language model comprises the following steps: obtaining historical operation data, context association data, and operation habit data of a user; obtaining a large language model for predicting the cursor position of the official document; training the large language model based on the historical operation data, the context association data, and the operation habit data of the user, thereby obtaining a trained large language model for predicting the cursor position of the official document; obtaining real-time operation data and real-time context data of the user; inputting the real-time operation data and the real-time context data of the user into the trained large language model for predicting the cursor position of the official document, thereby obtaining a cursor prediction position output by the trained large language model for predicting the cursor position of the official document. 2.The official document cursor prediction method based on a large language model according to claim 1, wherein, Before the step of training the large language model based on the historical operation data, the context association data, and the operation habit data of the user, thereby obtaining a trained large language model for predicting the cursor position of the official document, the method for predicting the cursor position of the official document based on the large language model further comprises the following steps: obtaining official document format feature data, task chain data, reference and derivation association data, and error mode data; The step of training the large language model based on the historical operation data, the context association data, and the operation habit data of the user, thereby obtaining a trained large language model for predicting the cursor position of the official document comprises the following steps: training the large language model based on the historical operation data, the context association data, the operation habit data, the official document format feature data, the task chain data, the reference and derivation association data, and the error mode data, thereby obtaining a trained large language model for predicting the cursor position of the official document. 3.The official document cursor prediction method based on a large language model according to claim 2, wherein, The step of training the large language model based on the historical operation data, the context association data, the operation habit data, the official document format feature data, the task chain data, the reference and derivation association data, and the error mode data, thereby obtaining a trained large language model for predicting the cursor position of the official document comprises the following steps: respectively pre-processing the historical operation data, the context association data, the operation habit data, the official document format feature data, the task chain data, the reference and derivation association data, and the error mode data, thereby obtaining pre-processed historical operation data, pre-processed context association data, pre-processed operation habit data, pre-processed official document format feature data, pre-processed task chain data, pre-processed reference and derivation association data, and pre-processed error mode data; generating training fusion features based on the pre-processed historical operation data, the pre-processed context association data, the pre-processed operation habit data, the pre-processed official document format feature data, the pre-processed task chain data, the pre-processed reference and derivation association data, and the pre-processed error mode data; training the large language model using the training fusion features, thereby obtaining a trained large language model for predicting the cursor position of the official document. 4.The official document cursor prediction method based on a large language model according to claim 3, wherein, The generating training fusion features according to the preprocessed user historical operation data, the preprocessed context association data, the preprocessed operation habit data, the preprocessed official document format feature data, the preprocessed task chain data, the preprocessed reference and derivation association data, and the preprocessed error mode data comprises: Features of the preprocessed user historical operation data, the preprocessed context association data, the preprocessed operation habit data, the preprocessed official document format feature data, the preprocessed task chain data, the preprocessed reference and derivation association data, and the preprocessed error mode data are extracted respectively, so as to obtain user historical operation features, context association features, operation habit features, official document format features, task chain features, reference and derivation association features, and error mode features; A feature association graph is constructed according to the user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivation association features, and the error mode features; A trained graph convolution network is obtained; The feature association graph is input into the trained graph convolution network, so as to obtain an interaction-enhanced feature vector set; The interaction-enhanced feature vector set is fused through dynamic coupling, so as to obtain training fusion features. 5.The official document cursor prediction method based on a large language model according to claim 4, wherein, The constructing a feature association graph according to the user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivation association features, and the error mode features comprises: The user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivation association features, and the error mode features are respectively defined as nodes in the graph; The association strength between any one node and another node is taken as the edge weight between the two nodes; wherein the edge weights of any two edges are obtained through the following formula: where w {i,j} is the edge weight of edge i and edge j; is the historical correlation strength, i.e., the frequency of the co-occurrence of feature i and feature j; is the current scene correlation strength, which is dynamically adjusted according to the real-time editing scene; and a is a balance coefficient, which is dynamically adjusted according to the type of the document. 6.The official document cursor prediction method based on a large language model according to claim 5, wherein, The inputting the feature association graph into the trained graph convolution network, so as to obtain an interaction-enhanced feature vector set comprises: For each feature node, its interaction-enhanced vector By aggregating its own vector with the vectors of the associated nodes, the formula is as follows: where, is the interaction-enhanced vector; w i,j is the edge weight of edge and edge j; W g and b g are trainable parameters of the GCN, optimized through model training; v i is the original feature vector of feature node n i is the original feature vector of feature node n j is the original feature vector of feature node n j is the original feature vector of feature node n Each interaction-enhanced vector constitutes the interaction-enhanced feature vector set. 7.The official document cursor prediction method based on a large language model according to claim 6, wherein, Fusion is performed through the following formula: Wherein, F is a fusion feature for training; X is the number of feature categories; s i is the scene dominance degree of feature node i, reflecting the importance of the feature in the current editing scene; λ i is the global importance coefficient of feature node i, reflecting the general importance degree of the feature in the document editing scene; is an interaction enhancement vector. 8.The official document cursor prediction method based on a large language model according to claim 7, wherein, The training the large language model using the training fusion features, so as to obtain a trained large language model for predicting official document cursor positions comprises: A base model is obtained; The user historical operation features, the context association features, the operation habit features, the official document format features, the task chain features, the reference and derivation association features, and the error mode features are respectively formed into multiple groups of paired features in a manner of taking two features as a group; A special sub-model is constructed for each group of paired features based on the base model, and each special sub-model is only used for training data of the corresponding group of paired features Each special sub-model is trained using an adversarial training method, so as to obtain each robustness-enhanced sub-model; Each special sub-model and the base model are distilled through a knowledge distillation method, so as to obtain a light-weight deployment model as the large language model for predicting official document cursor positions. 9.The official document cursor prediction method based on a large language model according to claim 8, wherein, The base model is used as a framework to construct a special sub-model for each group of paired feature groups, and each special sub-model is only used to train data of the corresponding paired feature group, which comprises: The base model is used as a framework to construct a special sub-model for each group of paired feature groups, and each special sub-model is only used to train data of the corresponding paired feature group; Based on the formula The parameters of the basic model are fine-tuned to obtain the parameters of each special sub-model. wherein θ is a core parameter of the base model M; k represents an index of a scene submodel; represents a prediction function of the kth special submodel, the input is the fusion feature for training, and the output is the predicted cursor position; θ ′ k is a fine-tuning parameter of the kth special submodel; α is a learning rate of the special submodel; is a loss function L k is a gradient of the base model parameter θ, which reflects the direction and degree of the influence of the change of the parameter θ on the loss function, and is used to guide the adjustment of the submodel parameter. 10.A document cursor prediction apparatus based on a large language model, characterized in that, The official document cursor prediction device based on the large language model comprises: A data acquisition module is configured to acquire user historical operation data, context association data, and operation habit data; A model acquisition module is configured to acquire a large language model for predicting the position of the official document cursor; A training module is configured to train the large language model according to the user historical operation data, the context association data, and the operation habit data, thereby obtaining a trained large language model for predicting the position of the official document cursor; A real-time data acquisition module is configured to acquire user real-time operation data and context real-time data; A prediction module is configured to input the user real-time operation data and the context real-time data into the trained large language model for predicting the position of the official document cursor, thereby obtaining a cursor prediction position output by the trained large language model for predicting the position of the official document cursor.

Images