A method and system for transforming security forces based on a large model

Abstract

This application belongs to the field of local resource and capacity support technology. A method for transforming support capabilities based on a large model is disclosed. The method includes: acquiring local resource and capacity support data to obtain a professional domain dataset; pre-training a ChatGLM-6B model using the professional domain dataset to obtain a pre-trained model; supervising the pre-trained model according to preset prompts and answer samples to obtain a supervised training model; fine-tuning the RLHF parameters of the supervised training model to obtain an RLHF training model; deploying the RLHF training model at the edge and integrating it with LangChain to obtain an RLHF interactive model; and using an AI-agent combined with the RLHF interactive model to perform support capability transformation. Driven by support needs, this method helps local resource and capacity to be agilely empowered and precisely released in current conflict scenarios, thereby enhancing local support capabilities.

Description

Technical Field

[0001] This application belongs to the field of local resource and strength support technology, and in particular relates to a method and system for support strength transformation based on a large model. Background Technology

[0002] Large-scale artificial intelligence models refer to machine learning models with extremely large parameters and powerful computing resources, capable of processing massive amounts of data and completing various complex tasks. With the launch of ChatGPT by OpenAI in 2022, various large-scale models have sprung up like mushrooms after rain both domestically and internationally.

[0003] In response to the need for local resource and force support in conflict scenarios, it is urgent to address the existing problems in the organization and utilization of resources and forces, and to conduct innovative research on the perception and utilization of local resources and forces. Guided by the needs of support, this research will help local resources and forces to be agilely empowered and precisely released in the current conflict scenarios, thereby enhancing local support capabilities. Summary of the Invention

[0004] Therefore, it is necessary to provide a method and system for the transformation of security forces based on a large model to address the aforementioned technical problems.

[0005] Firstly, this application provides a method for transforming security forces based on a large model, the method comprising:

[0006] Obtain data on local resource and capacity support to create datasets for specialized fields;

[0007] The ChatGLM-6B model was pre-trained using the aforementioned professional domain dataset to obtain a pre-trained model.

[0008] Based on preset prompts and answer samples, the pre-trained model is adjusted under supervision to obtain a supervised training model;

[0009] The supervised training model is fine-tuned by RLHF parameters to obtain the RLHF training model;

[0010] The RLHF training model is deployed at the edge and integrated with LangChain to obtain the RLHF interactive model;

[0011] By utilizing AI-agent and combining it with the RLHF interaction model, the transformation of security forces can be carried out.

[0012] In some feasible methods, the step of pre-training the ChatGLM-6B model using the domain-specific dataset to obtain a pre-trained model includes:

[0013] The data in the aforementioned professional domain dataset is divided into structured data, semi-structured data, and unstructured data.

[0014] The semi-structured data is transformed and combined with the structured data to obtain the first initial corpus;

[0015] The unstructured data is processed using regular expressions and natural language processing tools to obtain a second initial corpus;

[0016] According to a preset division ratio, the target corpus is divided into two parts to obtain a training set and a validation set. The target corpus includes a first initial corpus and a second initial corpus, both of which are text.

[0017] The data in the training set and the validation set are segmented respectively to obtain the paragraphs or sentences corresponding to the text;

[0018] Using a word segmenter, the paragraphs or sentences corresponding to the text are segmented to obtain the words or sub-words corresponding to the paragraphs or sentences;

[0019] The word or sub-word is mapped to an integer encoding to obtain the integer value corresponding to the word or sub-word;

[0020] Input the integer values ​​corresponding to the training set into the initial word segmentation model to obtain the intermediate word segmentation model;

[0021] The integer value corresponding to the validation set is input into the intermediate word segmentation model, and the parameters of the intermediate word segmentation model are adjusted according to the output of the intermediate word segmentation model to obtain the target word segmentation model.

[0022] The vocabulary in the target word segmentation model is merged with the original ChatGLM-6B word segmentation model to obtain a new word segmentation model;

[0023] The ChatGLM-6B model was trained using the new word segmentation model to obtain a pre-trained model.

[0024] In some feasible methods, the step of supervising and adjusting the pre-trained model based on preset prompts and answer samples to obtain a supervised training model includes:

[0025] Construct a preset training set by obtaining several preset samples with hints and answers;

[0026] In the preset training set, the prompts and answers of each preset sample are associated to obtain a continuous input sequence;

[0027] Using tags, the hints and answers in the preset samples are segmented to obtain the hint part and the answer part;

[0028] By using an autoregressive objective function, the loss from the prompt portion corresponding to the user prompt is reduced to zero, and the backpropagation of the answer portion is obtained.

[0029] Based on the backpropagation of the answer part, the parameters of the adjustable model part are adjusted, while the parameters of the frozen model part remain unchanged, to obtain the supervised training model. The adjustable model part and the frozen model part together constitute the pre-trained model.

[0030] In some feasible methods, the step of fine-tuning the supervised training model using RLHF to obtain an RLHF training model includes:

[0031] Inputting prompts into the supervised training model yields multiple candidate answers output by the supervised training model;

[0032] The multiple candidate answers output by the supervised training model are sorted and scored to obtain a score for each candidate answer;

[0033] Feature extraction is performed on the candidate answers to obtain candidate answer features;

[0034] The input prompts and the features of the candidate answers are used as inputs, and the scores of the candidate answers are used as labels to form reward samples. Based on multiple reward samples, a reward training set is obtained.

[0035] Using the aforementioned reward training set, an initial reward model is trained to obtain a target reward model;

[0036] Using the target reward model, the supervised training model is fine-tuned with RLHF parameters to obtain the RLHF training model.

[0037] In some feasible approaches, the step of deploying the RLHF trained model at the edge and integrating it with LangChain to obtain the RLHF interactive model includes:

[0038] An index is created for the professional domain dataset, wherein the index represents the text segmentation in the professional domain dataset to form a document block vector;

[0039] Obtain user input prompts and convert the input prompts into vectors to obtain the input vector;

[0040] The input vector is compared with the document block vectors in the professional domain dataset to obtain the K document blocks with the highest similarity. The K document blocks with the highest similarity are used as the enhanced context information of the input prompt.

[0041] The K document blocks with the highest similarity are merged with the input prompt to obtain the prompt for inputting the RLHF training model, forming an edge deployment of the RLHF training model.

[0042] In some feasible approaches, the step of deploying the RLHF trained model at the edge and integrating it with LangChain to obtain the RLHF interactive model includes:

[0043] Import the LangChain library into the interactive environment formed by the user and the RLHF training model;

[0044] Initialize LangChain and create a LangChain instance connected to the RLHF training model to obtain the RLHF interactive model.

[0045] In some feasible approaches, the step of utilizing AI-agent in conjunction with the RLHF interaction model to perform the security force conversion includes:

[0046] Obtain assurance requirement tasks, wherein the assurance requirement tasks include different categories of assurance tasks;

[0047] Based on the protection category, the protection requirement task is decomposed using AI-agent to obtain multiple protection category sub-tasks;

[0048] Multiple sub-tasks of the protection category are input into the RLHF interaction model and executed respectively to obtain the output result of each sub-task of the protection category.

[0049] Based on the output results of multiple sub-tasks of the aforementioned protection categories, the protection force transformation that addresses actual needs and provides protection prompts is obtained.

[0050] Secondly, this application provides a large-scale model-based security force conversion system, applied to the aforementioned large-scale model-based security force conversion method, the system comprising:

[0051] The acquisition unit is used to acquire local resource and capacity support data to obtain datasets for specific fields.

[0052] The training unit is used to pre-train the ChatGLM-6B model using the professional domain dataset to obtain a pre-trained model.

[0053] The supervisory unit is used to supervise and adjust the pre-trained model based on preset prompts and answer samples to obtain a supervised training model;

[0054] The fine-tuning unit is used to fine-tune the RLHF parameters of the supervised training model to obtain the RLHF training model.

[0055] The interaction unit is used to deploy the RLHF training model at the edge and integrate it with LangChain to obtain the RLHF interaction model.

[0056] The result unit is used to combine the AI-agent with the RLHF interaction model to perform security force conversion.

[0057] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the aforementioned large-model-based security force conversion method.

[0058] Fourthly, this application provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned large-model-based security force conversion method.

[0059] Beneficial effects:

[0060] This application provides a method for transforming support capabilities based on a large model. It acquires local resource support data to obtain a professional domain dataset; uses this dataset to pre-train a ChatGLM-6B model to obtain a pre-trained model; performs supervised adjustment on the pre-trained model based on preset prompts and answer samples to obtain a supervised training model; fine-tunes the RLHF parameters of the supervised training model to obtain an RLHF training model; deploys the RLHF training model at the edge and integrates it with LangChain to obtain an RLHF interactive model; and utilizes an AI-agent combined with the RLHF interactive model to perform support capability transformation. The method uses ChatGLM-6B with a 1:1 Chinese-English ratio as the basic training model. It addresses the need for local resource support in conflict scenarios, solving the problem of poor support capabilities in the current organization and utilization of resources. Driven by support needs, it helps local resources to be agilely empowered and precisely released in current conflict scenarios, thereby improving local support capabilities. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart of a large-model-based security force conversion method in one embodiment.

[0063] Figure 2This is a flowchart illustrating the process of obtaining a pre-trained model for a large-model-based security force conversion method in one embodiment.

[0064] Figure 3 This is a flowchart illustrating the process of obtaining a supervised training model for a large-model-based security force conversion method in one embodiment.

[0065] Figure 4 This is a flowchart illustrating the process of obtaining the RLHF training model for a large-scale model-based security force conversion method in one embodiment.

[0066] Figure 5 This is a flowchart illustrating the RLHF interactive model obtained from a large-model-based security force conversion method in one embodiment. Detailed Implementation

[0067] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all couplings of one or more of the associated listed items.

[0069] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another.

[0070] The following explanations of some terms used in this application are provided to aid in understanding the application:

[0071] In the fields of Natural Language Processing (NLP) and Machine Learning, a token typically refers to the basic unit of text data.

[0072] A tokenizer is a fundamental tool in Natural Language Processing (NLP). Its main function is to divide a continuous text string into smaller units, which can be words, phrases, subwords, or symbols.

[0073] A subword is a concept in natural language processing used to represent smaller linguistic units that make up a word.

[0074] An autoregressive objective function is a loss function used in generative models. Its core idea is to predict each element (such as a word or character) in a sequence step by step and complete the task of generating the entire sequence in a step-by-step manner.

[0075] The backpropagation algorithm is an optimization algorithm used to train artificial neural networks. It calculates the gradient of each layer in the network and updates the weights using gradient descent to minimize the loss function.

[0076] RLHF (Reinforcement Learning from Human Feedback) fine-tuning is a model optimization method that combines human feedback and reinforcement learning techniques. It's a model training technique that utilizes feedback from human evaluators to guide the model's learning process. In the learning process, reinforcement learning is a method that allows an agent to learn optimal behavioral strategies through interaction with its environment.

[0077] AI-agent refers to an artificial intelligence agent, which is a software program that uses artificial intelligence technology to perform automated tasks, make decisions, and interact with users or other systems.

[0078] SentencePiece is an open-source word segmentation library developed by Google for text segmentation in natural language processing tasks. It supports segmentation of multiple languages ​​and is particularly suitable for languages ​​without obvious word separators, such as Chinese, Japanese, and Korean.

[0079] like Figure 1 As shown, the first aspect of this application provides a method for transforming security forces based on a large model, the method comprising:

[0080] S100 obtains local resource and strength support data to obtain professional domain datasets.

[0081] Local resource and strength support data refers to data related to resource and strength support for a specific region or organization. It usually comes from government departments, social organizations, corporate records and historical archives. This data involves different professional fields. Data from each field is aggregated to form different professional field datasets.

[0082] S200, Using the aforementioned professional domain dataset, pre-train the ChatGLM-6B model to obtain a pre-trained model.

[0083] like Figure 2As shown, specifically, obtaining the pre-trained model includes the following steps:

[0084] S201, the data in the dataset of the professional field is divided into structured data, semi-structured data and unstructured data.

[0085] The large model mentioned in this application is a language model, which is a model formed by reasoning from text. Since the text input to the language model is a string, and strings cannot be recognized by the language model, it is necessary to convert the text into numbers. Thus, the data in the professional domain dataset is first structurally divided, and then converted into numbers. The professional domain data is diverse and heterogeneous, including structured and semi-structured data such as database tables, tables, and text, as well as unstructured data such as images and audio. These different types of data form a corpus, i.e., the professional domain dataset. The corpus mainly contains data on needs and support in different scenarios. Different scenarios result in different needs, support capabilities, and methods. The basis for structurally dividing the data in the professional domain dataset usually includes relatively fixed structures such as the data organization and storage methods. After defining the rules for structural division, data that conforms to the rules is considered structured data, data that partially conforms to the rules is considered semi-structured data, and data that does not follow a fixed format is considered unstructured data, such as images and videos.

[0086] S202, the semi-structured data is transformed and combined with the structured data to obtain the first initial corpus.

[0087] In the preceding steps, after obtaining the semi-structured data, it is necessary to convert it into structured data to facilitate subsequent data processing. This conversion can be achieved through conventional methods such as using regular expressions to extract specific patterns and data fragments from the semi-structured text, then organizing these fragments into a structured format; segmenting the text, identifying the boundaries of data records, and dividing them into individual records; and applying NLP techniques, such as entity recognition and relation extraction, to identify structured information in the text. This application does not limit the specific methods used in this regard. Next, the semi-structured data and the structured data are combined to obtain the first initial corpus.

[0088] S203, using regular expressions and natural language processing tools, the unstructured data is processed to obtain a second initial corpus.

[0089] Specifically, after obtaining unstructured data in the aforementioned steps, the unstructured data is processed using conventional regular expressions and natural language processing tools. In addition, when unstructured data includes images, videos, or audio, conventional feature extraction methods are needed to extract features from the images, videos, or audio. Next, the extracted features are processed using regular expressions and natural language processing tools to transform the unstructured data into structured data, thereby obtaining a second initial corpus composed of structured data.

[0090] It should be noted that the first initial corpus and the second initial corpus were obtained in the aforementioned steps. The Chinese in the first initial corpus and the second initial corpus can be divided into common Chinese words and rare words. Common words are divided into one token, and rare words can be divided into 2-3 tokens as needed so that ChatGLM-6B can be trained.

[0091] S204, according to the preset division ratio, the target corpus is divided into two parts to obtain a training set and a validation set, wherein the target corpus includes the first initial corpus and the second initial corpus, both of which are text.

[0092] Specifically, the target corpus is divided into a training set and a validation set according to a preset division ratio, such as an 8:2 ratio. For example, the ratio of training set to validation set is 8:2.

[0093] S205, the data in the training set and the validation set are segmented respectively to obtain the paragraphs or sentences corresponding to the text.

[0094] Specifically, splitting the data in the training and validation sets can include the following steps:

[0095] Construct text segmentation rules;

[0096] According to the text segmentation rules, the data in the training set and the validation set are segmented to obtain the corresponding paragraphs or sentences of the text.

[0097] The text segmentation rules can be adjusted as needed. For example, text segmentation rules may include one or more methods such as punctuation rules, paragraph separators, text structure, natural language processing tools, and contextual information to achieve paragraph or sentence segmentation. It should be noted that the above text segmentation methods are only illustrative examples, and other segmentation methods, such as machine learning models and rule engines, can be used as needed.

[0098] Punctuation rules: Use periods (.), question marks (?), exclamation marks (!) as the basis for sentence segmentation;

[0099] Paragraph separators identify the separators between paragraphs, such as blank lines or paragraph marks, and divide the text into paragraphs;

[0100] Text structure uses structural elements in the text, such as headings, subheadings, and lists, to determine the boundaries of paragraphs;

[0101] Natural language processing tools use text segmentation tools from NLP libraries, such as spaCy, NLTK, or Stanford NLP. Tools such as spaCy, NLTK, or Stanford NLP include sentence boundary detection capabilities.

[0102] Contextual information should be considered to ensure that text segmentation does not disrupt the coherence and readability of the text.

[0103] S206, Using a word segmenter, the paragraphs or sentences corresponding to the text are segmented to obtain the words or sub-words corresponding to the paragraphs or sentences.

[0104] Specifically, after obtaining the corresponding paragraphs or sentences in the aforementioned steps, a conventional word segmenter is used to further segment the corresponding paragraphs or sentences into smaller units, such as words or sub-words, so that the ChatGLM-6B model can be trained using these words or sub-words in subsequent steps.

[0105] S207, map the word or sub-word to an integer code to obtain the integer value corresponding to the word or sub-word.

[0106] Specifically, after obtaining words or sub-words in the aforementioned steps, the process of mapping words or sub-words to integer codes can be understood as vocabulary construction or vocabulary mapping. That is, first, a vocabulary is constructed, and each entry in the vocabulary is assigned a unique integer number. Next, words or sub-words are mapped to each entry, so that each word or sub-word has a unique integer code. In this way, if described using a dictionary, the key is the entry, and the value is the integer code, simply referred to as the integer value. It can be understood that after the above processing, each word or sub-word possesses a unique integer value. When a word or sub-word is input into the corresponding model, it will be input in the form of an integer value so that the model can recognize it.

[0107] It should be noted that after a word or subword has a unique integer code, the words or subwords in the validation set and the training set will be replaced with integer values.

[0108] S208, Input the integer values ​​corresponding to the training set into the initial word segmentation model to obtain the intermediate word segmentation model.

[0109] Specifically, after obtaining the integer values ​​corresponding to the training set in the aforementioned steps, the integer values ​​of the training set are input into the initial word segmentation model to train the initial word segmentation model. After the training is completed, the intermediate word segmentation model can be obtained.

[0110] It should be noted that the purpose of the intermediate word segmentation model is to build a model capable of segmenting text into words or sub-words. These segmented words or sub-words can then be further mapped to integer codes for pre-training of the word segmentation model. Specifically, the purposes of training the word segmentation model include the following aspects:

[0111] The training of the word segmentation model helps generate a vocabulary (including words or sub-words) that is tailored to a specific corpus (training text). The vocabulary can better capture the linguistic features in the corpus, thereby improving the adaptability of the pre-trained model to that corpus.

[0112] By segmenting text into appropriate words or subwords using a word segmentation model, the size of the vocabulary can be reduced while retaining sufficient semantic information. This segmentation method allows the model to be more flexible in handling unknown words or variant words, thereby improving the model's generalization ability.

[0113] Training the word segmentation model ensures consistent word segmentation when processing new text, thus guaranteeing consistent encoding of the RLHF interactive model input during pre-training and inference phases. This is crucial for maintaining the predictive accuracy and stability of the RLHF interactive model.

[0114] S209, input the integer value corresponding to the validation set into the intermediate word segmentation model, and adjust the parameters of the intermediate word segmentation model according to the output of the intermediate word segmentation model to obtain the target word segmentation model.

[0115] Specifically, after obtaining the intermediate word segmentation model in the aforementioned steps, it is necessary to verify the intermediate word segmentation model. At this time, the integer value corresponding to the verification set is input into the intermediate word segmentation model to obtain the output result of the intermediate word segmentation model. The accuracy of the intermediate word segmentation model is judged by the output result. When there are problems such as errors in the output result, the parameters of the intermediate word segmentation model are adjusted so that the target word segmentation model is finally formed after the parameter adjustment.

[0116] S210, the vocabulary in the target word segmentation model is merged with the original ChatGLM-6B word segmentation model to obtain a new word segmentation model.

[0117] Specifically, after obtaining the vocabulary from the target word segmentation model, it is necessary to merge the vocabulary with the original ChatGLM-6B word segmentation model. It should be noted that during the merging process, if duplicate terms exist, they are deleted, and only one term is retained to avoid duplication. Next, the vocabulary in the new word segmentation model is re-encoded and indexed to ensure that each term in the new model has a unique integer value.

[0118] S211, Using the new word segmentation model, the ChatGLM-6B model is trained to obtain a pre-trained model.

[0119] Specifically, the purpose of training the ChatGLM-6B model using the new word segmentation model is to enable the ChatGLM-6B model to better adapt to the new vocabulary, thereby improving its performance in security tasks. This retraining aims to achieve the following:

[0120] 1. Adapt to the new vocabulary list

[0121] Vocabulary expansion: The merged vocabulary typically includes new words or sub-vocabularies that may not have been present in the original ChatGLM-6B model's vocabulary. Retraining the model allows it to learn how to effectively use these new words, thus performing better when handling new tasks or domains.

[0122] Coding consistency: After retraining, the pre-trained model will relearn the mapping relationship between the text and the new vocabulary, thereby ensuring that the encoding of the text data matches the input of the model when using the new vocabulary.

[0123] 2. Improve the performance of pre-trained models in specific domains.

[0124] Domain adaptability: If the new vocabulary comes from data in a specific domain (e.g., the professional field of security forces), retraining can help the model better grasp the linguistic features of that domain, making its performance in related tasks more accurate and efficient.

[0125] Integrating new knowledge: Through pre-training, pre-trained models can absorb new language patterns, terms, and expressions, which helps to enhance the understanding and generation capabilities of pre-trained models in specific application scenarios.

[0126] 3. Optimize the generation quality of pre-trained models

[0127] Naturalness of generated text: New vocabularies may contain words or expressions that are more relevant to the target application. By retraining, pre-trained models can better utilize these new words, thereby improving the naturalness and relevance of the generated text.

[0128] 5. Enable customization of pre-trained models

[0129] Customized pre-trained model: Through pre-training, ChatGLM-6B can be customized to better suit support missions.

[0130] S300, based on preset prompts and answer samples, performs supervised adjustments on the pre-trained model to obtain a supervised training model.

[0131] like Figure 3 As shown, specifically, the steps to obtain a supervised training model include:

[0132] S301, construct and obtain several preset samples with hints and answers to obtain a preset training set.

[0133] Specifically, a pre-defined training set is constructed, containing several samples with explicit prompts and corresponding answers. This ensures that, even with data diversity and varying quality, high-quality SFT data examples are utilized, including the quality of the output results. It should be noted that the prompt can correspond to one or more answers.

[0134] S302, in the preset training set, the prompts and answers of each preset sample are associated to obtain a continuous input sequence.

[0135] Specifically, after obtaining the pre-set training set in the aforementioned steps, it is necessary to associate the prompts and answers for each pre-set sample in the training set to form a continuous input sequence, enabling the pre-trained model to receive complete contextual information. When generating the answer, the content of the prompt can be directly referenced, thereby improving the accuracy and relevance of the generated answer. Furthermore, the input sequence serves as the initial input, allowing the pre-trained model to progressively generate the answer content through autoregression. The prompt portion of the input sequence provides the conditions for generating the answer, enabling the pre-trained model to make predictions based on the prompts at each generation step. Moreover, when the prompts and answers are explicitly associated, the pre-trained model can better understand which part is the prompt and which part is the answer to be generated, reducing the possibility of ambiguity.

[0136] S303, using markers, the hints and answers in the preset sample are segmented to obtain the hint part and the answer part.

[0137] Specifically, to better segment related prompts and answers, tokens, such as [SEP], are used to separate the prompt and answer segments. This tokenization will result in a sample being segmented into a prompt portion and an answer portion. It is crucial to ensure that the pre-trained model correctly understands the structure of the input; that is, the pre-trained model can clearly distinguish between the part to be predicted (i.e., the answer portion) and the contextual information provided by the user (i.e., the prompt portion).

[0138] S304, using an autoregressive objective function, the loss from the prompt portion corresponding to the user prompt is reduced to zero, and the backpropagation of the answer portion is obtained.

[0139] Specifically, the autoregressive objective function progressively predicts the corresponding answer based on the given hints. In its implementation, the loss from user-provided tokens is set to zero, meaning these tokens do not affect the calculation of the loss function. The pre-trained model only needs to focus on learning how to generate the correct answer. Thus, in each training iteration, the pre-trained model calculates the loss for generating the answer and propagates this loss back to the parameters of the adjustable model via backpropagation. During this process, the pre-trained model gradually adjusts its parameters to minimize the loss, thereby improving the accuracy and quality of answer generation.

[0140] S305, based on the backpropagation of the answer part, the parameters of the adjustable model part are adjusted, while the parameters of the frozen model part remain unchanged, to obtain a supervised training model, wherein the adjustable model part and the frozen model part constitute the pre-trained model.

[0141] Specifically, the pre-trained model consists of two parts: an adjustable model part and a frozen model part. The adjustable model part refers to the model parameters that can be adjusted and optimized during training. Typically, this includes a subset of layers or certain parameters in the pre-trained model, which are updated during training to minimize the loss function. The frozen model part refers to the pre-trained model parameters that are not updated during training. These parameters remain unchanged during training and are not affected by gradients. Freezing the parameters helps maintain the capabilities of the pre-trained model, avoiding overfitting or the destruction of existing knowledge during training on new tasks.

[0142] S400, Fine-tune the RLHF parameters of the supervised training model to obtain the RLHF training model.

[0143] like Figure 4 As shown, specifically, the steps to obtain the RLHF training model include:

[0144] S401, Input prompts into the supervised training model to obtain multiple candidate answers output by the supervised training model.

[0145] Specifically, after inputting a prompt into the supervised training model, the model generates multiple candidate answers based on the prompt. These candidate answers may have different qualities and relevance. Therefore, the multiple candidate answers need to be filtered in subsequent steps.

[0146] S402, sort and score the multiple candidate answers output by the supervised training model to obtain a score for each candidate answer.

[0147] Specifically, a scoring mechanism is constructed, which scores candidate answers based on accuracy, fluency, and relevance, among other factors. It should be noted that the scoring mechanism can be set according to actual circumstances, and this application does not impose any limitations on it. After obtaining the scoring mechanism, each candidate answer is scored according to the scoring mechanism, and the score represents the candidate answer's accuracy, fluency, and relevance, among other factors.

[0148] S403, perform feature extraction on the candidate answer to obtain candidate answer features.

[0149] Specifically, feature extraction is performed on candidate answers. The feature extraction method used can be a conventional convolutional neural network or a conventional recurrent neural network; this application does not limit the method used for feature extraction. The features of candidate answers can include one or more of linguistic features, semantic features, or statistical features. Furthermore, the extracted features are summarized into a feature vector (word embedding vector), which represents the attributes and quality of the candidate answers.

[0150] It should be noted that step S403 can be adjusted to occur before S402. That is, feature extraction is performed on the candidate answers first, and then multiple candidate answers are scored.

[0151] S404, take the input prompt and the candidate answer features as input, and the score of the candidate answer as a label to form a reward sample, and obtain a reward training set based on multiple reward samples.

[0152] Specifically, the input prompt is converted into a word embedding vector. For example, using methods like Word2Vec and GloVe, context-aware word vector generation methods (such as BERT and GPT) can be used to convert the entire input prompt into a context-dependent vector representation. This method considers the different meanings of words in different contexts. When the candidate answer features are also word embedding vectors, the word embedding vectors of the input prompt and the candidate answer are concatenated to form a comprehensive input vector. This comprehensive input vector then has a corresponding score, resulting in a reward sample. Next, multiple reward samples are aggregated to form a reward training set. This reward training set is used to train the reward model to learn the mapping relationship from prompt and answer features to score labels.

[0153] S405, Using the aforementioned reward training set, train the initial reward model to obtain the target reward model.

[0154] Specifically, after obtaining the reward training set, an initial reward model is trained using the reward training set. After the initial reward model is trained, it is converted into a target reward model. The purpose of the target reward model is to simulate human scoring of text, and to predict which human-labeled score is higher.

[0155] S406, Using the target reward model, fine-tune the RLHF parameters of the supervised training model to obtain the RLHF training model.

[0156] For example, a series of questions or prompts are input into the supervised training model, causing it to generate multiple candidate answers. Next, a target reward model is used to score each candidate answer, evaluating their performance in terms of accuracy, fluency, and relevance. Feedback information about the candidate answers is collected based on the target reward model's scores; this information serves as the basis for fine-tuning the supervised training model. Then, the reward model's scores are converted into reward signals to guide parameter updates in the supervised training model. The parameters of the supervised training model are continuously updated through multiple iterations, adjusted based on new feedback in each iteration, to improve the model's ability to generate high-quality answers, ultimately resulting in the RLHF training model.

[0157] The S500 deploys the RLHF training model at the edge and integrates it with LangChain to obtain the RLHF interactive model.

[0158] like Figure 5 As shown, specifically, the steps to obtain the RLHF interaction model include:

[0159] S501, establish an index for the professional domain dataset, wherein the index represents the text segmentation in the professional domain dataset to form a document block vector.

[0160] Specifically, indexing can significantly improve the efficiency of retrieving information from documents, especially when dealing with large amounts of data. This involves converting document blocks into vectors (using techniques such as word embedding and sentence embedding) and creating vector indexes (e.g., using libraries like Annoy and Faiss) to accelerate similarity searches.

[0161] It should be noted that indexing is typically performed offline, including data cleaning and extraction, converting different file formats (such as PDF, HTML, Word, etc.) to plain text, then segmenting the text and creating the index. The indexing method is conventional, and this application does not specify how the index is created. Offline indexing ensures data consistency and stability. The data does not change during the index building process, thus ensuring the accuracy and completeness of the index. Offline indexing optimizes the use of computing resources, ensures data consistency, reduces system load, provides processing flexibility, and facilitates data preprocessing and system maintenance.

[0162] S502, obtain the user's input prompt and convert the input prompt into a vector to obtain the input vector.

[0163] Specifically, after receiving input prompts from the user, the user's input prompts (a piece of text) can be segmented into words or sub-words.

[0164] Each word or subword after segmentation is converted into a corresponding word embedding vector.

[0165] The embedding vectors of all words or subwords are concatenated to form a long vector, which is the input vector; the long vector contains all the information of the input prompt.

[0166] The above method can be used to convert input prompts into input vectors.

[0167] S503, compare the similarity between the input vector and the document block vector in the professional domain dataset to obtain the K document blocks with the highest similarity. The K document blocks with the highest similarity are used as the enhanced context information of the input prompt. K is a positive integer and can be adjusted according to specific application requirements.

[0168] Specifically, the input vector and document block vectors are compared using a vector similarity comparison method, such as cosine similarity. After the comparison, the K most similar document blocks are obtained. These K most similar document blocks are used as contextual information and combined with the user's input prompt to form a new prompt or context. The enhanced contextual information can provide more background information, helping RLHF train the model to generate more accurate and relevant answers.

[0169] S504, merge the K document blocks with the highest similarity with the input prompt to obtain the prompt for inputting the RLHF training model, forming an edge deployment of the RLHF training model.

[0170] Specifically, the user's input prompts, after being merged with K document blocks, can form richer prompt information. This prompt information can be more easily understood by the RLHF training model. Therefore, the K document blocks with the highest similarity are merged with the input prompts and used as input prompts for the RLHF training model, so that the RLHF training model can provide more effective answers or suggestions in practical applications.

[0171] It's important to clarify that edge deployment of the RLHF training model refers to deploying the RLHF training model at the edge of the network, i.e., close to the data source or user, rather than on a centralized cloud server. This reduces the distance data travels across the network, thereby lowering response times and providing faster service. It also better protects user privacy because sensitive data doesn't need to be transmitted to a remote server. Furthermore, it improves system reliability, allowing edge devices to operate independently even if cloud services are unavailable.

[0172] In some examples, the steps to obtain the RLHF interaction model also include:

[0173] S505, import the LangChain library into the interactive environment formed by the user and the RLHF training model;

[0174] S506, initialize LangChain and create a LangChain instance connected to the RLHF training model to obtain the RLHF interactive model.

[0175] Specifically, LangChain is a highly efficient framework designed to simplify and optimize interaction with large models. Leveraging LangChain's rich tools and interfaces, the RLHF fine-tuning model described above can be integrated into applications. This allows for the creation of a chat window for dialogue in different scenarios. Furthermore, when LangChain and the RLHF fine-tuning model are combined, complex data analysis of local support capabilities can be performed, generating easily understandable reports or visualizations.

[0176] S600 utilizes AI-agent, combined with the RLHF interaction model, to perform protection force conversion.

[0177] Specifically, S600 includes the following steps:

[0178] S601, Obtain assurance requirement tasks, wherein the assurance requirement tasks include different categories of assurance tasks.

[0179] S602, based on the protection category, the protection requirement task is decomposed using AI-agent to obtain multiple protection category sub-tasks.

[0180] S603, input multiple protection category subtasks into the RLHF interaction model, execute them respectively, and obtain the output result of each protection category subtask.

[0181] S604, based on the output results of multiple sub-tasks of the aforementioned protection categories, obtain the protection force conversion to address actual needs and provide protection prompts.

[0182] Specifically, the AI-agent first acquires a task containing multiple different categories of assurance requirements. Next, based on the assurance categories of these tasks, the AI-agent decomposes the complex assurance requirements into several more manageable and executable sub-tasks. Each sub-task is then input into the RLHF interactive model for processing, which generates corresponding outputs for each sub-task. Finally, combining these outputs, the AI-agent performs comprehensive analysis and transformation to form a solution capable of addressing actual assurance needs, thereby achieving effective deployment of assurance resources and optimized allocation of resources. This process not only improves the efficiency and accuracy of task processing but also enhances the responsiveness to complex assurance requirements through intelligent decomposition and integration.

[0183] Example

[0184] A method for transforming security capabilities based on a large model includes the following steps:

[0185] 1. Collect local resource and capacity support data to form specialized datasets.

[0186] Language models reason from text. Since text is a string, but for large models, the input can only be numbers, it's necessary to convert the text into numbers for representation. First, domain knowledge needs to be collected to form a professional domain Chinese corpus as a training dataset. The corpus collected in this application mainly consists of demand and support data under different scenarios. Different scenarios result in different demand categories, support capabilities, and methods. The professional domain data involved is diverse and heterogeneous, including structured and semi-structured data such as database tables, tables, and text, as well as unstructured data such as images and audio, forming the corpus.

[0187] 2. Using a domain-specific dataset, pre-train the ChatGLM-6B model to obtain the pre-trained model.

[0188] The corpus of this invention includes both structured and unstructured data.

[0189] (1) For the structured data collected in the corpus (training text), this application uses SentencePiece to train its own word segmentation model. First, the data needs to be divided into two parts: a training set and a validation set, with 80% of the data serving as the training set and the remaining 20% ​​as the validation set. Second, the data in the training set is classified, cleaned, and segmented into paragraphs or sentences. A word segmenter is used to segment the text into words or sub-words, and these words are then mapped to integer codes for pre-training. After the word segmentation model is trained, the validation set can be used to test the effectiveness of the word segmentation model.

[0190] (2) For unstructured data, regular expressions and natural language processing tools can be used for processing.

[0191] After testing, the trained word segmentation model is merged with the original ChatGLM-6B word segmentation model to form a new word segmentation model within the domain. The merged new word segmentation model is then used to pre-train the ChatGLM-6B model to obtain a pre-trained model.

[0192] 3. Perform supervised fine-tuning on the pre-trained model to obtain the supervised fine-tuned model.

[0193] Due to the diversity and varying quality of the data, several thousand high-quality SFT data examples are initially collected. During fine-tuning, each sample includes a hint and an answer; in this application, the hint represents the guarantee requirement, and the answer represents the resource guarantee method corresponding to the guarantee requirement. First, all hints and answers in the training set are concatenated, using a special token to separate the hint and answer fragments. An autoregressive objective is used to zero out the token loss from user hints, thus backpropagation is only performed on the answer token.

[0194] Model fine-tuning is divided into efficient parameter fine-tuning and full / partial parameter fine-tuning. This application adopts full / partial parameter fine-tuning, which means training directly on the model and then fine-tuning by freezing some of the model's parameters. For example, the model can be fine-tuned twice to obtain a supervised fine-tuned model.

[0195] 4. Perform RLHF fine-tuning on the supervised fine-tuning model to obtain the RLHF fine-tuned model.

[0196] (1) This step first requires training a reward model. In principle, the supervised fine-tuning model can be directly fine-tuned by domain experts through RLHF (Revised Learning Highlighting and Handling) based on the input and the labeled output. However, this requires providing the domain experts with samples and scoring them after each round of RLHF fine-tuning, which is labor-intensive and requires a large dataset. Therefore, a simpler approach is to train a reward model using a set of domain expert annotations, with the domain experts manually ranking the different outputs under the same input. The purpose of the reward model is to simulate human scoring of text, with the model predicting which human-annotated score is higher.

[0197] (2) Reinforcement learning based on human feedback: Based on the supervised fine-tuning of the model and the reward model in the previous step, a reinforcement learning loop is executed, which mainly consists of the following three steps:

[0198] ① Domain experts annotate and generate feedback on the input.

[0199] ② Use a reward model to score the feedback.

[0200] ③ Reinforcement learning to optimize the scoring strategy.

[0201] After the above three steps, the RLHF fine-tuning model is obtained.

[0202] 5. Deploy the RLHF fine-tuning model

[0203] RAG technology is used to deploy large-scale models at the edge. In large-scale AI models, when answering questions or generating text, RAG first retrieves relevant information from a large-scale document library, and then uses this retrieved information to generate responses or text, thereby improving the quality of predictions.

[0204] This application requires the deployment of the RLHF fine-tuning model obtained in the previous step, which mainly includes the following steps:

[0205] (1) Indexing: First, indexing is created. This process is usually done offline, including data cleaning and extraction, converting different file formats (such as PDF, HTML, Word, etc.) into plain text, then dividing the text into blocks and creating an index.

[0206] (2) Retrieval: The user input is converted into a vector using the same encoding model. The similarity between the question embedding and the document block embedding is calculated. The top K document blocks with the highest similarity are selected as the enhanced context information for the current question.

[0207] (3) The given question and related documents are combined into a new prompt, and then the RLHF fine-tunes the model to answer the question based on the provided information. If there is historical dialogue information, it can also be incorporated into the prompt for multi-turn dialogue.

[0208] 6. Integrate the RLHF fine-tuning model with LangChain.

[0209] LangChain is a high-efficiency framework designed to simplify and optimize interaction with large models. This application utilizes LangChain’s rich tools and interfaces to integrate the RLHF fine-tuning model obtained above into the application.

[0210] The integration of LangChain with large models mainly involves the following steps:

[0211] (1) Import the LangChain library: Import the LangChain library in the interactive environment;

[0212] (2) Initialize LangChain: Create a LangChain instance to begin interacting with the RLHF fine-tuning model obtained above;

[0213] (3) Configure the connection with the large model: Configure the RLHF fine-tuning model obtained above;

[0214] (4) Implement model interaction: Use LangChain instances to perform specific tasks, such as generating text, answering questions, etc.

[0215] In this application, taking the retrieval question answering task as an example, the task uses the RLHF fine-tuning model obtained above to complete automatic question answering for a specific document. The process includes: text reading, text segmentation, text / question vectorization, text-question matching, combining the matched text as context and the question to generate the corresponding input as the model input, and generating the answer.

[0216] The functions that can be implemented using LangChain in this application include:

[0217] (1) Use LangChain to quickly build a chatbot. By customizing the dialogue style, you can achieve natural and fluent dialogue in a variety of scenarios.

[0218] (2) By combining LangChain and RLHF fine-tuning models, complex data analysis of local resources and capabilities can be performed, and easy-to-understand reports or visualizations can be generated.

[0219] 7. Applying AI-agent to the RLHF fine-tuning model

[0220] Similar to the ChatGLM-6B model, the domain-specific RLHF fine-tuning model obtained in this application also possesses the analytical reasoning capabilities of a large model, can accept input, and can output text / media. However, it lacks the planning, thinking, and memory abilities of humans. In contrast, an AI-agent can be understood as a general-purpose problem solver. It possesses planning, thinking, memory, and the ability to use tool functions. It is a computer program capable of autonomously completing given tasks, providing accurate information retrieval services, maintaining contextual coherence in complex dialogues, and offering a smooth communication experience.

[0221] This application combines the domain-specific RLHF fine-tuning model obtained above with an AI-agent, equipping the RLHF fine-tuning model with an AI-agent. The domain-specific RLHF fine-tuning model is considered the brain; the AI-agent can think based on the task objective, break down the task, and then call upon appropriate tools to complete the task. It has three key components:

[0222] (1) Planning: In this application, a single support request typically involves multiple support categories. Taking emergency rescue as an example, the support categories may include road emergency repair support, rescue tool support, food support, and medical and health resource support. Upon receiving a support request, the AI-agent first breaks down large support tasks into sub-tasks according to the support categories and plans the execution process for each support sub-task. The AI-agent then reflects on the task execution process based on the task progress, thereby deciding whether to continue executing the task or determine that the task is completed and terminate its operation.

[0223] (2) Memory: Short-term memory refers to the context during task execution, which is generated and temporarily stored during the execution of sub-tasks and cleared after the task is completed; long-term memory is information retained for a long time, generally referring to an external knowledge base, usually stored and retrieved using a vector database. In this application, short-term memory mainly refers to the ability to generate a second support plan based on the breakdown and execution status of the previous unfinished support task when two support requests are received consecutively; long-term memory mainly refers to the saving of the support plan generated each time based on the support request and its execution status, forming a professional domain knowledge base, which facilitates the rapid retrieval and generation of support plans when the same or similar support requests are input in the future.

[0224] Tool Usage: The AI-agent is capable of using tools. In this application, the AI-agent is equipped with various APIs, such as calculators, search tools, and database query tools. By calling these tool APIs, the AI-agent can create various intelligent applications, facilitate human-computer interaction, and solve practical needs assurance problems.

[0225] In summary, a large-scale model-based approach to resource support transformation first constructs a specialized domain dataset by acquiring local resource support data. Then, this dataset is used to pre-train the ChatGLM-6B model, generating a pre-trained model. Next, the model is further optimized through supervised tuning and RLHF parameter fine-tuning. The optimized model is then deployed at the edge and integrated with LangChain to form an RLHF interactive model. Finally, combined with an AI-agent, this RLHF interactive model can decompose and process different categories of support needs, achieving effective resource conversion and optimized resource allocation.

[0226] This method uses ChatGLM-6B as the base training model. Because the original training data for ChatGLM-6B is bilingual (Chinese and English) with a 1:1 ratio of Chinese to English, and its vocabulary covers most common Chinese characters and words, resulting in high Chinese encoding efficiency, this application adopts ChatGLM-6B as the training model. Combining large-scale model pre-training, supervised fine-tuning, and RLHF fine-tuning significantly improves the model's language understanding and generation capabilities in specific professional domains. Edge deployment and LangChain integration enable rapid model response and efficient interaction. The introduction of an AI-agent allows the system to handle complex security needs more intelligently, improving task processing efficiency and accuracy. Overall, this method provides strong technical support for the agile empowerment and precise release of local resources, enhancing local security capabilities in complex scenarios.

[0227] Secondly, this application provides a large-scale model-based security force conversion system, applied to the aforementioned large-scale model-based security force conversion method, the system comprising:

[0228] The acquisition unit is used to acquire local resource and capacity support data to obtain datasets for specific fields.

[0229] The training unit is used to pre-train the ChatGLM-6B model using the professional domain dataset to obtain a pre-trained model.

[0230] The supervisory unit is used to supervise and adjust the pre-trained model based on preset prompts and answer samples to obtain a supervised training model;

[0231] The fine-tuning unit is used to fine-tune the RLHF parameters of the supervised training model to obtain the RLHF training model.

[0232] The interaction unit is used to deploy the RLHF training model at the edge and integrate it with LangChain to obtain the RLHF interaction model.

[0233] The result unit is used to combine the AI-agent with the RLHF interaction model to perform security force conversion.

[0234] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the aforementioned large-model-based security force conversion method.

[0235] Fourthly, this application provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned large-model-based security force conversion method.

[0236] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0237] The various embodiments in this disclosure are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0238] The scope of protection of this disclosure is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its scope and spirit. If such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A large model-based force transition method, characterized in that, The method comprises: obtaining local resource strength guarantee data to obtain a professional field data set; pre-training a ChatGLM-6B model using the professional field data set to obtain a pre-training model; supervising and adjusting the pre-training model according to preset prompts and answer samples to obtain a supervised training model; RLHF parameter fine-tuning the supervised training model to obtain an RLHF training model; edge deployment of the RLHF training model and integration with LangChain to obtain an RLHF interaction model; utilizing an AI-agent in combination with the RLHF interaction model to perform strength conversion; wherein the step of supervising and adjusting the pre-training model according to preset prompts and answer samples to obtain a supervised training model comprises: constructing a plurality of preset samples with prompts and answers to obtain a preset training set; associating the prompts and answers of each preset sample in the preset training set to obtain a continuous input sequence; segmenting the prompts and answers in the preset sample using markers to obtain a prompt part and an answer part; using a self-recurrent objective function to zero the loss of the prompt part corresponding to the user prompt to obtain back propagation of the answer part; parameter adjusting the adjustable model part according to the back propagation of the answer part while keeping the model part parameters unchanged to obtain a supervised training model, wherein the adjustable model part and the frozen model part constitute the pre-training model.

2. The large model-based force transition method of claim 1, wherein, The step of pre-training a ChatGLM-6B model using the professional field data set to obtain a pre-training model comprises: structurally dividing the data in the professional field data set to obtain structured data, semi-structured data, and unstructured data; converting the semi-structured data and combining it with the structured data to obtain a first initial corpus; processing the unstructured data using regular expressions and natural language processing tools to obtain a second initial corpus; dividing a target corpus into two parts according to a preset division ratio to obtain a training set and a validation set, wherein the target corpus includes the first initial corpus and the second initial corpus, and both the first initial corpus and the second initial corpus are texts; segmenting the data in the training set and the validation set respectively to obtain paragraphs or sentences corresponding to the texts; segmenting the paragraphs or sentences corresponding to the texts using a word segmenter to obtain words or subwords corresponding to the paragraphs or sentences; mapping the words or subwords to integer encodings to obtain integer values corresponding to the words or subwords; inputting the integer values corresponding to the training set into an initial word segmentation model to obtain an intermediate word segmentation model; inputting the integer values corresponding to the validation set into the intermediate word segmentation model and adjusting the parameters of the intermediate word segmentation model according to the output results of the intermediate word segmentation model to obtain a target word segmentation model; Merge the vocabulary table in the target word segmentation model with the ChatGLM-6B original word segmentation model to obtain a new word segmentation model; Train the ChatGLM-6B model using the new word segmentation model to obtain a pre-training model. 3.The large model-based force transition method of claim 1, wherein, The step of performing RLHF fine-tuning on the supervised training model to obtain an RLHF training model comprises: Inputting a prompt into the supervised training model to obtain a plurality of candidate answers output by the supervised training model; Obtaining a scoring of the plurality of candidate answers output by the supervised training model to obtain a score of each candidate answer; Performing feature extraction on the candidate answers to obtain candidate answer features; Using the input prompt and the candidate answer features as input, and using the score of the candidate answer as a label, to form a reward sample, and obtaining a reward training set according to a plurality of reward samples; Training an initial reward model using the reward training set to obtain a target reward model; Using the target reward model to perform RLHF parameter fine-tuning on the supervised training model to obtain an RLHF training model. 4.The large model-based force transition method of claim 1, wherein, The step of performing edge deployment on the RLHF training model and integrating it with LangChain to obtain an RLHF interaction model comprises: Indexing the professional field dataset, wherein the index represents text blocks in the professional field dataset, forming a document block vector; Obtaining an input prompt of a user and converting the input prompt into a vector to obtain an input vector; Comparing the input vector with the document block vector in the professional field dataset to obtain the K document blocks with the highest similarity, wherein the K document blocks with the highest similarity are used as enhanced context information of the input prompt; Merging the K document blocks with the highest similarity with the input prompt to obtain a prompt for inputting into the RLHF training model, forming edge deployment of the RLHF training model. 5.The large model-based force transition method of claim 1, wherein, The step of performing edge deployment on the RLHF training model and integrating it with LangChain to obtain an RLHF interaction model comprises: Importing a LangChain library in an interactive environment formed by a user and the RLHF training model; Initializing LangChain and creating a LangChain instance connected to the RLHF training model to obtain an RLHF interaction model.

6. The large model-based force transition method of claim 1, wherein, The step of using an AI-agent in combination with the RLHF interaction model to perform guarantee force conversion comprises: Obtaining a guarantee demand task, wherein the guarantee demand task comprises guarantee tasks of different categories; Decomposing the guarantee demand task according to the guarantee category using an AI-agent to obtain a plurality of guarantee category subtasks; Inputting the plurality of guarantee category subtasks into the RLHF interaction model for execution to obtain an output result of each guarantee category subtask; Obtaining guarantee force conversion for solving actual demand guarantee prompts according to the output results of the plurality of guarantee category subtasks.

7. A large model-based force assurance force conversion system, characterized by, The system is applied to the guarantee force conversion method based on a large model according to any one of claims 1-6, and the system comprises: An acquisition unit is configured to acquire local resource support data to obtain a professional field data set; A training unit is configured to pre-train a ChatGLM-6B model using the professional field data set to obtain a pre-trained model; A supervision unit is configured to supervise and adjust the pre-trained model according to a preset prompt and answer sample to obtain a supervised training model; A fine-tuning unit is configured to perform RLHF parameter fine-tuning on the supervised training model to obtain an RLHF training model; An interaction unit is configured to perform edge deployment on the RLHF training model and integrate the LangChain to obtain an RLHF interaction model; A result unit is configured to use an AI-agent to combine the RLHF interaction model to perform support force conversion.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the large model-based support force conversion method in any one of claims 1 to 6.

9. A computer storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the large model-based support force conversion method in any one of claims 1 to 6.

Images