131 results about "Fine-tuning" patented technology

The invention discloses a large language model parameter fine tuning method, system and device, a storage medium and a product, and relates to the field of natural language processing. The fine tuning method comprises the steps that a pre-training model and a parameter matrix of the pre-training model are acquired, a parameter fine tuning module is arranged on a self-attention layer of the pre-training model, the parameter fine tuning module comprises LoRA modules of different task types and a router, each LoRA module comprises a general feature matrix and a specific feature matrix, and the general feature matrixes of the multiple LoRA modules are shared; obtaining a fine tuning data set; loading and freezing a parameter matrix of the pre-training model, and initializing parameters of a parameter fine tuning module; and performing fine tuning on the parameters of the parameter fine tuning module based on the fine tuning data set to obtain a fine-tuned large language model. Parameters of the MoE architecture are remarkably reduced, it is ensured that the model captures differences of various tasks to the maximum extent, and the cross-task generalization of the model is ensured.

The embodiment of the invention provides a fine tuning method and device for a large language model, equipment and a computer readable storage medium. According to the method, the performance of a to-be-fine-tuned large language model is tested, a sample set of prediction errors of the large language model is collected, the samples of the prediction errors are classified on the basis of real categories and error prediction categories of the samples of the prediction errors, namely, the prediction errors of the large language model are classified, and the classification accuracy of the prediction errors of the large language model is improved. Then, a trained pre-training language model is utilized to analyze reasons for generation of the prediction errors, and similar error samples with the same error condition are generated based on the reasons, so that targeted fine adjustment is carried out on the model, and the over-fitting phenomenon is eliminated. According to the method, the performance of the model on different types of errors can be better analyzed, and the overfitting problem of the model in a specific scene can be deeply known and solved, so that the performance of the model in each vertical field is remarkably improved.

The invention relates to a rotating machine fault diagnosis method based on a self-supervised time-frequency comparison fusion network, and belongs to the technical field of fault diagnosis. The method comprises the following steps: converting a time sequence signal into a frequency signal by using fast Fourier transform, and respectively performing data enhancement on the time frequency signal; initializing model parameters, adopting a multi-stage contrast learning strategy, and combining instance-stage, clustering-stage and fusion-stage contrast loss to obtain a pre-training target loss function; variables in the target loss function are selected and updated until convergence or the maximum iteration number is reached, and the pre-training model is stored; pre-training model parameters are loaded, a cross entropy loss function is adopted, and a fine tuning target loss function is obtained through label guidance; selecting and updating the target loss function of the fine tuning model until convergence or reaching the maximum iteration number, and storing the fine tuning model; and loading the trained fine-tuning model parameters, and inputting diagnosis data to obtain a prediction label, namely a diagnosis result. According to the invention, the precision and efficiency of fault diagnosis of the rotating machinery are obviously improved.

The invention discloses a domain intention recognition method based on a positive and negative correlation sample fine adjustment large model, and the method comprises the following steps: S1, generating a sentence vector, and carrying out the fine adjustment of an industry-specific embedding model through the unlabeled data of an industry; s2, generating vectors of industry tagged data through the embedding model obtained in the step S1, and constructing a vector index database; s3, constructing candidate intentions and example sentences, performing similar sentence retrieval based on the index database, and generating the candidate intentions and example sentences of the current sample; and S4, large model fine tuning: constructing fine tuning data to perform large model fine tuning. According to the method, in a scene of intention identification or classification depending on a large model, the problem that the intention identification capability of the large model in a specific field or industry is limited due to lack of industry information can be solved, and the identification effect of the industry is improved by finely adjusting the large model under specific industry data; and a good identification effect can be achieved for the industry with a large number of intentions.

The invention relates to the technical field of natural language processing, in particular to a parameter fine tuning method and device for downstream task adaptation of a large language model.The method comprises the steps that parameter fine tuning is conducted on a target layer of a pre-trained large language model to obtain the large language model after fine tuning; calculating first output probability distribution of the last layer of the target layer and second output probability distribution of the last layer of the non-target layer in the large language model after fine tuning; and performing knowledge updating on the target downstream task according to the first output probability distribution and the second output probability distribution. Therefore, the problems that the parameter quantity of a large language model is large, the requirement for consumption of computing resources in the fine adjustment process is high, meanwhile, the training time is long, and the downstream task adaptation cost of the large model is high are solved.

The invention relates to a large model parameter fine tuning method and device for electric power multi-modal data fusion and a medium. The method comprises the steps of obtaining electric power multi-modal data, constructing an electric power multi-modal feature vector and performing time sequence calibration, dividing the electric power multi-modal data into a plurality of sub-data sets according to a geographic position and an electric power equipment type, and distributing the sub-data sets to a plurality of computing nodes of a large model; calculating the local gradient of each calculation node, obtaining a parameter difference coefficient by calculating the gradient similarity between the adjacent calculation nodes, updating the weight coefficient of the calculation nodes, carrying out weighted aggregation on the local gradients of the plurality of calculation nodes according to the updated weight coefficient, obtaining a global gradient, and updating model parameters; and the parameter updating process is repeatedly executed until the model converges, and a large model after parameter fine tuning is obtained and used for outputting a power multi-modal data fusion result. Compared with the prior art, the method has the advantages that the problems of inconsistent power multi-modal data time sequence and unbalanced parameter aggregation are solved, and the model fine tuning efficiency and accuracy are improved.

The invention relates to the technical field of artificial intelligence development, in particular to an instruction fine tuning method, device and equipment of a large language model and a medium, which can be applied to training of an AI doctor family inquiry model, and the instruction fine tuning method of the large language model comprises the following steps: obtaining an initial data set; selecting sample data conforming to preset preferences from the initial data set to construct a preference data set; dividing the initial data set into a plurality of fine tuning data sets; and according to the plurality of fine tuning data sets, performing batch fine tuning on the pre-trained large language model, obtaining a preset number of preference sample data from the preference data set after a preset number of batches of fine tuning, and inserting the preference sample data into the next batch of fine tuning data set. According to the method, the big language model is trained by using the preference sample data in the instruction fine tuning stage, so that the output of the obtained big language model can be accurately aligned with the preference, meanwhile, a reinforcement learning alignment step is omitted, and resource consumption during training of the big language model is reduced.

The invention relates to a deep synthesis audio cross-domain detection method and device based on a self-supervision auxiliary task, and the method comprises the steps: obtaining a to-be-detected audio as a test data set; carrying out augmentation processing on the test data set based on a preset self-supervision auxiliary task to obtain an augmented data set; based on the augmented data set, taking the self-supervision auxiliary task as an optimization target, and performing fine tuning on a pre-trained target detection model to obtain a target fine tuning model; training tasks of the target detection model comprise a forgery detection task and a self-supervision auxiliary task; and inputting the test data set into the target fine tuning model to obtain a detection result. Through the method and the device, the problems that generalization is insufficient and the cross-domain detection effect is obviously reduced in the prior art are solved, and the model training overhead can be greatly reduced while the model generalization is improved; and the method can adapt to the distribution difference of deep forging attributes in a test data set, so that the cross-domain detection performance is improved.

The invention relates to a causal reasoning-based retrieval enhancement generative model fine tuning method and system, and the method comprises the following steps: A, modeling a causal relationship to reveal an influence mechanism of an input variable on related knowledge extraction and irrelevant knowledge filtering; step B, defining a knowledge revenue score (KGS), and calculating the knowledge revenue score through expert labeling or automatic evaluation to construct a causal enhancement data set (CED); c, designing a fine tuning strategy based on intervention and anti-factual reasoning, systematically performing knowledge variable intervention and anti-factual condition simulation, and enhancing the sensitivity of the model to related knowledge and the robustness of the model to irrelevant knowledge; and D, dynamically integrating the context and the knowledge to generate a high-quality response in a reasoning stage. The method is beneficial to improving the accuracy and stability of the dialogue generation model.

The invention provides a GIS industry large language model parameter fine tuning method considering parameter adaptability difference, and relates to the field of geographic information science and deep learning, the method comprises the following steps: giving a GIS downstream task training data set and a pre-training large language model, and calculating the adaptability of each parameter of the model to the data set; calculating the adaptability of each level of the large language model according to the calculated adaptability of each parameter; based on the adaptability of each layer, distributing different trainable parameters for the model layers with different adaptability; and based on the distributed trainable parameters, performing fine tuning training on the pre-trained large language model to obtain a GIS industry large language model, and completing efficient fine tuning of the large language model parameters. According to the technical scheme, the heterogeneity of the large language model parameters in the GIS professional knowledge adaptation process is considered, and the characteristic is utilized to optimize the LoRA fine adjustment process so as to improve the performance of the large language model on GIS professional tasks.

The embodiment of the invention provides a fine tuning method and device for a large language model. The method comprises the steps that a weight matrix of a pre-trained large language model is acquired, the weight matrix is decomposed into an amplitude vector and a first direction matrix, a first low-rank matrix and a second low-rank matrix are initialized, and the product of the first low-rank matrix and the second low-rank matrix is used for fitting an increment matrix of the first direction matrix; the first direction matrix is kept unchanged, multiple rounds of fine tuning are executed, and each round of fine tuning comprises the steps that a training text is input into a large language model, and calculation of a loss function is conducted; updating the amplitude vector based on the loss function; determining a first gradient matrix of the loss function relative to the current direction matrix; obtaining an updated first low-rank matrix and an updated second low-rank matrix according to the first gradient matrix; and determining the increment matrix of the round based on the product of the updated first low-rank matrix and the second low-rank matrix, and superposing the increment matrix of the round on the first direction matrix to serve as the updated direction matrix of the round.

The invention provides a language model fine tuning method and device, electronic equipment and a storage medium, and relates to the field of natural language model processing, and the method comprises the steps: obtaining sample data which comprises a plurality of data with different data qualities; reasoning the sample data for multiple times based on the first language model to generate multiple pieces of result information and multiple pieces of process information; determining a loss weight based on the multiple pieces of result information and the first label of the sample data; on the basis of the multiple pieces of result information, the multiple pieces of process information and a first label of the sample data, determining thinking chain data; and based on the thinking chain data and the loss weight, determining a target loss, and performing fine tuning on the to-be-fine-tuned model based on the target loss to obtain a target language model. According to the method, the quality of the sample data and the logical thinking ability of model reasoning are concerned in the optimization of the to-be-fine-tuned model through the loss weight and the thinking chain data, so that the interpretability and the mathematical ability of the language model are effectively improved, and the error rate of the model is reduced.

The invention discloses a method and device for improving the fine tuning effect of a large model based on an iterative data enhancement strategy. The method comprises the following steps: manually generating a batch of seed data with different distributions according to requirements; enhancing artificially generated seed data based on a rule or a large model; checking the quality of the enhanced data; finely adjusting the model by using the checked data, and reasoning training data by using the trained model; performing comprehensive evaluation on the reasoning result in various aspects; whether the evaluated indexes meet requirements or not is judged; if the error sample does not meet the requirement, enhancing the error sample again, and iterating repeatedly; and if the evaluation result meets the requirement, ending the whole process. According to the method, the threshold of fine adjustment of the large model under the small data set is remarkably reduced, and the model is excellent in performance in a data scarce scene by generating the targeted synthetic data.

The invention provides a rainfall runoff time sequence simulation method based on pre-training-fine tuning, and belongs to the field of hydrological simulation. The method comprises the following steps: firstly, acquiring a consistent hydrometeorological data set for global multi-basin pre-training and target basin fine tuning, constructing an LSTM runoff prediction model, performing global multi-basin pre-training to obtain a base model parameter weight, migrating the base model parameter weight to a target basin, freezing LSTM network layer parameters, and only finely tuning a regression output layer; and testing and evaluating in the target drainage basin. According to the method, firstly, transferable hydrological response is learned on global multi-basin large samples, and then lightweight fine tuning is carried out on the target basin, so that the prediction precision and cross-regional generalization ability of a data scarce region can be effectively improved, and rapid deployment is facilitated.

The present technology provides a mechanism to obtain results of similar quality to that which can be obtained by fine-tuning a generative model from the foundational model without fine-tuning. In particular, the present technology can provide a suitability prompt to evaluate and improve the output of a generative model without fine-tuning. A suitability prompt is an engineered prompt that is provided to a generative model that prompts the generative model to evaluate a candidate response that has been generated by the generative model. Often the suitability prompt can include an indication of one or more attributes of a quality candidate response. When the generative model provides a response to the suitability prompt that indicates that the candidate response is a quality response, the candidate response can be deemed good enough to be returned to a user.

The invention discloses a large model fine tuning optimization method based on multi-strategy fusion, which comprises the following steps: designing a dynamic parameter selection mechanism, adaptively determining a parameter subset needing fine tuning according to a task demand and a model structure, and reducing unnecessary parameter updating calculation; constructing a dynamic low-rank decomposition framework, dynamically adjusting the rank of a low-rank matrix according to a model training state and data characteristics, and keeping key information while compressing a parameter scale; a self-adaptive task sensing mechanism is introduced, a fine adjustment strategy is automatically adjusted according to different task characteristics, and the adaptability of the model to various tasks is improved; and a mixed precision training method is adopted, so that the calculation complexity and the memory occupation are reduced on the premise of ensuring the model precision. According to the method, a parameter efficient fine tuning technology and a dynamic low-rank decomposition strategy are innovatively combined, and an adaptive task perception mechanism and a mixed precision training technology are introduced, so that the operand and resource requirements of model training are effectively reduced, and the fine tuning efficiency and the model performance are improved.

The invention discloses a dynamic structured efficient fine tuning method for a language model and electronic equipment. The method comprises the following steps: acquiring a natural language model needing to be finely tuned; acquiring a data set of a natural language downstream task, and designing a prompt statement for guiding the model to understand the downstream task; performing fine tuning on the natural language model by adopting the data set and the prompt statement; wherein the fine tuning process comprises the following steps of: updating parameters of the model by using a zero-order optimization method; adjusting a forward propagation process by using a structured pruning scheme; dynamically adjusting a pruning structure by using a genetic algorithm and a random selection method; and determining to explore or execute an optimal pruning scheme by using a reinforcement learning strategy. Structured pruning, reinforcement learning and genetic algorithms are introduced to solve the problems of high calculation overhead and long training time in the zero-order optimization process. According to the method, the calculation overhead of zero-order optimization on all models and tasks can be reduced on the premise of maintaining model performance, and the method can be widely applied to the field of natural language model fine adjustment.

The invention discloses a hybrid adaptive enhanced fine tuning method, system and device, relates to the technical field of artificial intelligence, and is particularly suitable for a deep reasoning task of a large language model. The problems of response level length deviation, problem difficulty level deviation, insufficient exploration efficiency, insufficient sample utilization and the like existing in an existing reinforcement learning algorithm are solved. The method comprises the steps of data preprocessing, model parameter and reference strategy initialization, multiple response generation, award calculation, advantage calculation and correction, model strategy updating, sampling probability adjustment and iterative optimization. Wherein all potential useful samples are ensured to be fully utilized by introducing the correction advantages of a length normalization factor and a difficulty normalization factor and combining a hybrid cutting mechanism and an adaptive sampling strategy update model. Deviation is effectively eliminated, the model reasoning ability and training efficiency are improved, the long reasoning task exploration ability is enhanced, and diversified task requirements are met.

The invention relates to the technical field of field detection, in particular to a zero-sample field detection method and system based on multi-task fine tuning and knowledge enhancement of argument data, and the method comprises the steps: firstly, searching related knowledge between a text and an object from an external knowledge base through a prompt project, and carrying out the knowledge enhancement of fine tuning data and test data; then analyzing and discussing that a plurality of tasks are included in the corpus data, and meanwhile, preprocessing data of each task; and finally, performing multi-task fine tuning on the open-source large model by using the discussion corpus data and external knowledge, and performing field reasoning by using the model after fine tuning. According to the method, multi-task fine tuning is carried out on the open-source large model by utilizing discussion corpus data with rigorous logic and relatively low noise, so that the performance of model site detection is improved; and meanwhile, the model takes related knowledge returned by an external knowledge base as a context, so that the accuracy of the model in text understanding and analysis aspects is enhanced.

The invention discloses a large-model intelligent question setting system and method based on fuzzy mathematics fine tuning, and belongs to the technical field of intelligent question setting. Comprising a fuzzification processing module, a multi-dimensional reward feedback module, a rule-driven attribute reasoning module, an attribute accurate quantification module, a self-adaptive membership degree adjustment module, a strategy optimization and intra-cluster standardization module and a question generation and structured output module. According to objective fields such as post specifications, post levels, technical stack depth and team scales, hard indexes are converted into soft boundary semantics through five membership functions, and then quantitative attributes of surface test questions on difficulty, openness, prejudice and distinction are reasoned by using N rules without subjective wording. And the business result data is used for driving rewards to be updated online, and finally, daily automatic fine tuning is realized on the 32B large model, so that each subjective surface test question generated by the AI not only meets the objective requirements of posts, but also has explainable, auditory and iterable closed-loop capabilities.

The invention relates to a large language model alignment optimization method based on small data volume fine tuning and a storage medium. The invention relates to the large language model alignment optimization method based on small data volume fine tuning and the storage medium. The invention aims to solve the problems that the existing large model fine tuning needs a large amount of annotation data, needs to consume a large amount of computing resources, and needs higher fine tuning cost and development time. The method comprises the steps that 1, a data set for a specific target task is constructed and divided into two parts, one part is a fine adjustment data set, and the other part is an alignment data set; meanwhile, a problem data set for a specific target task is constructed; 2, obtaining a fine tuning model; 3, obtaining an evaluation model; 4, inputting the alignment data set problem into a fine tuning model, outputting a result by the fine tuning model, and calculating the similarity between the output result of the fine tuning model and a target result; inputting an output result of the fine-tuning model into the evaluation model, and outputting an evaluation result by the evaluation model; and 5, obtaining the target model after alignment optimization.

The embodiment of the invention discloses a forgetting learning method and device for a large language model. The method comprises the steps that firstly, a first large language model and a forgetting sample set are obtained, the first large language model is initially a fine-tuning model obtained by fine-tuning a fine-tuning sample set, and the forgetting sample set is a subset of the fine-tuning sample set; then, for any first forgotten sample, a plurality of similar samples are determined based on a second large language model, and each similar sample and the first forgotten sample have similar semantics but different expressions; processing the first forgotten sample by using a first large language model to obtain a first hidden layer representation; then, training loss is determined, the training loss is negatively correlated to the distance between the first hidden layer representation and the hidden layer representation of each similar sample and is positively correlated to the distance between the first hidden layer representation and the random vector, and the hidden layer representation of each similar sample is obtained based on a fine tuning model; and then training the first large language model by using the training loss to realize forgetting learning.

The invention discloses a Tibetan language large language model fine tuning method, device and system, and a storage medium. The method comprises the following steps: S1, obtaining a TIFD data set; s2, finely adjusting the Tibetan language large language model according to the TIFD data set; wherein low-rank increments are injected into the weight matrix of the base model through LoRA fine tuning. By adopting the technical scheme of the invention, the problems of high model fine tuning cost and low efficiency in a Tibetan language data scarcity scene are solved; and the generation accuracy of the Tibetan large language model on complex grammar structures and cultural terms is improved.

The embodiment of the invention provides a method and device for obtaining training data used for fine adjustment of a large model, and the method comprises the steps: obtaining a first instruction for the large model, inputting the first instruction into a first large model, and obtaining a first answer; determining a complexity score for the first instruction by inputting the first instruction into the second large model, determining a response quality score for the first answer by inputting the first instruction and the first answer into the second large model, and determining a comprehensive score corresponding to the first instruction and the first answer according to the complexity score and the response quality score; and determining whether the first instruction and the first answer are used for fine tuning of the first large model according to the comprehensive score.

The invention relates to a brain-like scene safety assessment method combining process supervision and fast and slow thinking. The method comprises the following steps: firstly, designing a brain-like scene security cognition process and a scene security level division standard; then, constructing a supervision fine tuning data set and a reinforcement learning fine tuning data set of a process supervision normal form based on the brain-like scene security cognition process and a scene security level division standard; then, performing cold start on an open-source pre-training multi-modal large language model based on the supervision fine tuning data set to obtain a supervision fine tuning model; carrying out reinforcement learning training on the supervision fine tuning model based on the reinforcement learning fine tuning data set of the process supervision normal form; and finally, inputting a to-be-evaluated scene image into the trained supervision fine tuning model to obtain a scene security level. The inference process of the model can be aligned with the thinking process of human in the macroscopic level, the flexibility of the thinking process is maintained, and the interpretability of the result of each step is improved.

The invention discloses a small language model fine tuning method based on an LLaMA Factory tool, and the method comprises the following steps: 1) carrying out the thinking chain distillation of an open source data set through a large model, and generating a training data set for the fine tuning of a small language model; 2) based on an LLaMA Factory tool, building a model fine tuning environment; 3) fine-tuning real-time monitoring and adjustment, in the fine-tuning process, monitoring loss of the verification set and the test set, and if the performance of the model is reduced or an over-fitting phenomenon occurs, adjusting training parameters and retraining; and 4) completing training and exporting the model: after the fine tuning process is completed, if the model reaches preset performance on the verification set, completing training, exporting the trained small language model, and deploying the small language model to practical application for reasoning. According to the method, targeted training is performed on the pre-trained small model through a fine tuning technology, specific task requirements can be efficiently met, and rapid migration of a new task can be realized only by using a small-scale task data set to adjust part of parameters of the model.

The application provides a pre-training-fine-tuning-based precipitation runoff time series simulation method, and belongs to the field of hydrological simulation. First, a consistent hydro-meteorological data set for global multi-basin pre-training and target basin fine-tuning is obtained, an LSTM runoff prediction model is constructed, a base model parameter weight is obtained through global multi-basin pre-training, the base model parameter weight is migrated to the target basin, the LSTM network layer parameter is frozen and only the regression output layer is fine-tuned, and test evaluation is carried out in the target basin. The application learns the transferable hydrological response on a large sample of global multi-basins first, and then performs light fine-tuning in the target basin, which can effectively improve the prediction accuracy and cross-region generalization ability in the data scarce area, and is convenient for rapid deployment.

The present application provides a method for quantifying the value of scientific questions by embedding a multi-dimensional mixture-of-experts mechanism, which relates to the technical field of data processing and includes: reading predetermined quantification dimensions; introducing a predetermined triple strategy to construct a target instruction fine-tuning dataset; performing parameter quantification based on the fine-tuning principle of the QLoRA large language model; obtaining a scientific question text, performing semantic feature analysis through a gating network, and obtaining the target weight coefficient of the scientific question text according to the text semantics; analyzing the scientific question text through a target recognition large model and combining the target weight coefficient to obtain a target quantification result. Through the present application, the technical problem in the prior art that due to relying on expert evaluation and subjective judgment, there are inconsistent evaluation criteria, resulting in low efficiency of scientific question value evaluation can be solved. By integrating multiple dimensions and combining large language models for fine-tuning, the internal evaluation of the value of scientific questions is realized, and the efficiency of scientific question value evaluation is improved.

The invention relates to a large model compression method based on continuous layer pruning and endpoint tuning, and the method comprises the steps: firstly introducing a learnable continuous interval soft mask, and building a differentiable hierarchical mask mechanism in a model in cooperation with a residual bypass; secondly, by minimizing the KL divergence between output distributions before and after pruning, the optimal pruning starting point and length are automatically learned, and adaptive selection of continuous layer segments is achieved; then, executing physical layer deletion according to the optimized interval parameters, and reconnecting the network structures before and after pruning; and finally, implementing an endpoint tuning strategy, only carrying out all-parameter fine tuning on key layers on two sides of the sheared interval, and recovering the model performance at the lowest calculation overhead. According to the method, through combination of differential interval search and end point directional optimization, accurate compression and high-performance maintenance of the depth dimension of the large model are realized, model storage occupation and reasoning delay are remarkably reduced, the model output reliability in a key task scene is guaranteed, and the method is suitable for large-scale popularization and application. The method is particularly suitable for efficient deployment of the large language model in a resource-constrained environment.

The application discloses a large language model parameter fine-tuning method and device based on manifold constraint multi-track adaptation and a medium. In view of the problems of unstable training and limited expression capacity of an existing low-rank adaptation technology, a plurality of parallel low-rank tracks are constructed, and a double random matrix is introduced to constrain the information flow between the tracks, so that the stability of gradient propagation is ensured. Meanwhile, the expression capacity of the adaptation module is enhanced under a limited parameter budget by dynamically fusing the outputs of the tracks through a dynamic gating mechanism. The method can realize stable, efficient and high-performance model fine-tuning by fine-tuning a small number of parameters, and reasoning has no additional overhead, and is particularly suitable for application scenarios with limited resources and high stability requirements, and has wide practical value.