83 results about "Generic knowledge" patented technology

The invention discloses a novel dry bulk cargo port multi-scene simulation method based on a large language model, belongs to the technical field of port production and operation, and realizes the expansion of training samples by using the general knowledge of the large language model; based on an Adam optimizer, training the initial model by adopting the training set, and verifying the initial model by adopting the verification set; evaluating a multi-scene simulation result under the guidance of the model based on a test result of the test set, and performing iterative expansion training by utilizing a prompt learning mechanism in combination with multi-scene data to obtain a trained large language model; extracting key semantics of scene features through the trained large language model according to a current operation scene of a port; performing simulation resource configuration reasoning on the extracted key semantics; a novel dry bulk port simulation model in a corresponding scene is constructed by adopting AnyLogic simulation, and a big language model training corpus and a training set are updated according to a simulation preview result and port actual operation data. According to the invention, the production operation decision can be optimized.

Systems and methods for optimizing large language models (LLM) with domain-oriented model compression. Importance weights for general knowledge in a trained LLM, pretrained with deep learning, can be determined by computing the error when removing a weight from the trained LLM. The trained LLM can be iteratively optimized to obtain a domain-compressed LLM with domain knowledge while maintaining general knowledge by: fine-tuning the trained LLM iteratively with domain knowledge using the importance weights for general knowledge to obtain a fine-tuned LLM; determining importance weights for domain knowledge in the LLM with a regularization term by using gradient descent to optimize parameters when the fine-tuned LLM is trained with domain knowledge; and pruning learned knowledge based on importance weights for domain knowledge. A corrective action can be performed on a monitored entity using the domain-compressed LLM.

The invention relates to a steel bridge disease detection and identification method based on a large language model, and belongs to the technical field of artificial intelligence and civil engineering crossing. According to the method, a cross-modal feature alignment mechanism is constructed through a pre-trained multi-modal large language model by fusing a steel bridge image and a field customized text prompt, and a cascade detection process of'component identification-disease classification-region segmentation 'is realized. Comprising the following steps: designing a structured text prompt word bank to enhance semantic consistency, and dynamically fusing general knowledge and instance features in combination with a mixed prompt mechanism; a multi-level cross-modal alignment strategy is adopted to generate an anomaly graph, and a disease area is accurately positioned; a visual prompt enhancement module is introduced to improve the multi-scale feature discrimination ability, and the robustness in a complex environment is adjusted and optimized through data self-adaption. Under the condition of few samples or even zero samples, high-sensitivity detection and pixel-level segmentation of steel bridge cracks, corrosion and other diseases are achieved, and the problems that a traditional method is low in efficiency, poor in generalization, high in labor cost and the like are effectively solved.

An automated general knowledge worker may be used to perform knowledge worker tasks. The automated general knowledge worker may receive a text-based request to perform a knowledge worker task for an organization from a computing device. The automated general knowledge worker may retrieve one or more existing plans of action and contextual data that are relevant to the text-based request from a knowledge base. Further, the automated general knowledge worker may generate a text-based prompt that at least includes text information included in the text-based request, the one or more existing plans of action that are relevant to the text-based request, and the contextual data that are relevant to the text-based request. Subsequently, the automated general knowledge worker may send the text-based prompt to a large language model (LLM) to prompt the LLM to generate a new plan of action for performing the knowledge worker task.

The invention discloses an intelligent policy question-answering method and system based on retrieval enhancement generation and a medium. The method comprises the following steps: constructing a universal knowledge base and a plurality of mutually independent domain knowledge bases; in response to user input, the following steps are executed: performing routing analysis on the user input through a first large language model to generate a structured routing decision; retrieving the general knowledge base to obtain related general knowledge text segments and vectorization expressions thereof; according to the routing decision, a domain knowledge base corresponding to the at least one policy domain identifier is retrieved in parallel, and related policy text fragments corresponding to the at least one domain knowledge base and vectorization expressions of the related policy text fragments are obtained; obtaining an initial answer set based on retrieval results of the general knowledge base and the domain knowledge base; and performing intelligent fusion processing on the initial answer set through a second large language model, and generating and outputting response content. According to the method, the problem of knowledge updating lag is effectively solved, and the accuracy, timeliness and cross-domain specialty of policy questions and answers are improved.

The invention relates to the technical field of power systems, in particular to a method for constructing a general knowledge base in the power safety industry based on a large model. Performing data classification and sorting on the data source of the electric power safety industry to obtain classified data; constructing a plurality of agents based on the large model, and processing the classified data by using the plurality of agents to obtain structured data; and carrying out vectorization processing on the structured data to generate vector data, storing the vector data into a vector database, and constructing a universal knowledge base in the electric power safety industry. The data source comprises intrinsic safety analysis historical data, an intrinsic safety analysis report, electric power safety rules and regulations and operation specifications. The intelligent agents comprise a file analysis intelligent agent, an analysis report information extraction intelligent agent, a rule and regulation information extraction intelligent agent and an operation instruction information extraction intelligent agent. Through a multi-agent cooperative processing and vectorization storage mechanism, efficient organization and intelligent application of electric power safety knowledge are realized.

The invention relates to the technical field of data processing, in particular to a knowledge graph construction method and device, equipment and a storage medium, and the method comprises the steps: obtaining an industrial field general knowledge graph to construct a training data set, and training a block model based on the training data set; inputting the obtained private data to be constructed into the block cutting model to obtain a text block; acquiring example data, and inputting the text block and the example data into a pre-constructed large language model to obtain graph data; generating a weighted graph based on the graph data, and performing graph recursion clustering processing on the weighted graph to obtain community structure information; processing the community structure information by adopting a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed; according to the method disclosed by the invention, the block cutting flexibility and the block cutting accuracy can be improved in the face of various types of industrial private data by pre-training the block cutting model, so that the information fidelity and the time complexity are balanced, and the quality of the generated text block is greatly improved.

The invention discloses a data scarcity industrial vertical field-oriented knowledge-guided retrieval enhancement generation method, belongs to the technical field of natural language processing and industrial intelligence crossing, and can improve the retrieval accuracy and generation reliability of a large language model in an industrial scene. According to the method, a'sparse vector + dense vector 'mixed knowledge base is constructed, and general knowledge and industrial field texts are fused; after the model generates an initial text, judging whether external retrieval is needed or not through multi-dimensional evaluation token confidence; extracting attention weights for the low-confidence tokens, and screening key tokens to generate a retrieval query; dynamically adjusting the weight of a retriever based on a BGE-M3 model, and optimizing a retrieval result through reordering; the retrieval knowledge is converted into a context with an index, and a prompt template is constructed to generate a correction value iteration calibration text; and finally, optimizing the text format, and generating a structured response meeting industrial requirements. The method solves the problems of lack of professional knowledge of large language models in the industrial field, shallow retrieval and generation fusion, lack of knowledge calibration mechanisms and the like.

The present invention generally relates to the field of robotics, and, more particularly, to a method and system for task anticipation by integrating large language models and classical planning. Conventional methods for task anticipating use data-driven deep network architectures and Large Language Models (LLMs) for task estimation but they do so at the level of high-level tasks and require a large number of training examples. Thus, embodiments of present disclosure provide a method and system for task anticipation by integrating large language models and classical planning. The disclosed method and system leverages the generic knowledge of LLMs through a small number of prompts to perform high-level task anticipation, using the anticipated tasks as joint goals in a classical planning system to compute a sequence of finer granularity actions that jointly achieve these goals.

The invention relates to the technical field of retrieval intelligent generation, and discloses an RAG model-driven dynamic knowledge retrieval and intelligent generation system, which comprises a knowledge base establishment module and a general knowledge base for establishing an RAG model, the general knowledge base comprises sub-knowledge bases in a plurality of fields, and the content of each sub-knowledge base is a practical automatic acquisition mechanism. And the updated data source can be timely reflected into the sub-knowledge base. By acquiring the semantic similarity of the request content input by the user twice, the system can more accurately judge the change of the user demand, if the request content input by the user twice has high similarity in theme or context, the system can realize that the user demand is not met, then the answer strategy is optimized, and the user experience is improved. And invalid or repeated feedback is prevented from being provided.

The invention discloses a continuous learning method and system for a power transformation equipment fault diagnosis model, belongs to the technical field of image processing, and solves the problem of how to enhance the recognition capability of a visual language pre-training model for new and old knowledge. According to the method, fine-grained image semantic knowledge extraction is realized in a weak supervision mode, and the corresponding relation of a plurality of images with similar local features can be mined by calculating global semantic score alignment loss and local semantic feature comparison loss, so that the model can fully understand a new semantic concept and decouple knowledge, and the semantic knowledge extraction efficiency is improved. According to the method, new knowledge learning is carried out more accurately, general knowledge and special knowledge in new data are decoupled by calculating the matching degree of fine-grained text features and fine-grained semantic features, new semantic concepts are decoupled, the relation between the new semantic concepts and existing knowledge is established, and the knowledge learning efficiency is improved on the premise that the general knowledge is kept. And special knowledge of a new semantic category is better learned, good compatibility of new knowledge and old knowledge is realized, and the model has better forward compatibility capability.

The invention discloses a model training method and device, an information processing method and device, electronic equipment, a storage medium and a computer program product.The method comprises the steps that repeated sample question and answer pairs and / or sample question and answer pairs which cannot be predicted to obtain calibrated answers are filtered from sample question and answer pairs contained in a first training set, and a second training set is obtained, the first training set and the second training set comprise sample question and answer pairs related to the communication network; a first model is trained based on the second training set and a third training set, the third training set comprises sample question and answer pairs in the general knowledge field, and the first model is used for predicting corresponding answers based on questions in the sample question and answer pairs.

The invention discloses a data query tool method and system based on an AI language model. The method comprises the steps that a data query request input by a target user is obtained; inputting the data query request into a basic language layer of the language analysis model to obtain basic semantic request data; inputting the basic semantic data into a general knowledge layer of a language analysis model to obtain fused semantic request data; inputting the fused semantic request data into a professional knowledge layer of a language analysis model to obtain professional semantic request data; mapping the professional semantic request data into a database to obtain an SQL (Structured Query Language) query statement; based on the SQL query statement, accessing a database to obtain a preliminary query result; and auditing and verifying the preliminary query result to obtain a target query result. According to the method, the accuracy of the query result is improved, and comprehensive consideration and design are carried out in the aspect of specialty, so that the complex and changeable data query scene requirements are met.

Systems and methods for optimizing large language models (LLM) with domain-oriented model compression. Importance weights for general knowledge in a trained LLM, pretrained with deep learning, can be determined (110) by computing the error when removing a weight from the trained LLM. The trained LLM can be iteratively optimized (120) to obtain a domain-compressed LLM with domain knowledge while maintaining general knowledge by: fine-tuning (130) the trained LLM iteratively with domain knowledge using the importance weights for general knowledge to obtain a fine-tuned LLM; determining (140) importance weights for domain knowledge in the LLM with a regularization term by using gradient descent to optimize parameters when the fine-tuned LLM is trained with domain knowledge; and pruning (150) learned knowledge based on importance weights for domain knowledge. A corrective action can be performed (160) on a monitored entity using the domain-compressed LLM.

A language model updating method, performed by an electronic device includes, acquiring a first training sequence including at least one of text used for training or a token sequence corresponding to the text used for training; extracting a first general knowledge representation of the first training sequence using a first language model; using a second language model to extract a second general knowledge representation of the first training sequence and determine a plurality of prediction results corresponding to the first training sequence; and updating the second language model based on the first general knowledge representation, the second general knowledge representation, and the plurality of prediction results, wherein each of the first general knowledge representation and the second general knowledge representation include at least one of language patterns or semantic relationships, extracted from the first training sequence.

The invention relates to the technical field of code generation, in particular to a knowledge-adaptive code retrieval model, method and system. The knowledge adaptive code retrieval model provided by the invention comprises a code language selection module, a code knowledge module, a general knowledge module and a feature fusion module, the code language selection module and the feature fusion module are respectively connected with the code knowledge module and the general knowledge module; the code language selection module selects appropriate code languages for different user questions, the various code knowledge modules learn specific knowledge of each code language, and the universal knowledge module captures user intentions or universal knowledge of various code languages, so that under the interaction of the universal knowledge module and the various code knowledge modules, the user questions can be learned through the code language selection module. And the learned feature embedding is enabled to be richer and more professional, so that the accuracy of a code retrieval task is improved. And the code language selection module adaptively selects a proper code language for the user problem, and considers the applicability and the applicability degree of each code language, so that the code retrieval accuracy can be improved.

The invention relates to the technical field of text data processing, in particular to a specific domain model training method based on an expert knowledge base, and the method comprises the steps: carrying out word segmentation processing on each sentence in each text in an obtained initial general knowledge base and an initial expert knowledge base; determining a target similarity between every two texts; all texts are clustered, and the initial expert knowledge base and the initial general knowledge base are expanded and deleted respectively; determining a target professional contribution degree corresponding to each target segmented word and a target learning rate corresponding to each sentence in the initial expert knowledge base; and updating the scaling factor, corresponding to each sentence in the initial expert knowledge base, in the LoRA model to a corresponding target learning rate, and performing fine tuning on the language model through the LoRA model to obtain a trained language model. According to the method and the device, the text data in the knowledge base is processed, so that the rationality of training the language model in the specific field is improved.

The invention provides a method and a device for evaluating a space moving track, and relates to the technical field of artificial intelligence. The invention discloses a spatial movement track assessment method, which comprises the following steps of: generating track-risk image data according to a spatial movement track and a corresponding risk map; inputting the trajectory-risk image data and a pre-constructed task cue word into a pre-constructed visual-language large model to obtain an evaluation result; and transmitting the evaluation result to the target end. According to the technical scheme provided by the embodiment of the invention, the space movement track and the risk map which need to be evaluated are uniformly converted into the image representation as the input of the vision-language large model, and the image representation and the task cue word are input into the vision-language large model; the general knowledge and the cross-modal reasoning ability of the vision-language large model are utilized to finish track evaluation in real time, and the expansibility and the interpretability are good.

The embodiment of the invention provides a multi-modal universal model self-evolution method and system based on incremental learning, and the method comprises the steps: receiving a current target task and multi-modal data corresponding to the current target task, and determining a corresponding trained target universal model; extracting data features of the multi-modal data to obtain multi-modal data features, and processing the multi-modal data features and the multi-modal data through a target special model to obtain a target result; historical multi-modal data features corresponding to the historical multi-modal data are obtained, and general-purpose model parameters of the trained target general-purpose model are updated; and monitoring the performance index of the trained target general-purpose model according to a preset monitoring period, and if the performance index corresponding to the current preset monitoring period does not meet the preset performance index, adjusting the trained target general-purpose model according to a preset adjustment mode. According to the method, the defect of insufficient collaboration between general knowledge and special knowledge is effectively overcome, and the collaboration effect and the updating effect of updating according to new knowledge are remarkably improved.

The embodiment of the invention discloses a training method and device of a vertical domain question and answer model, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the steps that a first data set, a second data set and a third data set are obtained, the first data set is a data set representing general knowledge, the second data set is a data set representing vertical domain knowledge, and the third data set is a data set representing question and answer tasks based on the vertical domain knowledge; training a vertical domain model based on the first data set and the second data set, wherein the vertical domain model is a model obtained by adding at least one decoding layer in a basic model; and on the basis of the trained vertical domain model, a vertical domain question and answer model is trained based on the third data set, the vertical domain question and answer model is a model obtained by adding a low-rank adjustment structure in the trained vertical domain model, and the vertical domain question and answer model is used for executing a question and answer task based on vertical domain knowledge. By adopting the scheme provided by the invention, the training quality of the vertical domain question and answer model can be improved.

The invention discloses a privacy enhancement continuous learning method based on lexical-level differential privacy and memory shaping, which is characterized by comprising a lexical-level dynamic differential privacy module and a privacy guide memory shaping module, the lexical element level dynamic differential privacy module adaptively allocates privacy budget according to semantic sensitivity of different lexical elements in a text, and introduces Gaussian noise in an embedded layer; the privacy guide memory shaping module comprises memory regularization and privacy perception forgetting; the memory regularization is used for dynamically constraining model parameters; the privacy aware forgetting is used to selectively impair memory for highly sensitive information. Compared with the prior art, the method has the advantages that higher accuracy and lower forgetting rate are maintained while privacy is guaranteed, forgetting of sensitive information and long-term maintenance of general knowledge are realized, disastrous forgetting in continuous learning is effectively relieved, and the method is applicable to privacy sensitive application scenes such as recommendation systems, medical diagnosis and financial analysis.

The invention provides a code generation method and device based on an up-multiplexing mixed diversity expert large model, and belongs to the field of artificial intelligence, and the method comprises the steps: obtaining code description information; inputting the code description information into a trained code large model, and obtaining a code output by the code large model; the code large model is a large model based on an upper multiplexing mixed diversity expert; the large code model is constructed by adding a shared expert and a plurality of common experts on the basis of the dense model; sharing parameters of an FFN module of the expert multiplexing dense model; the shared experts are used for extracting general knowledge of the lexical elements; a common expert adopts a random initialization mode to initialize parameters; and common experts are used for extracting special knowledge of lexical elements. According to the method, a shared expert and a plurality of common experts are added on the basis of a dense model to construct a code large model, and a diversity multiplexing mechanism is adopted, so that the accuracy of generated codes is improved.

The invention provides a large language model staged pre-training method and system. The method comprises the following steps: training a Transform model by using a basic data set, optimizing a negative logarithm likelihood target, and adopting an AdamW optimizer and a cosine attenuation learning rate; continuing training by using the universal knowledge data set based on the first-stage parameters; and weighting the professional data in the training field by adopting an oversampling strategy. By structuring a training target and a data type, the model can efficiently learn language basis, general knowledge and professional skills in stages. Experiments show that according to the method, the training efficiency of the model in the basic stage is improved by 40%, the overall training time is shortened by 30%, and meanwhile the accuracy rate of tasks in the professional field is 15%-20% higher than that of traditional end-to-end training. Finally, model parameters are evaluated through professional ability, and both universal language understanding and domain specialty are achieved.

The invention discloses a method and device for relieving catastrophic forgetting of a language model, equipment and a storage medium, and relates to the technical field of machine learning. According to the method, linear interpolation is carried out between the pre-training model parameter set and the fine-tuning model parameter set, so that a linear interpolation point of the model can still keep a relatively low loss value when the model learns new task knowledge; then the pre-training model parameter set and the fine-tuning model parameter set are added to form a fusion model parameter set, and the linear interpolation operation only performs one-time simple arithmetical operation after fine-tuning is completed without introducing any additional trainable parameters or changing the model structure, so that when the model adapts to a new task, the model can be quickly and accurately trained. Original general knowledge and performance on old tasks can be reserved to the maximum extent, and meanwhile complexity and expenditure of a model reasoning stage are not increased.

The embodiment of the invention discloses a large model training data generation method, and the method comprises the steps: determining the risk control capability of a target large model and the description of a knowledge field according to a data generation task requirement of a risk control business scene, so as to determine a core concept and a format requirement based on a determined task portrait, the core concept is used for obtaining matched rich knowledge from a universal knowledge base, the format requirement is used for reprocessing the obtained knowledge, and a training sample is obtained by generating a large model to train a target large model for executing risk control business. Through the automatic training sample generation process corresponding to the demand, the existing generation and labeling problem depending on manpower is avoided, the efficiency is improved, the quality difference between samples can be reduced, and the accuracy of the training model is improved.

This invention relates to the field of computer vision technology, providing a method for training a behavior recognition model, a behavior recognition method, and an apparatus. The method and apparatus utilize a frozen pre-trained image-text model (a second video feature extractor and a text encoder) as a general knowledge base to guide a finely tuned student model (a first video feature extractor) with temporal modeling capabilities. A multi-head residual projection network is used for feature-level knowledge transfer and representation alignment. The behavior recognition model trained by this method can accurately understand dynamic behaviors in videos and possesses strong zero-shot reasoning ability (i.e., generalization ability), effectively recognizing behavior categories not seen during the training phase. Furthermore, the entire model framework is clear, training is stable, and the final model exhibits excellent recognition performance for both known and unknown behavior categories under open-vocabulary settings.

The invention relates to the technical field of student intelligent questioning and answering, and discloses a student intelligent questioning and answering method and system based on a large language model and rapid retrieval, and the method comprises the steps: combining a student portrait to generate question vectors, rejecting noise points of a question vector set, obtaining hot questions, and storing the hot questions into a hot knowledge base after the hot questions are expanded by experts; generating a summary of the knowledge document by using a large language model, and adding the summary to each text block; generating a block summary for each text block, and storing the block summary and the text blocks into a universal knowledge base; inputting questions proposed by students into the large language model to generate a thinking chain; for sub-questions needing to be retrieved, firstly retrieving the hotspot knowledge base, if the sub-questions are not matched with the hotspot knowledge base, retrieving the block summary of the general knowledge base, and obtaining corresponding text blocks through mapping to obtain retrieval results; and guiding the big language model to fuse the thinking chain and the retrieval result to generate a structured answer. According to the method, the summary information is introduced after the text blocks are segmented, so that the information loss in the text segmentation process is reduced, and the reasonability of knowledge base construction is improved.

The invention provides a neural symbol collaborative distillation method for improving complex reasoning of a small language model, and the method comprises the steps: firstly, extracting general reasoning capability and general knowledge from a large language model, and distilling the general reasoning capability and general knowledge into parameterized SLMs; secondly, aiming at specific capabilities and rare knowledge required by a complex reasoning task, storing the knowledge in a symbol knowledge base by adopting a symbol knowledge distillation means; by decoupling this general capability from a specific capability, high performance can be achieved with a smaller model scale and cost. In addition, the invention further comprises a multi-task learning method, by introducing auxiliary tasks such as enhanced distillation, answer prediction and direct reasoning prompt, the robustness of the SLMs to possible noise knowledge is further improved, and specific knowledge is effectively utilized. According to the method, the capability of processing complex reasoning tasks by a small language model is enhanced, and the method is suitable for complex reasoning scenes with requirements for general cognitive competence and professional domain knowledge.

The present invention relates to the field of intelligence analysis technology, and in particular, to a method and apparatus for generating an enhanced technical knowledge graph. The method includes extracting technical entities belonging to the types of methods, processes, devices, and functions respectively based on statistical methods and in combination with preset extraction rules; merging the technical entities according to their respective types, performing replacement processing on all technical entities in the specific text data by merging the technical entities, adding relationships to the corresponding merged technical entities, and associating them with the general knowledge graph corresponding to the specific text data to form a knowledge graph of enhanced technology. The present invention has a clear distinction for entities related to the technology itself, can query the knowledge graph from multiple different technical dimensions, and reduces the number of various entities related to the technology through merging, making it clearer in the graph display.

The invention discloses an AI model illusion suppression method and system based on retrieval enhancement generation and cue word engineering collaboration, and aims to solve the illusion problem of an existing large language model and the defects that a traditional scheme is high in cost, low in retrieval content utilization rate, universal knowledge conflicts, high in generation randomness and the like. The method comprises the following steps: constructing a high-precision domain vector index database, performing similarity retrieval and noise filtering on user query, generating a multi-level structured cue word containing role definition, thinking chain constraint and the like, dynamically switching a general knowledge shielding or fusion mode based on recall quality, and outputting a final result through multi-path reasoning sampling and consistency verification. According to the method, underlying model parameters do not need to be modified, the illusion rate is remarkably reduced, generated content is factual and accurate, logic coherence and traceability are achieved, deployment is flexible, cost is low, and the method is suitable for factual question and answer scenes in multiple fields such as medical treatment, industry and legal consultation.