44 results about "General-purpose language" patented technology

The invention relates to the technical field of language translation, in particular to a low-resource language translation method and system based on deconstruction distillation, and the method comprises the steps: obtaining parallel corpus data of a low-resource language and a general language; the method comprises the following steps: constructing a teacher model by taking parallel corpus data as input, constructing a student model trunk based on a pre-trained BERT big language model and optimizing the student model, calculating cross-task attention alignment loss based on the teacher model and the student model, and outputting and executing logits distillation based on the teacher model and the student model. Deployment of low-resource language translation is completed based on the trained student model, a high-quality knowledge migration source is provided for the student model by constructing the BERT teacher model subjected to full-parameter fine adjustment, and meanwhile, the problem of translation precision caused by insufficient low-resource language data is effectively solved by means of dual supervision of cross-task attention alignment and logits distillation.

The invention relates to a knowledge enhancement method and system based on a context structure. The method comprises the following steps: acquiring a query request input by a user; screening candidate evidences related to the query request from a pre-constructed case knowledge base based on multi-view retrieval; performing weighted calculation on a retrieval result of each view to generate an evidence score of each candidate evidence; screening based on the evidence scores to obtain a candidate evidence set and preprocessing the candidate evidence set; calculating the value and length of each candidate evidence in the pre-processed candidate evidence set; constructing an evidence selection problem based on a preset context window value and the value and length of each candidate evidence; solving the evidence selection problem to obtain an evidence subset with maximum value; generating a context structure based on the evidence subset with the maximum value and a preset prompt template; and inputting the context structure into a general language model for knowledge enhancement, and generating a knowledge enhancement result.

The invention discloses a multi-language PLC configuration software component library expansion method and system based on LLVM, and relates to the technical field of configuration software, the method comprises the following steps: obtaining language source codes of various types of programming languages supporting the LLVM front end; inputting the language source code of each type of programming language into a corresponding LLVM front-end compiler, and converting the language source code into a standard intermediate representation; establishing a general language characteristic analysis framework; establishing a general abstract metadata injection framework; building a mapping rule framework; integrating the general language framework, the general abstract metadata injection framework, the mapping rule framework and the output result, and generating a unified extended intermediate representation; and generating an interface component library conforming to the IEC standard. According to the method, any programming language with an LLVM front end is supported, a developer can directly utilize a mature algorithm library in each language ecosystem to select the programming language most suitable for a specific application scene, and the development period is remarkably shortened.

The application discloses a kind of data knowledge extraction method, system, equipment and storage medium based on large language model, method includes: the original data obtained is preprocessed;Feature is extracted from original data, select corresponding task feature, remove the feature of high correlation by calculating the correlation coefficient matrix between features;Select the large language model of task, capture general language and knowledge, fine-tune the large language model on the labeled data set of specific field or task;Based on large language model inference and prompt word engineering, knowledge source information is extracted using a secondary block iteration method, and the context understanding ability of the large language model after fine-tuning is used to integrate the information extracted from different data sources;Define evaluation criteria, analyze the error cases of the large language model, identify the improvement direction, and adjust the model parameters and training data according to the evaluation results and error analysis.The application has the advantages of improving the efficiency and accuracy of the knowledge extraction method.

The specification discloses a patent retrieval method, device and equipment for multiple retrieval scenarios. Patent field language models are obtained by training general language models with fields of patent documents as training data. For multiple retrieval scenarios, positive and negative samples of each patent document are screened through different weight multi-path recall and random sampling, and training data of a patent semantic representation model of each retrieval scenario is constructed to train the patent field language models, and then the patent semantic representation model of each retrieval scenario is obtained to generate semantic vectors of each field of the patent, and the semantic vectors are stored in a patent retrieval vector database of each retrieval scenario. According to the semantic vectors of the retrieval fields, similar patent documents are searched from the patent retrieval vector database of the target retrieval scenario. Through the above scheme, the semantic representation ability of the patent semantic representation model for the patent fields of different retrieval scenarios is improved, and the retrieval precision is improved.

A small sample based general image counting method, comprising: performing feature extraction on a first image to obtain first features; performing attention calculation based on the first features, a general language representation and a general visual representation to obtain first example features of a first example target, wherein the general language representation is used to describe categories of different objects, the general visual representation is used to describe visual information of different objects, the first example target is an object related to an example frame in the first image, and the first example features include a first language representation and a first visual representation, the first language representation is used to describe a category of the first example target, and the first visual representation is used to describe visual information of the first example target; performing matching on the first example features and the first features to obtain a correlation feature map; and obtaining a first counting result related to the first example target based on the correlation feature map. The method can greatly improve the generalization of the counting algorithm.

The invention provides a large language model staged pre-training method and system. The method comprises the following steps: training a Transform model by using a basic data set, optimizing a negative logarithm likelihood target, and adopting an AdamW optimizer and a cosine attenuation learning rate; continuing training by using the universal knowledge data set based on the first-stage parameters; and weighting the professional data in the training field by adopting an oversampling strategy. By structuring a training target and a data type, the model can efficiently learn language basis, general knowledge and professional skills in stages. Experiments show that according to the method, the training efficiency of the model in the basic stage is improved by 40%, the overall training time is shortened by 30%, and meanwhile the accuracy rate of tasks in the professional field is 15%-20% higher than that of traditional end-to-end training. Finally, model parameters are evaluated through professional ability, and both universal language understanding and domain specialty are achieved.

The invention discloses an NL2SQL-based power intrinsic safety question-answering system construction method, system and device and a medium, and belongs to the technical field of power safety. The method comprises the following steps: collecting fine tuning data and constructing a fine tuning data set; performing low-rank adaptive fine tuning on a pre-trained large language model through the fine tuning data set to obtain a fine tuning large language model, and triggering an annealing strategy for optimization; building prompts through the fine-tuning large language model in combination with user questions and database structure information; and generating a structured query language statement based on the prompt, querying a database to obtain a query result, and adjusting the prompt according to the query result to process an error. According to the method, low-rank fine tuning in the field of electric power intrinsic safety is carried out on a general pre-training large model by introducing the LoRA technology, so that the model has general language understanding ability and industry exclusive knowledge at the same time, and the problem that a traditional model is high in universality but low in specialty is solved.

This application discloses a method and related equipment for identifying hot words in a vehicle-mounted scene, relating to the field of Internet technology. The method includes: acquiring user-input voice data; extracting hot word text from the voice data; and acquiring fusion features between the voice data and the hot word text; parsing the hot word text to generate multiple mapping paths; constructing a hot word language model based on the multiple mapping paths; decoding the fusion features based on the hot word language model to obtain a hot word decoding result; decoding the fusion features based on the general language model to obtain a general decoding result; and determining a target recognition result based on the hot word decoding result and the general decoding result.

Some implementations described herein provide techniques and apparatuses for software parameter management through a universal interface. The techniques and apparatuses include a user device including a universal parameter management application. The user device may receive multiple sets of changeable parameters that may be provided to the user device using different formats, different machine languages, and / or different language scripts. The universal parameter management application may translate the multiple sets of changeable parameters into a common format, common machine language, and / or common language script. The user device may then provide each of the multiple sets through a common, changeable graphical representation of the changeable parameters on a graphical user interface of the device. The user may change each of the multiple sets of changeable parameters through an input that changes the common, changeable graphical representation.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a fine tuning model construction and multi-round incremental training framework oriented to power grid innovation management, which is characterized in that power grid multi-source heterogeneous data is introduced to carry out pre-training on the basis of a general pre-training model, and the core is to design a text-time sequence joint masking loss function to obtain a fine tuning model; while the understanding capability of the universal language / time sequence structure of the model is reserved, the model deeply masters the professional characteristics of the power grid, and a basic model with both the universal adaptability and the professional property of the power grid field is constructed. According to the invention, through modularization, low-rank increment, dynamic sparse and rolling increment training and strict online risk management and control, an efficient, controllable and sustainable evolution model training framework oriented to a multi-service scene of the smart power grid is constructed, a landing engineering solution is provided for stable, efficient and sustainable operation of the smart power grid, and the development of the smart power grid is facilitated. The method plays an important role in promoting digitization and intellectualization of a power grid.

This invention discloses a heterogeneous hybrid expert large model architecture and alignment method for high-concurrency online medical dialogue. First, the method proposes a dual-pool heterogeneous routing strategy of "4 general experts + 2 departmental experts." Through physically isolated parallel routing channels, it forces the model to simultaneously activate general language capabilities and specialized medical reasoning when processing input, fundamentally ensuring the focus and professionalism of responses. Second, it adopts a structural evolution strategy based on parameter expansion and selective freezing. By completely freezing the pre-trained basic parameters and training only newly added departmental experts, it completely eliminates catastrophic forgetting. Finally, in the model alignment stage, it employs the Group Relative Policy Optimization (GRPO) algorithm for valueless models to reduce memory overhead and designs a dual-track hybrid reward function combining low-level semantic similarity rewards and high-level structured large model referee scoring, effectively solving the "reward hacking" problem and guiding the model to generate refined responses that combine clinical accuracy and humanized interaction. This invention significantly improves the performance of medical dialogue systems. Its sparse activation and parameter freezing characteristics further ensure low latency and high throughput in online high-concurrency scenarios, providing a complete solution for the deployment of reliable medical dialogue systems.

The application provides a pre-training language model fine-tuning method, device and computer readable storage medium, and belongs to the technical field of natural language processing. The pre-training language model fine-tuning method comprises the following steps: obtaining a pre-trained pre-training language model, determining the gradient threshold of each encoding layer in the pre-training language model; training the pre-training language model, and calculating the gradient norm of any encoding layer before updating the model parameters of the encoding layer each time in the training process; and determining whether the model parameters of the encoding layer need to be updated this time according to whether the gradient norm of the encoding layer is less than the gradient threshold of the encoding layer. The application can achieve a better compromise in retaining the general language knowledge of the pre-training language model and training a model suitable for a downstream task, so that the model can retain more general language knowledge and improve the performance of the model.

The invention relates to a knowledge distillation optimization-based semantic bidirectional encoder method special for the field of open source software supply chain security. The method comprises the following steps: acquiring security data of an open source software supply chain; inputting the security data into a pre-trained student model to obtain a security analysis result of the open source software supply chain; the security analysis result comprises a vulnerability detection result; the pre-trained student model is a language model obtained by training according to the sample security data, the first sample label and the second sample label; the first sample label is a sample semantic understanding result output by a plurality of teacher models according to the sample security data, and the second sample label is a sample vulnerability detection result; the plurality of teacher models include a first teacher model for semantic understanding of a general language, a second teacher model for semantic understanding of a secure text, and a third teacher model for semantic understanding of a code. By adopting the method, the security analysis accuracy of the open source software supply chain can be improved.

This invention discloses a method, apparatus, terminal device, and storage medium for TCM knowledge question answering based on retrieval enhancement generation, belonging to the field of artificial intelligence. The method comprises: encoding a TCM-related query input by a user to obtain a first encoding vector; invoking a retrieval engine in a preset retrieval enhancement generation platform, causing the retrieval engine to calculate similarity based on a built-in TCM knowledge vector database and the first encoding vector, sorting all similarity scores from highest to lowest, and selecting several top-ranked TCM prescription files as target TCM prescription files; the TCM knowledge vector database stores several TCM prescription files; and inputting the target TCM prescription files and the query statement into a preset large language model to obtain TCM knowledge question answering results. By implementing this application, the problem that existing general-purpose language models cannot meet users' needs for professional TCM consultation can be solved.

This application provides a method, apparatus, storage medium, and electronic device for automated identification of customs declaration documents, relating to the field of automated information processing technology. The method includes: identifying the document type of the documents to be identified based on a general language model, determining the document type of each document among multiple documents to be identified; converting each document to be identified into a document image, and performing image preprocessing operations on each document image to obtain a target document image for each document to be identified; and using a visual language model corresponding to the document type to identify and extract key information from the target document image of each document to obtain the target information corresponding to each document to be identified. The automated identification method for customs declaration documents provided by this application can quickly extract the required information from the original customs declaration documents and convert it into structured data for subsequent processing and analysis, greatly reducing labor costs and improving customs declaration efficiency.

The invention provides a method and system for enhancing online speech recognition based on online OCR, and relates to the technical field of speech recognition, and the method comprises the steps: capturing a shared digital image in a collaborative session online in real time, and extracting a text in the shared digital image; performing word segmentation processing on the extracted text to obtain a first vocabulary set; training and generating a session exclusive language model corresponding to the collaborative session based on the first vocabulary set; fusing the session exclusive language model and a pre-created universal language model to obtain a corresponding fused language model; performing conversion processing on the fused language model to obtain a corresponding language model decoding graph; in the decoding stage, a language model decoding graph and a pre-created acoustic vocabulary decoding graph are loaded, the two loaded decoding graphs are dynamically decoded in real time, a text recognition result of the voice in the collaborative session is obtained, and the acoustic vocabulary decoding graph is created in advance based on a universal acoustic model and a universal language model. The objective of the invention is to improve speech recognition precision and efficiency.

The present disclosure provides a training method for a large language model, a relevance determination method, and related devices. The present disclosure relates to the field of artificial intelligence technology, particularly to technical fields such as intelligent search, intelligent recommendation, deep learning, and large models, and can be used in application scenarios such as generative search, intelligent document editing, intelligent assistants, virtual assistants, and intelligent e-commerce. The specific implementation scheme is as follows: obtaining a first type of task sample and a second type of task sample, wherein the task type of the first type of task sample is a relevance task, and the relevance task is used to determine search relevance; the task type of the second type of task sample is a general language task; prompting the large language model with the task type to be performed based on the task instruction, so as to train the large language model based on samples of different task types.

The application relates to a text management method and device, computer equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: obtaining a target text, performing data preprocessing on the target text to obtain a semantic vector corresponding to the target text; classifying the target text according to the semantic vector and a trained general language model to obtain a target classification result of the text type of the target text; determining a target storage path corresponding to the target classification result according to a mapping relationship between a storage path and a classification result; and storing the target text according to the target storage path. The method can improve the document management efficiency.

A system is provided.SOLUTION: A system comprising: means for receiving information entered in different languages; means for translating the received information into a common language; means for checking and modifying grammar of the translated information; means for consolidating a plurality of information; means for summarizing the consolidated information; and means for reporting the summarized information to an administrator.SELECTED DRAWING: Figure 1

This section describes a system and techniques for processing spoken language for use as input to a control system. After an utterance is received from a user in a general language, a model of a generative neural network is invoked to transform the utterance into an expression corresponding to a domain-specific language. The expression in the domain-specific language is then fed to a control system that accepts expressions of the domain-specific language as input and controls a device based on that input.

The invention discloses a semiconductor-oriented semantic map construction and natural language query method and system, and belongs to the field of natural language processing. The method comprises the following steps: constructing a semantic map offline, extracting metadata of a semiconductor production line database, mapping the metadata into a map skeleton, and creating Data node enhancement in combination with field data; analyzing a natural language problem on line, extracting entity links to atlas anchor points, and expanding and pruning to obtain context sub-graphs; after the sub-graphs are serialized, the sub-graphs, NLQ and task instructions form dynamic cue words, and the dynamic cue words are input into a large language model to generate SQL; the system comprises an atlas construction module, a subgraph retrieval module, a cue word generation module, an SQL generation module and a data storage module. According to the method, the relationship between entities can be clearly represented, domain ontology knowledge is fused, professional terms and general languages are distinguished, LLM reasoning difficulty is reduced, SQL generation precision is improved, and the method is particularly suitable for knowledge-intensive scenes such as semiconductor manufacturing.

An embodiment of the present invention discloses a method, apparatus, device, and storage medium for recognizing speech data, wherein the method includes: obtaining speech data input by a target user and location information of the target user; determining a target region language model corresponding to a target region to which the location information belongs from a plurality of region language models, wherein any region language model of the plurality of region language models is trained based on information points included in the any region; calling the target region language model and a universal language model to perform a first decoding process on the speech data to obtain N candidate recognition results; calling the target region language model and a neural network language model to perform a second decoding process on each of the N candidate recognition results to select a target recognition result from the N candidate recognition results, thereby improving the recognition accuracy of the speech data.

A system, article, and method of automatic context-bound domain-specific speech recognition uses general language models.

An online system automates the generation and presentation of video content for physical objects. The system stores object data and 3D models for users, including models of objects for sale. Users can generate and manage visual markers on 3D models to highlight specific features. The system generates videos depicting the physical objects using the stored models and object data, and produces synchronized audio voiceovers describing the objects' features. Script generation prompts are created based on object data and input to a generative language model to generate scripts for the voiceover. The system generates voiceover audio data using text-to-speech or generative AI models and combines the audio with the video. The resulting video and audio data are transmitted to a client device for user presentation and interaction. The system provides an integrated workflow for creating, editing, and presenting detailed, object-specific video content with automated narration.

The application provides a multi-model fusion vertical field knowledge question answering method and system, belonging to the field of artificial intelligence natural language processing. The scheme disassembles the traditional single large model into multiple specialized models for collaborative work: first, an enhanced small parameter model trained for vertical fields is used for document analysis, and fixed position information is extracted in combination with rule assistance; then a general language model with lower parameters is used for intent understanding, supplemented by a prompt word strategy, a rule model or a knowledge graph; then based on the analyzed document information and the understood user intent, a larger parameter model is used for knowledge retrieval and answer generation; finally, a special small parameter model is used for style optimization to generate answers that meet the characteristics of the vertical field. The application also designs an answer backflow mechanism to realize continuous optimization of the system. The scheme greatly reduces resource consumption, improves the accuracy and efficiency of vertical field knowledge question answering, and solves the "hallucination" problem existing in the prior art.

The application provides a cross-language abstract generation method based on a multi-language model, and relates to the technical field of cross-language abstract generation, and the method comprises the following steps: acquiring multi-language text data; pre-training a general language model based on the multi-language text data to obtain a multi-language model; acquiring multi-language abstract data, learning the multi-language model based on the multi-language abstract data to obtain a multi-language abstract generation model; acquiring to-be-processed text data, inputting the to-be-processed text data into the multi-language abstract generation model to generate abstracts corresponding to the to-be-processed text data in multiple different languages. The application realizes cross-language abstract generation, can generate abstracts in different languages, can summarize the gist of the text at the same time, and improves the precision of the abstracts.

The invention discloses a special model fine tuning and training method for a generative marketing scheme in the field of content marketing. The method comprises the following steps: collecting and preprocessing multi-source marketing data; performing feature extraction and semantic annotation to generate a semantic annotation data set; calling a pre-trained universal language generation model, and outputting an adaptive generation model; performing supervised fine tuning training to obtain a fine tuning model; weight parameters of the fine tuning model are optimized through an improved RL-CEM algorithm, and an enhanced training generation model is obtained; outputting a marketing generation content set; establishing a model evaluation and feedback mechanism, and generating a final special generation model; according to the method, marketing release feedback data is collected in real time to trigger a retraining mode, a self-adaptive model training mechanism is formed, self-adaptive model optimization is achieved, the semantic consistency, the marketing matching degree and the conversion effect of generated content can be effectively improved, and the content marketing automation and intelligence level is remarkably improved.

The application relates to the technical field of language translation, in particular to a low-resource language translation method and system based on a deconstructive distillation, the method comprising the following steps: acquiring parallel corpus data of a low-resource language and a general language; constructing a teacher model by taking the parallel corpus data as input; constructing a student model trunk based on a pre-trained BERT large language model and optimizing the student model; calculating a cross-task attention alignment loss based on the teacher model and the student model; executing logits distillation based on the teacher model and the student model output; and completing the deployment of low-resource language translation based on the trained student model. The application provides a high-quality knowledge migration source for the student model by constructing a BERT teacher model fine-tuned by all parameters, and effectively solves the translation precision problem caused by the lack of low-resource language data by means of double supervision of cross-task attention alignment and logits distillation.