33 results about "Language module" patented technology

A system for creating functionality modules for deployment in a workflow for use in visual programming including a configuration server with a processing element operable to implement the functionality modules and workflow, at least one large language model, a customizable functionality module in a workflow including at least one interface defining one or more customizable properties, and wherein the workflow executes a first operational environment different from a second operational environment executed by the large language models.

The embodiment of the invention relates to the technical field of large language modules, and discloses a multi-task processing method of a large language model fused with LoRA, which comprises the following steps: receiving task processing information input by a user, and performing feature extraction on the task processing information to determine each scene task feature; performing data matching with a pre-LoRA module parameter set based on the scene task features to obtain a LoRA module parameter set associated with the scene task parameters, and performing parameter updating operation on the large language model according to the LoRA module parameter set; and inputting the scene task features into the large language model subjected to the parameter updating operation for identification to obtain corresponding identification result information, and outputting the corresponding identification result information. According to the scheme of the invention, by fusing the LoRA module and the routing mechanism, the adaptive selection and optimization capability for specific tasks is realized, and the performance of the model in processing diversified tasks is improved.

A speech module is joint trained with a knowledge module by transforming a first knowledge graph into an acoustic knowledge graph. The knowledge module is trained on the acoustic knowledge graph. Then, the knowledge module is integrated with the speech module to generate an integrated knowledge-speech module. In some instances, the speech module included in the integrated knowledge-speech module is aligned with a language module to generate an optimized speech model configured to leverage acoustic information and acoustic-based knowledge information, along with language information.

The invention relates to the technical field of engineering process control, and discloses a ball mill granularity soft measurement method based on a time sequence large model. Constructing a multi-feature fusion module for extracting multi-scale features based on Convld K3, Convld K5 and Convld K1, and constructing a soft measurement model in combination with multi-head attention and a large language module with a fixed weight; the soft measurement model generates a first feature and a query matrix, generates a key matrix and a value matrix according to a fixed weight of the large language module, generates a second feature based on the query matrix, the key matrix and the value matrix, and fuses the second feature with the first feature to obtain a fused feature; and then a granularity prediction result corresponding to the field data is obtained through a large language module, so that the problems that an existing soft measurement model is too high in dependence on large-scale sample data, insufficient in modeling capability for complex nonlinear process parameters and low in prediction precision are solved.

The invention discloses a Manchu file word recognition method based on a vision-language model. The method comprises the steps of constructing an initial feature module, a stroke extraction module, a vision module, a language module, a modal alignment module and a fusion module; inputting the Manchu word picture into an initial feature extraction module to obtain an initial image feature, inputting the feature into a stroke extraction module and a visual module to obtain a stroke feature and a visual feature, inputting the visual feature into a language model, and outputting a language feature; the stroke features, the visual features and the language features are fused through a multi-mode Transform method, and a final result is controlled and output through a gating mechanism; according to the method, the recognition precision of Manchu characters with covering and staining and Manchu words with large background interference can be effectively improved.

A computer-implemented method (100) for controlling a vehicle (400), comprising providing (110) an environment module (220) configured to determine an environment model based on environment information characterizing an environment of the vehicle (400); determining (120), by means of the environment module (220), an actual environment model of the vehicle (400) based on actual environment information of the vehicle (400); converting (130) the actual environment model into an actual language model, wherein the actual language model describes the environment of the actual environment model in a semantic language; providing (140) a language module (230) and inputting (150) the actual language model into the large language module (230); Using (160) the language module (230) to plan and / or explain a movement trajectory of the vehicle (400) based on the actual language model.

The invention discloses a remote sensing visual language agent construction method and device based on a hybrid strategy. The method comprises the following steps: constructing a multi-modal data set in the field of remote sensing; taking the multi-modal data set as a supervision signal, and performing supervision fine tuning on the visual language model to obtain an initial visual language model; constructing a reward mechanism, sampling the initial visual language model for multiple times, and determining an advantage value of each sampling according to the reward mechanism; converting the advantage value of each sampling into a relative advantage value, and optimizing parameters of the initial visual language model based on the relative advantage value to obtain a remote sensing visual language module; and receiving a remote sensing task input by a user through a remote sensing visual language module, generating a natural language answer, and synchronously outputting a thinking chain. The problems that an existing visual language model is high in sample requirement and poor in generalization ability and performance are solved. Therefore, various downstream tasks in the remote sensing field are solved, and the generalization ability and the processing efficiency of the model in complex remote sensing tasks are remarkably improved.

The invention discloses an auxiliary diagnosis system, an auxiliary diagnosis method, an auxiliary diagnosis program product and auxiliary diagnosis equipment for heart valve alignment repair, which can improve the safety and the effectiveness of a TAVR operation by analyzing operation images and problems of doctors, automatically answering related diagnosis, treatment and explaining problems and providing real-time operation suggestions and optimization schemes. The auxiliary diagnosis system is based on a trained basic visual question and answer model and comprises an input module, a visual module, a language module and an output module. The input module receives medical images and natural language questions as system inputs. The visual Transform model is used for processing the medical image and extracting visual features of the image; processing the question by a language Transform model, and generating an answer based on the visual features extracted by the visual module; and the output module outputs the generated answer in a text or voice form.

The invention is applied to the technical field of dialogue interaction systems, and particularly discloses an induced dialogue interaction system based on a psychological intervention artificial intelligence model, which comprises a voice recognition module, a deep language module, a voice output module, a closed feedback module and an abnormal condition processing module, the voice recognition module is connected with the deep language module through wireless signals, and the deep language module is connected with the voice output module through wireless signals. According to the induced dialogue interaction system based on the psychological intervention artificial intelligence model, psychological theories such as a social emotion selection theory, a communication adaptation theory and the like are introduced in the system, and an old people age layering optimization deep language model is combined, so that the recognition and fitness reply ability of the model to common emotion expression of old people is enhanced; meanwhile, by means of intervention boundary setting, psychological intervention scenes of the old people are accurately adapted, and intervention safety and effectiveness are guaranteed.

The present invention discloses a book recommendation method based on a deep-wide model, the method comprising the following steps: constructing a data set, inputting the data set into two pre-trained language modules, obtaining the hidden state representation of the last layer of the two pre-trained language modules, averaging the hidden state representation of the last layer of the two pre-trained language modules, obtaining a comprehensive representation vector of the entire input, inputting the data set into an interest extraction module for processing, obtaining an embedded representation vector of the embedding layer, grouping and splicing the embedded representation vector of the embedding layer, obtaining the output of the interest extraction module, fusing the comprehensive representation vector of the entire input and the output of the interest extraction module, and obtaining the probability of the reader for the candidate borrowing book. The present invention adds an attention mechanism to the deep-wide model, and the attention mechanism dynamically assigns weights according to the impact of each behavior on the user's preference, thereby accurately identifying the user's key interests and significantly enhancing the recommendation capability.

The application relates to an explainable language model training method and system for a knowledge-intensive task, and the method comprises the following steps: S1, acquiring original text data; the original text data is a question proposed according to a knowledge-intensive task; S2, inputting the original text data into a preset large language model for training; S3, adopting a hierarchical functional circuit recognition algorithm to recognize a functional circuit in the trained preset large language model; S4, calculating the correlation degree of the functional circuit and the knowledge-intensive task, wherein the correlation degree is a weighted combination based on a selectivity index and an influence index; and S5, outputting an explainable language module, wherein the explainable language module comprises the functional circuit and the corresponding correlation degree. The functional circuit inside the large language model is induced and recognized, mechanism-level transparency is provided, and the method is far superior to existing surface explanation methods.

This invention relates to the field of flood remote sensing image understanding, specifically disclosing a method and system for language understanding of flood remote sensing images based on a priori-guided large-scale model. The method includes: constructing a dataset containing multi-temporal flood remote sensing images and multi-granularity linguistic descriptions; constructing and training a flood prior model to extract prior information on the spatial distribution of floods and convert it into textual prior information; constructing a large-scale language model containing a visual encoder, a projection layer, and a language module, and training it by inputting the images, textual descriptions, and textual prior information together; inputting the image to be understood and the corresponding textual prior information into the trained large-scale language model, and outputting the language description result. This invention, by introducing a flood prior model to provide explicit spatial distribution guidance, effectively alleviates the semantic understanding bias and spatial positioning error of general visual language models in flood scenarios, significantly improving the accuracy of linguistic descriptions of flood images.

This invention discloses an auxiliary diagnostic system, method, program product, and device for transcatheter aortic valve repair (TAVR). By analyzing surgical images and physician questions, it automatically answers relevant diagnostic, treatment, and interpretation questions, and provides real-time surgical suggestions and optimization plans, thereby improving the safety and effectiveness of TAVR surgery. The auxiliary diagnostic system is based on a pre-trained visual question-answering model and includes an input module, a visual module, a language module, and an output module. The input module receives medical images and natural language questions as system input. The visual Transformer model processes the medical images and extracts visual features; the language Transformer model processes the questions and generates answers based on the visual features extracted by the visual module; the output module outputs the generated answers in text or speech form.

An embodiment of the present invention provides a speech recognition method and system, belonging to the field of speech recognition. The method includes: respectively extracting corresponding sample speech feature vectors from the preprocessed sample speech through a feature extraction module to be trained, and extracting a feature representation and a temporal dependence relationship of the speech from the sample speech feature vectors; inputting the mapping representation and the temporal dependence relationship of the speech into a language module to be trained to form a plurality of probability factor sequences; for each probability factor sequence, taking the probability factor sequence with the highest matching degree as the language text of the sample speech signal and outputting it until the output language text conforms to the marked text corresponding to the sample speech signal, obtaining a trained speech recognition model. The speech recognition model is used to receive a speech signal to be recognized and output a corresponding language text, improving the speech recognition accuracy and generalization ability.

The invention provides a thread compatible scheduling method, a multi-language mixing system, a multi-language mixing device and related equipment, and relates to the technical field of computer control. According to the method, tasks are uniformly submitted to a central scheduling unit through a language adaptation unit; the central scheduling unit periodically receives and maintains the thread resource state of each heterogeneous language module; the central scheduling unit performs scheduling decision making according to thread resource states, including preferentially scheduling to an initiation language module, scheduling to other sufficient modules supporting the function when resources are insufficient, or adding tasks into a waiting queue; after the task is distributed to the target module, the language adaptation unit creates a thread and binds a task context to execute the task; and after the task is completed, an execution result is reported to the central scheduling unit. The method aims at solving the problems that in the prior art, thread scheduling splitting, resource conflicts, context breakage and the like cannot be solved in the multi-language environment, and the performance, stability and maintainability of a high-concurrency system can be improved.

A system for creating functionality modules for deployment in a workflow for use in visual programming including a configuration server with a processing element operable to implement the functionality modules and workflow, at least one large language model, a customizable functionality module in a workflow including at least one interface defining one or more customizable properties, and wherein the workflow executes a first operational environment different from a second operational environment executed by the large language models.

The invention discloses a generative adversarial network data expansion method based on entropy item influence and a large language model. The large language model comprises a large language module, a multi-agent module, a knowledge base, an inference engine and a real-time interaction interface. The large language model uses a pre-training language model as a master control and is used for processing natural language understanding and generation input by a user; the multi-agent module is provided with a plurality of functional agents which are used for independently completing specific tasks; domain-specific knowledge is stored in the knowledge base and the inference engine; the real-time interaction interface is used for providing an entrance for interaction between a user and the system. Wherein the generative adversarial network data expansion method based on entropy item influence performs data expansion, and then inputs the data to the pre-training language model for training. According to the method, the performance and the data distribution simulation capability of the large oracle model on scarce category data generation are improved, and meanwhile, modularized and parallelized efficient task processing is realized.

The invention relates to the field of language model pruning, and particularly discloses a task-adaptive visual language large model collaborative pruning method, which comprises the following steps of: S1, performing task perception, acquiring a training data set containing image text pairs, shielding a corresponding data set of a single mode, and calculating loss value fixed mode dependency so as to adaptively bias and finely adjust visual and language module shares; s2, calculating importance scores, dividing model weights, hierarchically calculating gradient norm approximate values, and multiplying the gradient norm approximate values with modal shares to obtain scores of all layers; s3, sparseness is distributed, summarizing scores are normalized, the sparseness of each layer is calculated in combination with a total parameter retention proportion, and a threshold value is set; and S4, structured sparseness is carried out, a pruning strategy and token screening are combined, a fusion score of modal correlation degrees is calculated, effective tokens are reserved to calculate channel scores, and key channels are reserved according to groups. According to the technical scheme, the problems that traditional pruning is poor in adaptation, task modal dependence is not considered, and key parameters are easily cut by mistake are solved, compression and reasoning efficiency is improved, cross-modal information loss is avoided, and performance loss is reduced.

This invention relates to the field of engineering process control technology and discloses a soft measurement method for ball mill particle size based on a time-series large model. A multi-feature fusion module for extracting multi-scale features is constructed based on Convld K3, Convld K5, and Convld K1. A soft measurement model is built by combining multi-head attention and a large language module with fixed weights. Field data of ball mill grinding is acquired and input into the soft measurement model. The soft measurement model generates a first feature and a query matrix. A key matrix and a value matrix are generated according to the fixed weights of the large language module. A second feature is generated based on the query matrix, key matrix, and value matrix, and fused with the first feature to obtain the fused feature. The particle size prediction result corresponding to the field data is then obtained through the large language module. This method solves the problems of existing soft measurement models, such as excessive reliance on large-scale sample data, insufficient modeling ability for complex nonlinear process parameters, and low prediction accuracy.

This application discloses a method and apparatus for constructing a remote sensing visual language agent based on a hybrid strategy. The method includes: constructing a multimodal dataset in the remote sensing domain; using the multimodal dataset as a supervisory signal to fine-tune a visual language model under supervision, obtaining an initial visual language model; constructing a reward mechanism to sample the initial visual language model multiple times, and determining the advantage value for each sample based on the reward mechanism; converting the advantage value for each sample into a relative advantage value, and optimizing the parameters of the initial visual language model based on the relative advantage value, obtaining a remote sensing visual language module; receiving remote sensing tasks input by the user through the remote sensing visual language module, generating natural language answers, and simultaneously outputting the thought chain. This solves the problems of existing visual language models having high sample requirements and poor generalization ability and performance. Furthermore, it addresses various downstream tasks in the remote sensing domain, significantly improving the model's generalization ability and processing efficiency in complex remote sensing tasks.

Aspects include methods, systems, and computer-program products providing virtual assistant domain functionality. A natural language query including one or more words is received. A collection of natural language modules is accessed. The collection natural language modules are configured to process sets of natural language queries. A natural language module, from the collection of natural language modules, is identified to interpret the natural language query. An interpretation of the natural language query is computed using the identified natural language module. A response to the natural language query is returned using the computed interpretation.

A system for creating functionality modules for deployment in a workflow for use in visual programming including a configuration server with a processing element operable to implement the functionality modules and workflow, at least one large language model, a customizable functionality module in a workflow including at least one interface defining one or more customizable properties, and wherein the workflow executes a first operational environment different from a second operational environment executed by the large language models.

The application discloses a method and device for translating a source language into a target language, and the method comprises the following steps: acquiring at least one first source language module in a source language document, wherein the at least one first source language module is any one or more source language modules in the source language document; translating the at least one first source language module into at least one first target language module corresponding to the at least one first source language module, and generating a correlation relationship file between the source language document and a target language document, wherein the target language document comprises the at least one first target language module; and if any one first source language module in the source language document is updated, updating the first target language module corresponding to the first source language module according to the correlation relationship file, so that the translation efficiency is improved compared with manually searching for an update point, and the update point is not missed, and the translation quality is improved.

A method of controlling a household appliance with a language module comprising a large language model (with a language module to control the household appliance), the method comprising:-the language module receiving an input query; -the language module determines whether the input query comprises a task-based query; -the language module translates the task-based query into a machine syntax, wherein the machine syntax is understandable by the home appliance; the language module outputs the machine grammar; -the household appliance receiving the machine syntax; and-the household appliance performs the machine syntax.

Aspects include methods, systems, and computer-program products providing virtual assistant domain functionality. A natural language query including one or more words is received. A collection of natural language modules is accessed. The collection natural language modules are configured to process sets of natural language queries. A natural language module, from the collection of natural language modules, is identified to interpret the natural language query. An interpretation of the natural language query is computed using the identified natural language module. A response to the natural language query is returned using the computed interpretation.

The present invention discloses a method for extracting junior high school mathematics knowledge point relationships based on a knowledge graph, which includes the following steps: constructing a data set for preprocessing to obtain sentence text data, inputting the sentence text data into a pre-trained language module for processing to obtain text semantic feature encoding vectors, inputting the data set into a pre-trained language module based on knowledge representation learning to obtain entity feature encoding vectors and a set of relationship type feature encoding vectors, inputting the text semantic feature encoding vectors, entity feature encoding vectors and the set of relationship type feature encoding vectors into a local entity semantic fusion module to obtain relationship-semantic feature encoding vectors, and inputting the relationship-semantic feature encoding vectors into a classifier module for prediction. By fully fusing the set of relationship type feature encoding vectors, sentence text data and entity feature encoding vectors, this method improves the accuracy of the relationship extraction effect.

The application provides a thread compatible scheduling method, a multi-language mixed system, a device and related equipment thereof, and relates to the technical field of computer control. The method submits tasks to a central scheduling unit through a language adaptation unit; the central scheduling unit periodically receives and maintains the thread resource state of each heterogeneous language module; the central scheduling unit makes scheduling decisions according to the thread resource state, including preferentially scheduling to the initiating language module, or scheduling to other abundant modules supporting the function when resources are tight, or adding the task to a waiting queue; after the task is distributed to the target module, the language adaptation unit creates a thread and binds the task context to execute the task; after the task is completed, the execution result is reported to the central scheduling unit. The method aims to solve the problems of thread scheduling splitting, resource conflict, context rupture and the like in a multi-language environment that cannot be solved by the prior art, and is beneficial to improving the performance, stability and maintainability of a high-concurrency system.

The invention discloses a book recommendation method based on a profound model, and the method comprises the following steps: constructing a data set, inputting the data set into two pre-training language modules, and obtaining the hidden state representation of the last layer of the two pre-training language modules, performing average pooling on hidden state representations of the last layer of the two pre-training language modules to obtain a comprehensive representation vector of the whole input, inputting a data set into an interest extraction module for processing to obtain an embedded representation vector of an embedded layer, performing grouping splicing on the embedded representation vector of the embedded layer to obtain an output of the interest extraction module, and outputting the output of the interest extraction module. And fusing the overall input comprehensive representation vector and the output of the interest extraction module to obtain the probability of the candidate borrowed books by the reader. According to the method, the attention mechanism is added in the deep model, and the attention mechanism dynamically allocates the weight according to the influence of each behavior on the user preference, so that the key interest of the user is accurately identified, and the recommendation ability is remarkably enhanced.

The present application relates to the field of language model pruning, and specifically discloses a task-adaptive visual language large model collaborative pruning method, comprising: S1, task perception, obtaining a training data set containing image text pairs, shielding the corresponding data set of a single mode, calculating the loss value and the mode dependency, and adaptively biasing the visual and language module share; S2, calculating the importance score, dividing the model weight, calculating the gradient norm approximation in layers, multiplying the mode share to obtain the score of each layer; S3, allocating sparsity, normalizing the score, calculating the sparsity of each layer combined with the total parameter retention ratio and setting a threshold; S4, structured sparsity, combining pruning strategy and token screening, calculating the fusion score of mode correlation, calculating the channel score by retaining effective tokens, and retaining key channels according to groups. The technical scheme provided by the present application solves the problems of poor adaptation of traditional pruning, not considering task mode dependency, and easy mis-pruning of key parameters, improves compression and inference efficiency, avoids cross-modal information loss, and reduces performance loss.

The invention discloses an asymmetric fine tuning method and system for a visual language model, and a storage medium, and the method employs an asymmetric decoupling fine tuning strategy for the structural heterogeneity of a visual and language module of a large visual language model: unfreezing a visual encoder, carrying out the fine tuning of a layer normalized affine parameter in a Transform block, and calibrating the feature distribution offset caused by high-resolution input; for the language model backbone, a learnable Switch GLU-Adapter module is inserted through residual parallel connection, and the fine-grained reasoning capability is enhanced; only a small number of key parameters of the model are updated, and other pre-training parameters are frozen; in a high-resolution visual question and answer task, the method not only adapts to differentiated requirements caused by heterogeneity of modules, but also remarkably reduces training calculation and storage overhead, avoids disastrous forgetting, and meanwhile ensures the adaptation performance of efficient fine tuning of parameters.