165 results about "Input language" patented technology

The invention discloses a multi-modal model visual perception ability enhancement method and device and a medium, and the method comprises the steps: collecting image data, and proposing a text problem according to an image; selecting a plurality of multi-modal language large models trained based on the same language model, and merging parameters of the language models; based on the input image, utilizing a plurality of visual encoders to extract visual features; inputting a text, and performing feature extraction on the input text through the language model to obtain text features; splicing the visual features and the text features to obtain fusion features; and inputting the fusion features into a language model for reasoning, and outputting a reasoning result. According to the method, extra training is not needed, the performance of the multi-modal task can be effectively improved, and the deployment overhead is reduced. Besides, by means of the technology, the visual perception effect of the multi-modal large model can be remarkably improved by fusing the advantages of different visual encoders, and meanwhile the requirement for computing resources is lowered. The method can be widely applied to the field of artificial intelligence.

The invention discloses an extensible structured text generation method, device and equipment in the field of PLC and a medium. The method comprises the following steps: constructing a case library and an instruction library; based on semantic similarity of task description, similar task cases are retrieved from a case library through a language model and sorted according to similarity scores; for each task case, common programming instructions used by the task case are retrieved from an instruction library, and an instruction information set is generated according to corresponding instruction information to serve as knowledge of a specific field; constructing a context prompt, inputting a language model, and generating a structured text draft code; taking the structured text draft code as the input of an ANTLR analyzer for grammatical analysis, obtaining grammatical error information and constructing an abstract syntax tree; the abstract syntax tree is traversed through ANTLR Walker for semantic analysis, and semantic error information is obtained; the error information is provided for the language model, a corresponding code repairing scheme is generated, and codes are improved. According to the method, the ST code specific to the supplier can be automatically generated, and the quality of the generated code is ensured.

The methods, systems, and computer networking apparatuses described herein enable language models to receive input (e.g., a query or a request) from a user or application, and without any additional training data or instructions, determine to generate a function call based on the received input from the user and generate the function call based on the determination. In some embodiments, a language model may further access an external tool or application to request an output as a response to the generated function call. The disclosed methods, systems, and networking apparatuses improve the technical field by incorporating language model capabilities within the function calling process, and allowing for function-related information to be provided to a language model via input received in any number of formats or types, including structured or unstructured input, language or non-language input, or any combination thereof.

The embodiment of the invention provides a multi-modal fusion entity retrieval enhancement generation method and device, and the method comprises the steps: carrying out the blocking and adaptive text extraction of multi-modal data in an offline stage, obtaining the text block data corresponding to each modal data, extracting the entity and relation of the text block data according to a language model, constructing an entity triple, and carrying out the segmentation and adaptive text extraction of the entity triple. Fusing the entity triad with the text block data to obtain an offline knowledge graph, and constructing a data index; in the present stage, a query statement of a user is received, text block data most similar to the query statement are retrieved in a knowledge graph through a data index, after entity aggregation is carried out on the retrieved text block data, the text block data are reordered according to retrieval scores, an entity aggregation result is obtained, entity ordering is carried out according to the entity aggregation result, and the entity aggregation result is obtained. By means of the multi-modal retrieval method and device, the efficiency and accuracy of multi-modal retrieval can be improved.

The present application discloses a test code repairing method applied to a code development platform. The method comprises: obtaining a test code corresponding to a source code, and then executing the test code to obtain execution information of the test code, wherein the execution information comprises abnormal stack tracking information output during execution of the test code; on the basis of the execution information, performing fault localization on the test code to obtain location information of a faulty code; on the basis of the location information of the faulty code, extracting context information, wherein the context information comprises at least one of a faulty test case, a fault description, or the faulty content; and inputting a prompt constructed on the basis of the context information into a language model and performing inference, so as to obtain a first repaired code. The method is used to perform fault localization in light of the execution information, extract the context information on the basis of the location information of the faulty code, and, on the basis of the context information, generate accurate and complex repaired code via an advanced language model, thereby adapting to various complex fault scenarios and improving repair efficiency and accuracy.

Use of a language model to automatically perform visual assessment of quality of an item being fulfilled by a picker. The online system receives an image of the item and identifies a set of potential problems associated with the item. The online system generates a plurality of prompts for input into the language model including the image and one or more questions each corresponding to a respective potential problem of the set potential problems. The online system requests the language model to generate, based on the plurality of prompts, a feedback response for each potential problem. The online system generates an aggregated output by aggregating the feedback response for each potential problem, and based on the aggregated output, a second message that identifies one or more relevant problems associated with the item. The online system causes a device of the picker to display the second message.

The invention discloses an international remote translation system based on AI and an implementation method thereof, and belongs to the technical field of artificial intelligence. Comprising a voice input module, an emotion perception module, a predictive simultaneous interpretation module, a language adaptation module based on few-sample learning, a dynamic translation quality evaluation and feedback module and a multi-language cooperative processing module, and can automatically recognize an input language and dynamically adjust a translation model in a multi-language environment. The emotion information in the voice and the text is captured in real time, the translation path is optimized in combination with the context weight, the emotion transmission accuracy and the translation context consistency are remarkably improved, the term requirements of different fields can be rapidly met, the translation quality is continuously optimized through dynamic feedback, the translation accuracy and smoothness are ensured, and the translation efficiency is improved. And the method has wide applicability in multi-language, cross-cultural communication and professional field application, and has relatively high innovativeness and practical value.

The invention provides an interaction method and device based on retrieval enhancement, equipment and a storage medium, and after question information input by a user is detected, the question information and a retrieval prompt word are input into a strategy selection model. Obtaining a target retrieval strategy output by the strategy selection model, performing information retrieval according to the target retrieval strategy, inputting a retrieval result and question information into a language model, outputting reply information through the language model, and utilizing a retrieval prompt word for indicating a matching relationship between the retrieval strategy and a question type to retrieve the question. According to the method, the strategy selection model is guided to select the appropriate retrieval strategy for the question input by the user to obtain the retrieval result, the accuracy and the retrieval efficiency of the retrieval result can be improved, the reply information is generated based on the retrieval result and the question information, and therefore natural language-based interaction is assisted by utilizing the language model, and the man-machine interaction efficiency is improved.

The invention discloses a knowledge distillation-based Hainan dialect speech recognition optimization system, which comprises a data preprocessing module for processing Hainan dialect speech data and Hainan dialect text data, and outputting to obtain an MFCC speech feature sequence, a labeled text label and Hainan dialect text data; the teacher model module comprises an RNN (Recurrent Neural Network) language model, a CNN (Convolutional Neural Network) language model and a Transform language model, dialect text data are respectively input into the RNN language model, the CNN language model and the Transform language model, and a soft label and a middle layer feature are obtained through dynamic temperature regulation; the student model training module is used for performing knowledge distillation and parameter adjustment and optimization on a student model according to the MFCC voice feature sequence, the marked text tag, the soft tag and the middle layer feature; the output module is used for performing voice recognition on the Hainan dialect by using the student model after knowledge distillation and parameter optimization to obtain a recognition result; moreover, the trained student model is small in size and low in calculation complexity, and an accurate recognition result can be obtained finally.

A data processing system and method include receiving a set of documents to summarize a trend across the set of documents, inputting the set of documents into an information extraction model, executing the information extraction model to extract a first plurality of text segments, determining a second plurality of text segments based on the first plurality of text segments, determining a third plurality of text segments from the second plurality of text segments, generating a compressed representation of the set of documents from the third plurality of text segments to include in a prompt for a language model, inputting the prompt into the language model, and executing the language model to generate the summary based on the prompt for the set of documents.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of mechanical arm grabbing, financial science and technology, medical treatment and health and the like, and discloses an action sequence generation method and device, equipment and a medium. Respectively generating a visual feature vector and a language feature vector by using a visual encoder and a language encoder; fusing the visual feature vector and the language feature vector to obtain a multi-modal fusion feature, and inputting the feature into a language model for processing to generate an initial action strategy; an action sequence is generated according to an initial action policy by an action decoder integrated with a language model. According to the method, by fusing vision and language information, self-adaptive decision making in a complex task environment is realized, and the adaptive capacity of the system in a dynamic change environment is enhanced; and through multi-modal feature fusion, the operation precision and the generalization ability of the system are effectively improved.

The invention provides an online text customer service reply generation method and device, and the method comprises the steps: carrying out the weighting of a text feature vector of a to-be-processed text based on the intention of the to-be-processed text inputted by a user, and obtaining a user text feature; based on the context of the to-be-processed text, extracting context features; based on a retrieval result of the to-be-processed text in a pre-constructed knowledge base, static knowledge features are extracted and obtained; based on the behavior data of the user for the product and the service in the previous preset time period, extracting to obtain dynamic knowledge features; and inputting the user text feature, the context feature, the static knowledge feature and the dynamic knowledge feature into a language model to obtain an answer text output by the language model. According to the method, the answer of the online intelligent customer service is generated by combining the historical record, the preference information and the context information of the user and the semantic features of the to-be-processed text.

The invention discloses a knowledge base self-learning evolution method and system and a storage medium. The method comprises the steps that a to-be-processed text containing entity mention is received, candidate standard entities are recalled in a knowledge graph, and enhanced knowledge sub-graphs are adaptively extracted based on ambiguity scores of the candidate standard entities; encoding a to-be-processed text, the enhanced knowledge sub-graph and the like into cue words, and inputting the cue words into the language model to obtain a standardized result; when the confidence coefficient of the standardization result is lower than a threshold value or any candidate standard entity is not recalled, a generative probe instruction containing a predefined ontology mode is submitted to the language model, and the language model is guided to generate structured candidate knowledge; calculating knowledge entropy based on the candidate knowledge; pushing the candidate knowledge to a man-machine collaborative auditing queue according to the knowledge entropy; and executing transaction-based atomization write-in operation on the candidate knowledge which is checked and confirmed, so as to realize a self-learning closed loop of the knowledge base. According to the method, a credible and automatic self-learning closed-loop method is constructed, so that the knowledge base can be dynamically evolved.

The invention discloses an e-commerce platform marketing content matching method based on semantic reasoning, and the method comprises the following steps: S1, generating a multi-hop interaction chain according to a user historical behavior sequence; s2, gradually performing mask prompt and inputting a language model, performing interactive chain thinking semantic reasoning, and generating a recommended candidate content set; s3, calculating the matching degree of the candidate content and the context semantics, and determining an initial sequence; s4, iteratively updating the sorting weight by adopting a secretary bird optimization algorithm; s5, adjusting the sorting of the candidate contents according to the sorting weight; and S6, updating a language model reasoning strategy and optimization algorithm parameters according to user feedback. According to the invention, the accuracy of the recommended content and the user experience are improved.

The invention discloses a singular value decomposition-based multi-modal task fine tuning method for enhancing a routing function, which comprises the following steps of: mapping input language and visual features from a high-dimensional space to a low-rank space by using a PEFT method, performing singular value decomposition on language features in the low-rank space, performing routing function alignment through a tensor after efficient reconstruction, and performing multi-modal task fine tuning on a multi-modal task based on a singular value decomposition-enhanced routing function. And finally, after the low-rank space is recovered to the original dimension again, performing residual connection with the original language features, and outputting the features. According to the method, singular value decomposition is applied to language features before a routing function, a low-rank dominant mode of the language features is extracted, the alignment precision of vision and language features is enhanced, interference of high-dimensional noise is eliminated, and meanwhile calculation efficiency and model stability are kept. Routing calculation is carried out through the reconstructed tensor, key information in the features can be better extracted and aligned, and therefore the precision and effect of feature alignment are improved. The method is suitable for VL tasks such as visual questioning and answering and image description generation, and model performance can be obviously improved.

The embodiment of the invention discloses a cue word generation method of a language model, a question and answer method and electronic equipment, and the cue word generation method comprises the steps: receiving first information related to a target business of the language model, and determining business identification information corresponding to the first information, at least one cue word sample is obtained from a plurality of cue word databases corresponding to the target service according to the service identification information, the types of the cue word samples stored in the cue word databases are different, and a target cue word used for inputting the language model is determined according to the obtained cue word samples. Therefore, the obtained target cue word more accurately reflects the key information of the first information related to the target service, a relatively good cue word corresponding to the first information input by the user can be obtained, the accuracy of language model output is improved, and the user experience is improved.

A method for (3D) object detection includes: receiving an input image with respect to a 3D space, an input point cloud with respect to the 3D space, and an input language with respect to a target object in the 3D space; using an encoding model to generate candidate image features of partial areas of the input image, a point cloud feature of the input point cloud, and a linguistic feature of the input language; selecting a target image feature corresponding to the linguistic feature from among the candidate image features based on similarity scores of similarities between the candidate image features and the linguistic feature; generating a decoding output by executing a multi-modal decoding model based on the target image feature and the point cloud feature; and detecting a 3D bounding box corresponding to the target object by executing an object detection model based on the decoding output.

The invention discloses an integrated circuit design Verilog code generation method and device based on a large language model, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps that semantic analysis is conducted on an input language of an integrated circuit user side through the large language model, and a target constraint set is determined based on an obtained performance index set and through a regular expression; a candidate architecture scheme is generated by utilizing thinking chain technology reasoning, the candidate architecture scheme is predicted by utilizing an XGBoost regression model, and a target architecture scheme is determined based on an index prediction result and the candidate architecture scheme by utilizing a non-dominated sorting genetic algorithm; and determining an initial Verilog code based on the target architecture scheme and a preset code template library, and performing code style conversion on the initial Verilog code based on a code style feature corresponding to the integrated circuit user side to obtain a target Verilog code. And the efficiency and availability of integrated circuit design are improved.

The invention discloses an active optical alignment method based on a local lightweight language model, and particularly relates to the technical field of precision installation and adjustment, comprising the following steps: extracting image indexes and platform attitude information through image acquisition, constructing a unified input format, verifying and inputting the language model; generating a six-degree-of-freedom pose adjustment strategy and a credibility score; the strategy is executed by the platform control device after being verified by a preset criterion; scoring is performed again after execution, and whether a feature posture is recorded or error compensation is triggered is judged according to a scoring result, so that subsequent input and strategy generation are optimized, and alignment precision and stability are improved; according to the method, the data consistency is guaranteed through a unified input format and a verification mechanism, the alignment precision is improved in combination with a dynamic scoring strategy, and the stability and fault tolerance of the system are enhanced by introducing a feature attitude recording and error back-filling mechanism, so that a strategy generation and execution process with high reliability, high adaptation and high robustness in an active optical alignment task is realized.

The invention relates to the technical field of 3D digital humans, and discloses a real-time interaction method and device of a 3D digital human. First dialogue data initiated by a target user is converted into a first dialogue text through a voice recognition unit in sequence, and then a response text is recognized by inputting a language recognition model; and the response voice is synthesized into response voice, target lip language data is generated through the lip language generator, and finally the 3D digital human is driven to make voice response, so that the target user can obtain accurate, natural and real feedback. Moreover, interaction is carried out in a voice conversation mode, the method is simple and convenient, a highly vivid conversation scene can be displayed, and the interaction experience of the target user is enhanced.

Disclosed herein are method and system aspects for translating between a sign language and a target language and presenting such translations. For example, a method receives input language data and translates the input language data into sign language grammar. The method retrieves phonetic representations that correspond to the sign language grammar from a sign language database and generates coordinates from the phonetic representations using a generative network. The phonetic representations are digital representations of individual signs created through manual input of body configuration information corresponding to the individual signs. Further, the method renders an avatar that moves between the coordinates. In another example, a bidirectional communication system allows for realtime communication between a signing entity and a non-signing entity.

The invention discloses a multi-language neural machine translation model training method based on meta learning and language pair subnets, and belongs to the technical field of machine translation. The problem that an existing multi-language translation sharing model is poor in translation performance is solved. According to the method, initial parameter optimization is carried out on a multi-language translation model by means of an ML method to obtain a local optimal parameter value, and the ML learns sharing representation among tasks and carries out adaptive adjustment on each language pair, so that parameter interference can be reduced, and translation performance is ensured. A mask matrix of each language to a task can be obtained by learning a language specific subnet, and only model parameters corresponding to input language to data are updated each time during model training, so that a multi-language translation model can avoid parameter interference, and meanwhile, the training efficiency is improved. In the subsequent translation task, only the model parameter corresponding to the element 1 in the mask matrix is needed to participate in translation result prediction, so that the complexity of model calculation is reduced. The method can be applied to the field of machine translation.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a response optimization method, device, equipment and medium based on term consistency. Inputting a language model to generate a candidate response; performing term consistency comparison and context consistency comparison on the candidate responses to generate a sorting result; term anomalies and safety risks in the candidate responses are detected, and the sequence is adjusted according to a detection result; and outputting the target response and obtaining feedback information, and updating the term knowledge base according to the feedback information. Through linkage of the term knowledge base and the security policy, response term consistency, semantic coherence and security are ensured, the knowledge base is dynamically updated based on feedback information, and reliability and specialty of a language model in multiple rounds of dialogues are achieved.

The invention discloses a reference multi-target tracking method based on language decoupling and fine-grained multi-modal feature alignment, and relates to a computer vision technology. The method comprises the following steps: A, giving a training data set containing a video sequence and language description; b, inputting the video sequence in the step A into a backbone network to extract visual features, and inputting language description into a language model to extract text features; and C, performing multi-modal alignment and fusion through a cross attention mechanism according to the visual features and the language features extracted in the step B. And D, decoupling the language features extracted in the step B into local description and a motion state. And E, inputting the refined features extracted in the step C and the local description extracted in the step D into a static semantic enhancement module to extract target information. And F, associating the current frame target obtained in the step E with the existing trajectory by using a Hungary matching algorithm. G, inputting the matched target features in the step F and the motion state in the step D into a motion perception alignment module to enhance the target recognition capability; the tracking performance of the method is improved.

The invention provides a work order clustering and theme extraction method, system and device and a medium, and belongs to the technical field of natural language processing and data mining. The method comprises the steps of obtaining original work order data, performing data cleaning, extracting work order abstracts from the original work order data by using a text abstract generation technology according to preset abstract constraints, and generating an abstract set; inputting the work order abstract into a language model, generating a semantic vector of a preset dimension, constructing a vector index through an FAISS library, and converting the abstract set into a set of numerical vectors; clustering the numerical vectors by using an improved K-means algorithm, and outputting K groups of work order digests with balanced quantity; for each group of work order abstracts, screening representative abstracts based on text similarity; extracting keywords from the representative abstracts by adopting a keyword extraction technology, and generating a keyword list of each group of work order abstracts; and for the keyword list of each group of work order abstracts, generating a theme tag by combining keywords.

The invention discloses a test code repairing method which is applied to a code development platform and comprises the steps that a test code corresponding to a source code is obtained, then the test code is executed, execution information of the test code is obtained, and the execution information comprises abnormal stack tracking information output when the test code is executed; then error positioning is conducted on the test code according to the execution information, position information of an error code is obtained, context information is extracted according to the position information of the error code, and the context information comprises at least one of an error test case, error description or error content; and inputting a prompt constructed based on the context information into a language model for reasoning to obtain a first repair code. According to the method, error positioning is carried out by combining execution information, context information is extracted according to position information of error codes, and based on the context information, more accurate and complex repair codes are generated through an advanced language model so as to adapt to various complex error scenes, and the repair efficiency and accuracy are improved.

A system (100) for context-sensitive language translation, comprising: an input interface module (102) configured to allow the user to input text for language translation, wherein the input interface module (102) includes an interface that allows the user to input text and select the language for translation; a context processing module (104) connected to the input interface module (102) and configured to enable context-dependent extraction of information from the provided input text, wherein the context processing module (104) comprises the following: an input processing module (106) configured to decompose the input text into tokens and identify context-sensitive segments; and a context extraction module (108) connected to the input processing module (106), wherein the context extraction unit (108) comprises an extraction processor configured to generate context vectors from source input text; a translation processing unit (110) comprising a machine learning-based processor configured to translate the provided input text, wherein the translation processing unit comprises: an attention mechanism (112) configured to select words based on their relevance to the context; and a translation generation module (114) configured to generate translations using a neural network that incorporates the attention-weighted context vectors; a post-processing unit (116) connected to the translation processing unit (110) and configured to refine the translated output to ensure fluency and coherence, refining the translation grammar, ensuring semantic consistency, validating contextual appropriateness, and generating a fluency score; and an output module (118) connected to the post-processing unit (116) and comprising a display with an output interface on which the input text and the translated output text are displayed to the user.

The invention discloses a feature fusion optimization-based multi-modal map enhancement retrieval method and a dialogue system. The method comprises the following steps of: respectively carrying out pre-training and fine tuning on a visual model and a language model by utilizing a domain image and text data; constructing a knowledge graph based on the text data in the knowledge base and constructing a vector database containing associated image data; performing semantic analysis and optimization on the original query of the user by using the language model and forming a structured retrieval intention; searching related sub-graphs, text semantic vector information and associated image data based on the search intention; encoding the sub-images into knowledge contexts, inputting the knowledge contexts into a dynamic prompt generator to generate visual prompts, and extracting enhanced visual features from the associated image data through a visual model; and inputting the subgraph, the text semantic vector information and the enhanced visual features into a language model for collaborative reasoning, and generating and outputting a final answer. According to the method, deep fusion and accurate retrieval of multi-modal knowledge can be realized, and the accuracy and efficiency are remarkably improved.

The invention relates to the technical field of automatic driving, in particular to a multi-mode automatic driving training method based on a DeepSeek training framework. Comprising the following steps: reading multi-view camera images and text instructions of a DriveLM-nuScenes data set, and splicing the images according to a look-around layout to form panoramic representation; performing zooming, normalization and standardization processing on the panoramic image to obtain an image tensor; performing marking processing on the text instruction, inserting an image placeholder and a dialogue role mark, and structuring text input representation; dimensionality alignment, position code addition and cross-modal attention fusion of vision and text marking sequences are realized through a multi-modal alignment module, and multi-modal embedding representation is generated; and inputting the embedded representation into a DeepSeek language model to generate a decision text through autoregression, and taking the cross entropy loss with a mask as an optimization target. According to the method, the problems of insufficient multi-view fusion, weak modal alignment and the like in the prior art are solved, the cognitive reliability and the decision interpretability in a complex scene are improved, and vehicle-mounted edge deployment is adapted.

The invention provides an LLM-based vulnerability risk repairing method, device and equipment and a storage medium, the method comprises the steps that scanning data scanned by a vulnerability scanner is processed by a first cue word generator and then input into an LLM language large model, a repairing suggestion is generated, and the scanning data comprises a request packet sent by the vulnerability scanner and a response packet received by the vulnerability scanner; finding a code snippet with the vulnerability according to a code path where the vulnerability is located, processing the code snippet through a second cue word generator, inputting the processed code snippet into the LLM language large model, and generating a repair code; and formatting the repair suggestion and the repair code to generate a vulnerability repair scheme. According to the method, multi-dimensional features of vulnerability codes are deeply analyzed by using an LLM language large model, vulnerability risk points in the codes are accurately positioned, more targeted repair suggestions and repair codes are provided, it is ensured that a generated repair scheme is matched with an actual code scene, the vulnerability risk repair cost is reduced, and the repair efficiency is improved. And a more efficient and reliable vulnerability risk management solution is provided.