2252 results about "Text entry" patented technology

The invention provides a cross-modal image-text retrieval processing method and a cross-modal image-text retrieval processing system, which are applied to the field of information retrieval, and the method comprises the following steps: obtaining a query text input by a user in an image-text retrieval process; encoding the query text through a text encoder to generate a query text feature vector; through a cross-modal image-text retrieval model, similarity matching is carried out based on query text feature vectors and multi-modal embedding representations stored in an external knowledge base, related results corresponding to the multi-modal embedding representations larger than a matching threshold value are returned, and the multi-modal embedding representations are used for representing joint features of images and texts; when the related result comprises the image and the text at the same time, the related result and the query text are input into a preset multi-modal large model, image question answering with text assistance is carried out, and a retrieval result output by the multi-modal large model is obtained; according to the method, the semantic association between the image and the text can be better captured, so that the accuracy of image-text retrieval is improved.

The embodiment of the invention discloses a context-based big language model dialogue processing method and device and electronic equipment, and can solve the problem of how to provide a dynamic management method based on context similarity analysis and intelligent screening to optimize big language model interaction. The method comprises the steps of obtaining current text information input by a user; coding the current text information and a plurality of pieces of pre-stored historical dialogue information to obtain a current text vector and a plurality of historical dialogue vectors; according to the semantic similarity between each historical dialogue vector and the current text vector, screening the plurality of historical dialogue vectors to obtain screened target historical dialogue information; splicing the current text information and the screened target historical dialogue information to obtain a target input text; the target input text is input into a target language model, target output response information is obtained, and the target language model comprises a large language model.

The invention discloses a large-model real-time voice text intention recognition method and system based on text prediction, and the method comprises the steps: obtaining the real-time voice data of a user, carrying out the real-time voice recognition processing through a streaming voice recognition interface, and obtaining a part of transcriptional text; inputting the partial transcription text into a mask language model for text prediction, and generating a plurality of high-credibility complete sentence candidates; based on the complete sentence candidates, the complete sentence candidates are input into a large language model in parallel for intention recognition, a corresponding intention result is obtained, and a mapping relation between the candidate sentences and the intention recognition result is established; and obtaining a sentence completely expressed by the user, calculating the similarity between the complete actual sentence and a plurality of high-credibility complete sentence candidates through a multi-level text similarity algorithm, selecting the candidate sentence with the highest similarity score, and directly obtaining a corresponding final intention recognition result based on the mapping relationship. The objective of the invention is to solve the technical problem of high response delay of an existing voice intention recognition system.

Embodiments described herein provide a video generation framework built on a decoupled multimodal cross-attention module to simultaneously condition the generation on both an input image and a text input. The video generation may thus be conditioned on the visual appearance of a target object reflected in the input image. In this way, zero-shot video generation may be achieved with little fine-tuning efforts.

The invention discloses an intelligent data query method and device based on a large model and a medium, and relates to the technical field of large model application. The method comprises the steps of receiving a natural language query text input by a user, and determining a target interface group from pre-associated interface groups through a preset business line classification rule and a keyword matching algorithm; inputting the natural language input text into a pre-trained large model to analyze a parameter set required by interface calling; wherein the parameter set comprises at least two of the following items: a time range field, a business main body field and a statistical dimension field; calling an API interface in the target interface group according to the parameter set, and obtaining original business data; performing context supplementation on the original business data based on the business logic rule to generate a standardized data result containing business background description; and reconstructing the standardized data result according to a display format preset by a user, and outputting a combined visual result containing charts, tables and text description.

Provided in the embodiments of the present disclosure are a question-answering processing method, and a device, a product and a storage medium. In the question-answering processing method, after a query instruction is acquired, target knowledge information that matches the query instruction can be acquired from among a plurality of pieces of knowledge information in a knowledge base, and the query instruction and content-parsed text that corresponds to the target knowledge information are input into a large language model for question-answering processing, wherein the content-parsed text that corresponds to the target knowledge information is obtained by means of performing content parsing on a target document element that corresponds to the target knowledge information, and when the target document element comprises a document element of a non-text modality, content parsing is performed on the target document element before the target document element is input into the large language model, such that the document element of the non-text modality in the target document element can be understood by the large language model, so as to provide question-answering reference knowledge with a relatively high reliability for the large language model. Therefore, the accuracy of answering of the large language model for the query instruction can be improved.

The invention provides a bank financial management product recommendation method and system, and the method comprises the steps: firstly collecting a multi-modal dialogue data flow which is generated by interaction of a user terminal and an intelligent customer service system and contains voice input signals and text input contents, and then carrying out the semantic recognition of the voice input signals, and generating a structured demand description set; performing entity extraction and intention analysis on text input content to generate a multi-dimensional semantic tag set, performing time-space alignment and fusion on the text input content and the multi-dimensional semantic tag set to form a user portrait enhanced feature matrix, and performing multi-level association analysis on the user portrait enhanced feature matrix and a financial product knowledge graph based on a deep matching model to generate an initial recommendation result queue; and finally, according to the real-time environment perception data, queue context perception optimization is carried out, a dynamic adaptive recommendation list is generated, and visual rendering output is carried out through a multi-channel interaction interface, so that accurate and dynamic financial management product recommendation is realized by utilizing multi-modal data, deep analysis and combination with a real-time environment.

The invention discloses a visual language model illusion suppression method based on self-adaptive dynamic attention intervention. The method is used for reducing the problem that a visual language model generates wrong associated information. The method comprises the steps that text input, visual input and historical response are acquired, and an attention accumulation vector is initialized; in the layer-by-layer calculation process of the language model, dynamically adjusting a non-normalized attention matrix, enhancing the weight of a visual sensitive attention head, and performing visual Token pruning in a deep network to optimize cross-modal information interaction; and finally, generating an output Token based on the adjusted attention mechanism, and carrying out loop iteration until a complete response is generated. According to the method, a method of combining text deviation correction through self-adaptive attention head modification and visual attention convergence-based Token pruning is adopted, so that the performance of the model in a multi-modal task is remarkably improved, and the illusion phenomenon caused by a language modal dominant reasoning process is effectively relieved.

The invention relates to the technical field of man-machine interaction, in particular to a mixed interaction system based on an AI large model, which comprises a multi-modal question acquisition module, a preprocessing and classification module, a retrieval and analysis module, a mixed processing module, a result integration and output module, an interaction module and an answer display module, the problem preprocessing and classification module performs text cleaning, semantic analysis and classification, and the hybrid processing module distributes tasks to natural language processing or AI large models according to a data complexity threshold value, so that the problem processing flow is optimized, the system performance is improved, and the system performance is improved. According to the system, model parameters are optimized based on a machine learning classification algorithm and transfer learning training, classification accuracy is ensured, the system adapts to problem changes, through cooperation of all the modules, the existing technical problems are effectively solved, the man-machine interaction efficiency and quality are improved, the system has wide application prospects in multiple fields, and the man-machine interaction technology is promoted to be developed to be intelligent and efficient.

The invention relates to a method, a device, equipment and a product for executing tasks. The method comprises the following steps: acquiring an operation sequence of a user for a first task on a user interface, wherein the operation sequence comprises operation steps required for completing the first task; the method includes converting the operating steps into a model context protocol service based on the sequence of operations. The method includes obtaining a user input, the user input indicating a second task, the user input including at least one of a text input or a voice input. The method includes determining a model context protocol service to be executed based on the second task, the model context protocol service to be executed being associated with the second task. The method includes executing a second task based on task parameters of the second task and service parameters of a to-be-executed model context protocol service, the second task being composed of the to-be-executed model context protocol service associated with the second task, and the task parameters of the second task determining and displaying an execution result of the second task based on user input.

The invention relates to the technical field of semantic analysis, can be applied to business scenes of financial science and technology, medical treatment and health and the like, and discloses an intelligent correction method and device based on semantic analysis, equipment and a medium. The method comprises the following steps: extracting a structured document content from an explanation data source, extracting an auxiliary explanation text corresponding to each unit in the structured document content from the explanation data source, carrying out semantic understanding processing on the structured document content and the auxiliary explanation texts by utilizing a semantic analysis module to generate a semantic analysis result, and executing a correction operation through an intelligent correction module based on the semantic analysis result. And after the correction operation is completed, the structured document content and the auxiliary explanation text are input into a format reconstruction module, and a specification-combined document is generated and output. According to the method, semantic understanding is carried out by combining structured document content and the auxiliary explanation text, and an automatic and standardized document compliance processing flow is realized based on intelligent correction and format reconstruction driven by a compliance semantic analysis result.

The invention provides a financial fraud detection method based on a large language model. The method comprises the steps of obtaining a to-be-recognized text; performing word segmentation on the to-be-recognized text through the target word segmentation tool and the financial fraud dictionary, and determining a fraud sensitive word list; calculating the weight of each sensitive word in the sensitive word list according to a target algorithm to obtain a sensitive word weight feature vector; inputting the sensitive word weight feature vector and a to-be-recognized text into a large language model, and determining a context semantic vector of the sensitive word in combination with a word embedding technology; determining the similarity between the context semantic vector of the sensitive word and a preset financial fraud type semantic vector; and determining a financial fraud type according to the similarity. Through the implementation of the method, the generalization ability of the pre-training model is utilized to capture text deep semantics, priori knowledge is injected in combination with a sensitive word weight mechanism, the model is guided to focus high-risk vocabularies, the defect of a traditional method in semantic comprehension is overcome, the financial fraud recognition rate is increased, and the omission ratio is reduced.

The invention, which relates to the technical field of intelligent interaction, discloses an AI-based adaptive scene intelligent interaction system comprising a multi-modal data acquisition module, a modal preprocessing module, a multi-modal embedded coding module, an intention fusion and representation module, a service scene matching module and a service execution and reinforcement learning module. The method comprises the following steps: acquiring multi-modal original data in a user interaction process, including voice signals, text input and user behavior tracks, and synchronously recording an acquisition timestamp; according to the method, through a multi-modal unified embedding and dynamic weighting mechanism, the problem of characteristic dimension imbalance is effectively solved, and the user intention recognition accuracy is improved; meanwhile, reinforcement learning and a multi-factor scoring model are combined, personalized scene matching and dynamic response are achieved, the adaptive capacity and service accuracy of the system in a complex environment are improved, and therefore the stability and user experience of the intelligent interaction system are remarkably optimized.

In some embodiments, while a keyboard is visible in a three-dimensional environment, the computer system detects a gaze of a user move from a position away from a first key of the keyboard to the first key. In some embodiments, in response to detecting the gaze of the user moving from the position away from the first key to the first key, in accordance with a determination the one or more criteria are satisfied, the computer system initiates a process to select a first character corresponding to the first key for entry. In some embodiments, in response to detecting the gaze of the user moving from the position away from the first key to the first key, in accordance with a determination that the one or more criteria are not satisfied, the computer system forgoes initiating the process.

A neural network system is trained to generate textual reports (that is, medical reports, such as radiology reports) from one or more medical images, by fine-tuning a pre-trained neural network system (a visual language model, "VLM") operative, upon receiving an input comprising at least one image and a textual input, to generate a value indicative of a predicted likelihood of one or more candidate text continuations of the textual input. The fine-tuning of the neural network system is performed to reduce the value of a cost function which includes a first prediction cost term based on a first training database including first training datasets of at least one medical image and an associated text report, the first training datasets corresponding to first individuals. The first prediction cost term further includes a cost value for each individual, inversely dependent on a likelihood value of the associated textual report, conditioned on the at least one medical image, and created by the neural network system.

The invention provides a construction and training method and device for an eye fundus image multi-task model, and belongs to the field of image processing, and the method comprises the steps: S1, collecting and sorting a public eye fundus image data set, constructing an image text pair according to a real label, and carrying out the two-stage training of a multi-modal large language model, the multi-mode large language model comprises a visual encoder, a visual projector and a large language model; s2, inputting the image data # imgabs0 # into a visual encoder in the trained multi-modal large language model to obtain an enhanced visual feature # imgabs1 #, and extracting a visual feature # imgabs3 # from the # imgabs2 # through a visual projector; and S3, embedding the text input # imgabs4 # to obtain a text feature # imgabs5 #, splicing the text feature # imgabs5 # with the visual feature # imgabs6 #, and inputting the spliced text feature # imgabs5 # and the visual feature # imgabs6 # into a large language model to generate a prediction text A. According to the method, a wide range of fundus image data is collected for training, multilevel lesion features in the fundus image are fully utilized, and the performance of the model for executing a fundus disease auxiliary diagnosis task can be effectively improved.

The invention discloses a multi-agent computer-aided engineering and a computer-aided design method. CAE simulation and design optimization for the design of a complex product are high in threshold and difficulty. According to the method, autonomous planning and automatic execution of CAE simulation and CAD design optimization tasks are realized by using a multi-agent technology based on a large language model. A user interacts with the multi-agent system in the modes of text input, voice, a graphical user interface and the like, the multi-agent system autonomously plans workflow steps according to the requirements of the user, all agents cooperatively execute the working steps in a labor division mode, CAE and CAD software is called to execute simulation and design change, CAE and CAD result data are read and analyzed, the task completion condition is evaluated, and the task completion condition is evaluated. And the task can be re-planned if necessary. The system can automatically process error information, and can notify a user to process when user intervention is needed. The performance of the system can be improved through fine tuning of a large language model and retrieval enhancement generation technology.

A method for providing mental health advice, a method for training a deep-learning network and a deep-learning based mental health advisory system using Acceptance and Commitment Therapy (ACT). The method for providing mental health advice comprises the steps of: receiving textual input from a user in a consultation session; processing the textual input by applying a mental health condition relationship to the textual input to identify the mental health status of the user; and providing an output associated with the mental health status of the user.

The invention provides a multi-modal visual understanding model based on double-path visual coding, a training method, a reasoning method and equipment. The model comprises a first visual encoder used for extracting global visual features of a natural general image and outputting first image features, and weight freezing of the first visual encoder; the input of the second visual encoder is an image of which the size is adjusted to a preset high resolution, and the second visual encoder is used for extracting local detail information of the image and outputting a second image feature; the feature fusion layer is used for splicing the first image feature and the second image feature in a channel dimension to form a fused visual feature; the linear layer is used for converting the fused visual features into input dimensions required by a large language model; the large language model is used for generating natural language answers based on the fused visual features after dimension conversion and text input. According to the method, a double-path visual coding structure is adopted, the image analysis capability of the multi-modal visual understanding model is improved, and the alignment mode of the visual features and the large language model is optimized.

The present application relates to the field of natural language processing or other related fields, and discloses a method and apparatus for text sentiment recognition, a computer storage medium, and an electronic device. The method comprises: obtaining text to be recognized, wherein said text at least comprises an attribute word and an emotion word, and the attribute word refers to a subject that the emotion word describes; and inputting said text to a sentiment task model for processing to obtain a sentiment recognition result, wherein the sentiment task model is configured to determine an attribute category to which the attribute word belongs and an emotion category to which the emotion word belongs, determine all permutation and combination results of the attribute word, the emotion word, the emotion category, and the attribute category, and determine a target permutation and combination result among all the permutation and combination results as the sentiment recognition result. The present application solves problems in the related art that accuracy of recognition results of text sentiment recognition is low and content of the recognition results is incomplete.

The invention discloses a multi-mode driven virtual digital human face animation generation method and system, and relates to the field of computer graphics, and the method comprises the steps: obtaining voice input and text input, and extracting voice features and text features; dynamically fusing the two features through an attention fusion model to generate facial expression and head posture control parameters, and dynamically adjusting contribution weights of a voice mode and a text mode to the control parameters by adopting a driving strategy of differentiation of facial upper expression and facial lower expression; and performing local deformation on the virtual digital human face image based on the control parameter to generate an initial animation frame, and performing refining processing on the initial animation by using a generative adversarial network to obtain a refined face animation. By means of the technical scheme, the natural and vivid virtual human face animation matched with the voice content and the text semantics can be generated.

The invention relates to a BOM analysis method, system and device based on multi-modal semantic fusion and a medium, the method comprises the following steps: receiving BOM data, the BOM data comprising text input data and image input data; performing semantic feature extraction on the text input data by adopting a BERT model to generate a semantic embedding vector of the text; performing row and column structure identification on the image input data by adopting a Table Transform model so as to obtain a corresponding table image, and extracting table content information from the table image; based on the semantic embedding vector and the table content information, analyzing the relationship between the material and the process, and adopting a graph neural network to construct a knowledge graph of the material and the process; dynamically optimizing a mapping rule in the knowledge graph through a reinforcement learning algorithm, adjusting a corresponding field mapping rule, and generating an optimized knowledge graph; and based on the optimized knowledge graph, performing standardized conversion on the corresponding BOM data, and outputting BOM structured data. The BOM analysis method and device have the effect of improving the BOM analysis efficiency.

Methods, systems, and apparatus, including computer-readable media, for managing interactions with multiple artificial intelligence chatbots. In some implementations, a text input from a user is received. The system identifies multiple chatbots that the user is authorized to access, and the system selects a subset of the multiple chatbots based on the text input from the user. The system provides the text input from the user to each of the chatbots in the subset to generate a response to the text input from each of the chatbots in the subset. The system provides an output response to the text input from the user for presentation at the user device, where the response is based on one or more of the responses generated the chatbots in the subset.

The invention provides an AI auxiliary reading method and system based on dynamic partitioning knowledge classification self-verification, and relates to the technical field of artificial intelligence, the method comprises the following steps: carrying out dynamic semantic partitioning on a super-long text to obtain a plurality of initial text units with consistent internal semantics; performing topic model construction according to the initial text unit and a layered Dirichlet process to obtain a topic distribution tag; receiving a user query request, and extracting a retrieval text set according to the hierarchical retrieval strategy and a retrieval text corresponding to the user query request; inputting the constraint cue word extracted by the topic distribution tag corresponding to the retrieval text set, the retrieval text set and the retrieval text into a text generation model to output a to-be-calibrated text; if the error scoring function of the to-be-calibrated text and any initial text unit in the retrieval text set is smaller than a preset error threshold value, the to-be-calibrated text serves as a credible text to be output to a user. According to the invention, the credibility and accuracy of the answer can be enhanced.

Machine learning models for autonomously navigating the web and / or apps are provided that receive (i) visual input that includes one or more images representing the current, and optionally one or more prior, visual states of the website or app being navigated, and (ii) textual input that represents a user request to be implemented by the model, the current HTML or other code defining the website or app, and optionally additional text like prior model outputs that represent prior steps in the navigation of the website or app by the model. The visual input and textual input are transformed into respective sets of visual and textual tokens, and this multi-modal token set is applied to an encoder. The output of the encoder is applied to a decoder to generate textual output that represents an action to take, relative to the code defining the website or app, to pursue the user request.

The invention relates to a memory enhanced vision-language-action submerged space dynamic fusion automatic driving method. Comprising the following steps: generating a bird's-eye view feature map; extracting scene, agent and map marks from the aerial view feature map, and fusing the mark at the current moment and the previous i historical marks to generate memory enhanced visual marks; the memory enhanced visual mark and the vehicle state information are converted into a submerged space, text input of a driver is marked and unified into the submerged space, the submerged space is represented and fused, and fusion representation is generated; an automatic driving instruction data set is introduced for adjustment, and a large language model adapting to an automatic driving task is obtained; according to the fusion representation, track planning is carried out in an autoregression mode by using a large language model, and path point coordinates are obtained; and designing a transverse controller and a longitudinal controller based on PID (Proportion Integration Differentiation) to track and control the coordinates of the path points. According to the invention, the visual representation capability of end-to-end driving is improved, and the visual-language-action fusion effect is improved.

The invention discloses a visual language alignment-based visual narrative generation method and device, electronic equipment and a storage medium, and belongs to the technical field of image understanding. The method comprises the steps of obtaining a to-be-processed image sequence and an instruction text; inputting the to-be-processed image sequence and the instruction text into a multi-modal large model to obtain a visual narrative text output by the multi-modal large model; the multi-modal large model comprises a visual encoder, a multi-modal mapper and a large language model which are connected in sequence, and further comprises a text encoder, and the output end of the text encoder is connected with the input end of the large language model; the step of inputting the to-be-processed image sequence and the instruction text into a multi-modal large model to obtain a visual narrative text output by the multi-modal large model comprises the following steps: sequentially encoding images in the to-be-processed image sequence through the visual encoder to generate visual features; sequentially projecting the visual features to a word embedding space of the large language model for semantic alignment through the multi-modal mapper to obtain a language embedding vector; through the text encoder, converting the instruction text into an instruction embedding vector; and through the large language model, performing logical reasoning according to the language embedded vector and the instruction embedded vector, and generating the visual narrative text. According to the method, visual information and language output can be effectively aligned, and the visual correlation of multi-graph narration is improved.

The invention discloses a traffic question and answer method, device and equipment based on a large language model, and a medium. A traffic safety knowledge graph is constructed in advance. After the question text is divided into a plurality of sub-question texts, for the plurality of sub-question texts, determining a target entity with the highest similarity corresponding to the sub-question texts in the traffic safety knowledge graph; and determining a target sub-graph having a first-order relationship with the target entity. And for the multiple sub-question texts, inputting the sub-question texts, the entities and relationships in the corresponding target sub-graphs and the traffic long text data into a large language model, and determining sub-answer texts corresponding to the sub-question texts based on the large language model. And finally, inputting the question text, the plurality of sub-question texts and the corresponding sub-answer texts into a large language model, and determining a target answer text corresponding to the question text based on the large language model. And the accuracy of the determined target answer text is ensured.

A system can receive, via a chatbot interface, a textual input related to an electronic report. The system can generate, using a large language model, an output including a set of keywords, a rephrased version of the textual input, and an intent sentence. The system can filter, using the output from the large language model, an ontology stored in a database to generate a filtered ontology, including one or more ontology elements, intents, and examples. The system can generate, using the large language model and the filtered ontology, a plurality of actions that are compatible with the electronic report. The system can display, via the chatbot interface, the actions. The system can receive, via the chatbot interface, an indication to execute an action. The system can provide, responsive to the indication, instructions to execute the action on the electronic report to modify the electronic report.

The invention relates to a digital life individuation implementation method and device, equipment and a medium. The method comprises the steps of obtaining user interaction information in response to an obtained user input instruction; preprocessing the user interaction information according to a set mode to obtain a user interaction text; inputting the user interaction text into the digital life entity to obtain a personalized reply; the personalized reply comprises a target person and audio data; and generating an interactive video corresponding to the personalized reply based on digital human rendering. By adopting the method, personalized digital cloned construction, cross-space-time emotional interaction and cultural heritage digital inheritance can be realized through emotional analysis, behavior pattern learning and the like, and the method is suitable for scenes of emotional science and technology, digital immortality, family archive management and the like.