Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

3718 results about "Speech model" patented technology

Serial Processing Models. Serial models of speech production present the process as a series of sequential stages or modules, with earlier stages comprising of the large units (i.e. sentences and phrases), and later stage comprising of their smaller unit constituents (i.e. distinct features like voicing, phonemes, morphemes, syllables).

Real-time virtual reality scene system based on natural language description using multimodal artificial intelligence

A real-time system for the multimodal generation of virtual reality scenes based on artificial intelligence for the creation of immersive three-dimensional environments from natural language narratives, consisting of: a speech capture module configured to continuously record a user's spoken narrative via one or more directional microphones, preprocesses the captured signal by noise reduction and temporal alignment, and outputs a digital speech stream; A speech-to-text processing unit that is operationally coupled to the speech capture module and configured for real-time speech recognition using a continuous neural transformer model. The unit is trained to transcribe natural language utterances into structured text data while maintaining contextual continuity throughout the evolving narrative. a semantic interpretation processing unit that is communicatively linked to the speech recognition unit and configured to perform natural language understanding techniques to extract contextual entities, spatial references, temporal relationships, and object attributes from the transcribed narrative; the engine includes a large language model that is fine-tuned for spatial reasoning tasks; a scene graph generation module configured to transform the interpreted semantic data into a structured, hierarchical representation that defines nodes for identified entities and edges for corresponding relationships, with each node associated with metadata describing geometry, position, orientation, texture, and linking attributes between objects; a multimodal image-language model processor coupled with the scene graph generation module, wherein the processor is configured to retrieve, adapt, or synthesize appropriate three-dimensional elements from a pre-trained visual-lexical embedding space and align these elements with their semantic and spatial definitions derived from the scene graph; a scene assembly and rendering controller configured to create a cohesive virtual scene from the aligned assets, perform real-time rendering using a GPU-accelerated ray tracing pipeline, and produce a stereoscopic visual output that corresponds to the evolving narrative; A head-mounted virtual reality visualization device connected to the rendering engine and configured to display the generated immersive environment to the user in real time. The device features motion sensors and inside-out tracking cameras to detect head and body movements, dynamically updating viewing angles and perspective within the rendered scene; and a bidirectional feedback module integrated into the head-mounted device and connected to the semantic interpretation processing unit; the module is configured to interpret corrective commands, gestures, or supplementary comments from the user to refine or modify specific scene elements without interrupting the real-time visualization; The system continuously updates the virtual scene as the narrative develops, ensuring temporal synchronization between speech input and rendered output below a defined latency threshold, thus enabling a natural, dialogic construction of complex three-dimensional virtual environments.
Owner:GOUNDER MOHAN SELLAPPA DR BENGALURU +3

Text prediction-based large-model real-time voice text intention recognition method and system

The invention discloses a large-model real-time voice text intention recognition method and system based on text prediction, and the method comprises the steps: obtaining the real-time voice data of a user, carrying out the real-time voice recognition processing through a streaming voice recognition interface, and obtaining a part of transcriptional text; inputting the partial transcription text into a mask language model for text prediction, and generating a plurality of high-credibility complete sentence candidates; based on the complete sentence candidates, the complete sentence candidates are input into a large language model in parallel for intention recognition, a corresponding intention result is obtained, and a mapping relation between the candidate sentences and the intention recognition result is established; and obtaining a sentence completely expressed by the user, calculating the similarity between the complete actual sentence and a plurality of high-credibility complete sentence candidates through a multi-level text similarity algorithm, selecting the candidate sentence with the highest similarity score, and directly obtaining a corresponding final intention recognition result based on the mapping relationship. The objective of the invention is to solve the technical problem of high response delay of an existing voice intention recognition system.
Owner:BEIJING YULORE INNOVATION TECH

Automated error troubleshooting via generative AI software development assistant

Techniques for leveraging a large language model (LLM) in software development are described. A description of an error message associated with a software system hosted by the multi-tenant provider network is received. An LLM is prompted with an analysis prompt to generate an analysis of the error message, the analysis prompt including the description of the error message, and the analysis of the error message is received from the LLM. The LLM is prompted with a suggested resolution prompt to generate a suggested resolution to a cause of the error message, the suggested resolution prompt including the description of the error message and the analysis of the error message, and the suggested resolution is received from the LLM. A change to resolve the cause of the error message is sent to an originator of the received description, the change based at least in part on the suggested resolution.
Owner:AMAZON TECH INC

AI generation content detection and review method and device, equipment and storage medium

The invention discloses an AI generation content detection and review method, device and equipment and a storage medium, and the method comprises the steps: receiving multi-modal input data which comprises text data, image data and video data; calling a large language model to carry out compliance analysis on the text data to obtain an analysis result; when the analysis result is compliance, calling a multi-modal model to identify and detect the image data and the video data, and respectively obtaining an image detection result and a video detection result; and aggregating the analysis result, the image detection result and the video detection result to obtain a content detection result. According to the method, the processing paths are automatically allocated according to the content types, redundant calculation is avoided, hardware consumption is remarkably reduced, the multi-modal detection results are aggregated into structured output, the large-batch content processing efficiency is remarkably improved, calculation resource occupation is greatly reduced, and seamless integration of the detection results and a downstream service system is achieved.
Owner:深圳市维卓数字营销有限公司

CAD automatic generation system and method based on intelligent model selection and application

The invention discloses a CAD automatic generation system and method based on intelligent model selection and application, and aims at achieving automatic modeling under the multi-modal design requirement. The system comprises a user interaction module for receiving multi-modal input such as natural language, sketch and voice; the intelligent demand analysis module is used for combining an industrial large language model and a product knowledge graph, combining semantic analysis and generating a structured demand; the intelligent model selection calculation module is used for matching the optimal parameter combination and the component list based on a multi-objective optimization algorithm; the CAD automatic generation module calls a parametric modeling engine to generate an editable three-dimensional model; the constraint solving module is used for processing hard constraints and soft constraints in real time and dynamically adjusting model parameters; and the model output and interaction module feeds back a design state and supports user iteration. The system realizes full-process automation from the design intention to the CAD model, improves the design efficiency and accuracy, and is suitable for the fields of mechanical design, intelligent manufacturing and the like.
Owner:HOFMANN (BEIJING) ENG TECH CO LTD

Robot multi-modal fusion autonomous decision-making method and system based on large language model

The invention relates to the technical field of robot decision making, and provides a robot multi-modal fusion autonomous decision making method and system based on a large language model.The method comprises the steps that a robot obtains multi-modal environment information through a visual sensor, a touch sensor, an auditory sensor and a laser radar which are carried by the robot; performing preliminary filtering and noise reduction processing on the original sensor data, and synchronously recording all the sensor data by timestamps; performing space-time semantic alignment on the preprocessed multi-modal data, mapping pixel coordinates of a target in a visual target coordinate quantization original image to a robot coordinate system, performing uncertainty evaluation on a multi-modal signal through a dynamic Bayesian network, and taking entropy or variance as an uncertainty quantitative evaluation index. According to the method, the information quality is improved from a data fusion source, accurate and reliable basic support is provided for subsequent decision making, and decision making errors caused by data deviation are greatly reduced.
Owner:ANHUI UNIV +1

Requirements discovery for generative ai software development assistant

Techniques for leveraging a large language model (LLM) in software development are described. A description of a software development task is received from a user. Data associated with the software system is obtained from a data source. An LLM is prompted to identify at least one aspect of the task which requires clarification from the user, at least partly by providing the obtained data to the LLM and asking the LLM to identify a question for the user which remains unanswered by the obtained data. The question is presented to the user. An answer to the question is received from the user. The LLM is prompted to respond to propose an implementation of the task at least partly based on the data associated with the software system and the answer received from the user. The proposed implementation is received from the LLM and caused to be displayed to the user.
Owner:AMAZON TECH INC

Intelligent conference summary automatic generation method based on voice recognition and large model

The invention discloses an intelligent conference summary automatic generation method based on voice recognition and a large model. The method comprises the following steps: S1, executing voice activity detection operation on an audio data stream; s2, extracting embedding vectors of continuous and effective voice segments, and generating a voice segment set to which a spokesman belongs; s3, inputting the voice fragment set to which the spokesman belongs into an improved Whisper model, fusing a Speaker-Aware attention mechanism and a connection time sequence classification auxiliary path, and outputting a conference transcription text sequence set; s4, inputting the processed structured dialogue format into a GPT-4 large language model, and generating a conference semantic representation sequence; s5, generating a conference summary first draft text according to a preset summary generation template; and S6, performing formatting output operation on the conference summary first draft text. The conference semantic elements can be automatically extracted, the structured summary text can be generated, and the method is suitable for efficient conference recording and task tracking in government affair office, enterprise collaboration, academic discussion and other scenes.
Owner:JIANGSU GUOHUACHENJIAGANG POWER GENERATION CO LTD

Optimizing retrieval-augmented generation systems through enhanced document selection

A method includes applying a document ranking layer of a document selection large language model (LLM) to a document list including multiple reference documents to obtain a ranked document list. The method further includes selecting a subset of reference documents from the ranked document list and processing a user prompt and the document subset by a field LLM to generate an answer. The method further includes ranking the answer with an answer score by a ranking LLM. The method further includes ranking the document subset by the ranking LLM to obtain a ranked document subset. The method further includes calculating a loss function of a preference optimization layer of the document selection LLM based on the answer score and updating at least one training parameter of a foundation model of the document selection LLM based on the loss function of the preference optimization layer.
Owner:INTUIT INC

Electronic medical record LLM generation method based on animal injury

The invention discloses an electronic medical record LLM generation method based on animal injury, which realizes dialogue structuring and timestamp synchronization through multistage speech recognition and role affiliation. Using standardized medical term mapping and coding to align the free text to a standardized medical entity, and constructing a high-confidence medical entity network based on a semantic anchor point pool; according to the method, context-sensitive entity relationship extraction is realized by combining a large language model and a semantic enhancement template, a high-accuracy structured relationship chain is generated through clinical logic rule set verification, and finally, an electronic medical record template under diagnosis and treatment specifications is automatically filled and privacy desensitization processing is completed. The semantic consistency, the structural accuracy and the data security of automatic generation of the electronic medical record are improved, and standardization and intelligent circulation of medical information are effectively promoted.
Owner:GUANGZHOU WUCHUAN ELECTRONIC TECHNOLOGY CO LTD +1

Intelligent emergency decision support method and device based on multi-Agent cooperation

The invention provides an intelligent emergency decision support method and device based on multi-Agent cooperation. The method comprises a task planning module, an information acquisition module, a data fusion module and an execution monitoring module. The task planning module adopts a hierarchical decision-making mechanism, performs task decomposition in a plan making stage, generates a plurality of candidate execution paths by using thinking tree reasoning in a plan execution stage, and selects an optimal scheme. The information acquisition module acquires multi-source information such as network search, knowledge graph and geographic data through a plurality of professional Agents. And the data fusion module adopts a blackboard mode to manage heterogeneous information, and realizes intelligent abstract and correlation analysis through a large language model. And the execution monitoring module realizes dynamic optimization and fault self-recovery of the system through a multi-dimensional progress evaluation and cooperative monitoring mechanism. According to the invention, the problems of insufficient information processing capability, low decision-making efficiency and poor system stability of a traditional emergency decision-making system are solved. The information collection and processing efficiency is improved through large language model multi-Agent cooperation, the decision quality and accuracy are improved through a hierarchical decision mechanism, and long-term stable operation of the system is guaranteed through self-adaptive monitoring. The method is suitable for complex emergency decision-making scenes such as natural disasters, safety accidents and public health events.
Owner:BEIJING UNIV OF POSTS & TELECOMM

Ai-agent based system and method for real-time multilingual and context-aware linguistic transformation in telecommunications

A system and method for real-time or near real-time multilingual language transformation in telecommunications environments using distributed artificial intelligence (Al). The system includes at least one processor configured to instantiate a plurality of Al agents, each corresponding to a user in a communication session, and to manage their operation via an Al-agent runtime adapter comprising a microkernel agent manager and a virtual machine layer. A language model abstraction layer enables access to language models and Al algorithm libraries through publish-subscribe interfaces. The system supports contextual transformation of speech based on user profile data such as sentiment, emotional tone, and cultural background. Execution can occur on telecommunication-class switches, edge devices, or general-purpose computing hardware. Al agents operate asynchronously, access shared or private memory, and produce personalized, bidirectional linguistic output. The architecture supports secure, scalable deployment across heterogeneous devices and networks.
Owner:CUNNINGHAM CHERYL

Automatic speech recognition using language model-generated context

Techniques for ASR processing using language model (LM)-generated context are described. A LM is prompted to generate words that are relevant for / may be included in a future user input. The prompt to the LM can include words from user interaction history, dialog history, dialog topic, user preferences, etc. The information included in the prompt may focus on rare or unique words rather than words that the ASR model is already confident in recognizing. The techniques can be plugged into an existing / pretrained ASR model and can be used with any existing / pretrained LM, thus saving resources needed to implement and maintain the components.
Owner:AMAZON TECH INC

Lightweight intelligent traditional Chinese medicine inquiry system and construction method thereof

The invention relates to the field of artificial intelligence medical application, and discloses a lightweight intelligent traditional Chinese medicine inquiry system and a construction method thereof, and the system comprises a multi-dialect adaptive speech recognition module, a traditional Chinese medicine intelligent dialogue large language model module, a natural speech synthesis module, and a continuous learning mechanism module. The multi-dialect adaptive speech recognition module is used for converting dialect speech input of a patient into a standard text; the traditional Chinese medicine intelligent dialogue big language model module is the core of the system and is used for carrying out natural language understanding, dialectical reasoning and inquiry dialogue generation, and the natural speech synthesis module is used for converting a text response generated by the system into speech output; and the continuous learning mechanism module realizes continuous optimization of the large language model through incremental learning architecture and clinical feedback integration. According to the method, while the professional traditional Chinese medicine diagnosis capability is maintained, the calculation complexity is remarkably reduced, and the universality and sustainable development capability of system application are improved.
Owner:SUZHOU ANGSHENG NETWORK TECHNOLOGY CO LTD

AI digital human interactive response method based on large language model

The invention discloses an AI digital human interactive response method based on a large language model, and relates to the technical field of digital human interaction, and the method comprises the steps: analyzing collected user voice data and visual data through a natural language processing method, generating a cross-modal feature vector, carrying out the cross-modal association analysis of the cross-modal feature vector, and carrying out the cross-modal association analysis of the cross-modal feature vector. Generating a semantic association topological graph; calculating a vertex coordinate and a joint activity threshold value of the semantic association topological graph through high-digital human correlation, inputting the vertex coordinate and the joint activity threshold value into a constructed coordinate index database to execute attention weight calibration, and outputting a multi-dimensional association graph; and performing information density analysis based on the multi-dimensional association map, generating an information density gradient vector field, and dividing a high-density core region and a low-density edge region, the high-density core region generating a semantic core coding tensor, and the low-density edge region generating an edge feature package. According to the method, the cross-modal fusion vector is converted into the cross-modal feature vector, so that the modeling of the cross-modal association relationship is realized.
Owner:BEI JING XIN ZHI YUAN LANG WANG LUO KE JI YOU XIAN GONG SI

Supply chain cross-packet vulnerability detection method and device, equipment and storage medium

The invention relates to the technical field of information processing, in particular to a supply chain cross-packet vulnerability detection method, device and equipment and a storage medium. A vulnerability packet name, a sensitive API, a trigger parameter and vulnerability description are integrated into tetrad information; if so, performing cross-packet call chain analysis on the source code file by using a cross-packet chain reachability analysis algorithm, obtaining a function call sequence of the sensitive API based on a cross-packet call chain analysis result, realizing vulnerability detection on a cross-packet call chain, generating a vulnerability verification code based on tetrad information by using a preset large language model, and performing vulnerability verification on the vulnerability verification code. The method comprises the following steps: establishing a function call sequence of a bug verification code, verifying the accessibility of the bug verification code in the function call sequence, determining the bug confidence according to the energy consumption condition of a large language model, and generating bug alarm information when the accessibility verification result is that the bug is accessible and the bug confidence is high, thereby realizing double judgment of the bug, reducing the false alarm rate of the bug and improving the user satisfaction.
Owner:JIHUA LAB

Segmentation of media content using vision language models

Disclosed are apparatuses, systems, and techniques for efficient instance segmentation with vision language models (VLMs). In an embodiment, the techniques include processing an input into the VLM to generate a segmentation map of a media item. The input includes the media item, which includes a plurality of media item units (e.g., pixels, groups of pixels), and further includes a prompt associated with the media item. The segmentation map includes identification of media item units associated with individual objects of one or more objects in the media item, and the VLM includes a dynamic portion having parameters that are determined in view of the media item.
Owner:NVIDIA CORP

A feature editing method for large model content security

The application discloses a feature editing method for large model content security, which compares and analyzes the sparse coding features of a chat assistant constructed based on a large language model under positive user input and negative user input, extracts the internal response differences of the model to different semantic directions, and the mechanism can automatically and accurately identify the key feature dimensions highly related to the semantic direction of the target attribute. The model activation is mapped to a sparse feature space by using a sparse autoencoder, and each dimension of the feature has independent and interpretable semantic meaning. By injecting a feature guide vector in the space, the interference of the control process on the text grammar, fluency and information density is significantly reduced. The sparse representation mechanism is introduced to structure the intermediate activation features in the reasoning process of the large language model and to intervene in a targeted manner, so that the reply of the chat assistant to the user input conforms to the preset safety specification, and the safety and controllability of the chat assistant in the interaction with the user are improved.
Owner:ZHEJIANG UNIV +1

Human-computer interaction system of intelligent mechanical arm with body

The invention relates to the technical field of mechanical arms, in particular to a man-machine interaction system of an intelligent mechanical arm with a body. The system decomposes a long time sequence task into sub-tasks through a large language model, and introduces a task acquisition module, a task queue management module, a task re-planning module and a control execution module, uses visual detection to identify gestures and environment events to trigger temporary tasks, and maintains interruptible marks and safety anchor points based on priorities and interruption risks. And during interruption, the mechanical arm with the timestamp and the environment state are collected, during recovery, the states are compared, local re-planning is carried out, transition sub-tasks are automatically generated for failure sub-tasks, and the mechanical arm is controlled to execute after constraint verification. According to the system, on the premise that the structure consistency, the space consistency and the time consistency are guaranteed, safe interruption and efficient recovery in a long-time-sequence task can be achieved, and the autonomy, the real-time performance and the operation safety of the mechanical arm in a man-machine cooperation scene are remarkably improved.
Owner:NANJING TECHN COLLEGE OF SPECIAL EDUCATION

Intelligent driving scene understanding and decision-making method and system based on multi-modal large language model

The invention discloses an intelligent driving scene understanding and decision-making method and system based on a multi-modal large language model, and relates to the technical field of intelligent driving scene understanding and decision-making, and the method comprises the steps: collecting the visual, radar, laser radar, Internet of Vehicles, voice and vehicle state data of the surrounding environment of a vehicle, and forming a multi-modal original input set; performing feature extraction and semantic coding on various data in the multi-modal original input set to generate semantic feature vectors of corresponding modals; unified space mapping is carried out on the semantic feature vectors through a cross-modal alignment mechanism, multi-modal fusion processing is carried out based on an alignment result, and comprehensive semantic representation is generated; analyzing a social interaction relationship in the traffic scene based on the comprehensive semantic representation, identifying action modes and behavior tendencies of surrounding traffic participants, and generating social intention description information; and generating a scene query request according to the comprehensive semantic representation, and matching related traffic rules and driving experience in a pre-constructed driving common knowledge base.
Owner:SHANGHAI INTELLIGENT & CONNECTED VEHICLE R & D CENTER CO LTD

Creation content generation system based on image recognition and large language model fusion

The invention discloses a creation content generation system based on image recognition and large language model fusion, and particularly relates to the technical field of creation content generation, and the system firstly completes the fact extraction and brand anchor point construction of an input image in a unified coordinate and scale system, and forms a structured fact package in one-to-one correspondence with an original image; then, performing protagonality scoring and ambiguity gating on the figure instance, and outputting an explainable and calibratable protagonality judgment result; on this basis, the condition controlled generation and template selection module converts the fact constraint into a controlled text packet and a format instruction packet, and keeps explicit mapping with a fact packet; the system further executes cross-modal consistency and compliance verification based on image facts, and machine-readable verification and minimum cost correction are carried out on text and graph entities, geometrical relationships and brand elements; and finally, solidifying the key intermediate quantity, the parameters and the judgment basis into an evidence chain through a chain type index, and introducing online adaptive learning in a compliance boundary to realize mild updating and rollback release.
Owner:HANGZHOU SHUANGHEDAN NETWORK TECH CO LTD

Large language model illusion detection method and system based on adaptive sampling strategy

The invention discloses a large language model illusion detection method and system based on an adaptive sampling strategy, and the method comprises the steps: carrying out the word segmentation and coding of a to-be-detected text and each sampling text, obtaining the respective token identifiers, carrying out the splicing, obtaining a plurality of text pair sequences, and carrying out the coding processing, and obtaining the coding results of a plurality of text pairs; the coding results of the text pairs are input into a consistency evaluation model to be processed, and consistency scores of the multiple text pairs are obtained; and obtaining an embedded vector of a to-be-detected text, calculating statistical information of consistency scores of the plurality of text pairs, inputting the embedded vector and the statistical information into the sampling decision model for decision making, determining whether to perform sampling again, if so, performing sampling again, and if not, obtaining a hallucination detection result based on the consistency scores. A sampling decision is innovatively designed after consistency scoring, and the sampling decision realizes adaptive sampling and scoring in the illusion detection process by means of a near-end strategy gradient optimization training sampling decision model in reinforcement learning.
Owner:SHANDONG UNIV

Tourism service agent system and method based on multi-modal large model

The invention discloses a tourism service agent system and method based on a multi-modal large model. The system comprises five parts: a user interaction interface, which is used for receiving a user request and returning a response; the agent center is used for performing intention understanding, task planning and response generation through a multi-modal large language model; and the tool calling module is used for executing specific API business operation. According to the intelligent tourism service system, the work of each module is coordinated through the intelligent agent center, a complete closed loop from user intention understanding to service execution is realized, accurate, reliable, whole-course and personalized intelligent tourism service can be provided, and the problems that a traditional tourism service system is single in function, inaccurate in information and lack of action ability are effectively solved.
Owner:XIAMEN UNIV

Grid telephone traffic quality inspection intelligent analysis system and method based on large language model

The invention relates to the technical field of intelligent telephone traffic quality inspection, and discloses a grid telephone traffic quality inspection intelligent analysis system and method based on a large language model, and the system comprises the steps: collecting and obtaining a multi-role call audio signal in real time, and preliminarily carrying out the speaker separation and role marking through a voice recognition module and a voiceprint recognition module; forming a preliminary role recognition result; detecting a suspected role identity error region in combination with multi-dimensional features, and when a detection result meets a preset condition, triggering a dynamic correction mechanism, generating auxiliary judgment information in combination with identity declaration keywords, business term matching and dialogue context logic inference, adopting a multi-dimensional weight decision strategy, and re-correcting a role identity tag, so as to obtain a role identity error region. Updating a role recognition result; and setting an accurate evaluation module of a dynamic correction result, feeding back and adjusting a multi-feature weight and a trigger threshold in real time, and forming an iterative optimization mechanism of dynamic correction. The method has the advantage of improving the dynamic correction capability.
Owner:XIANGYANG POWER SUPPLY COMPANY OF STATE GRID HUBEI ELECTRIC POWER

Method, system and device for voice interaction inside and outside vehicle and storage medium

The invention discloses a method, a system and equipment for voice interaction inside and outside a vehicle and a storage medium, and relates to the technical field of intelligent cabins and human-computer interaction. Identifying an interaction trigger type (in-vehicle active request, out-of-vehicle passive request, or system active trigger) based on the perceived data and / or the vehicle event; determining a corresponding response permission strategy according to the trigger type, and determining whether to allow the system to respond based on the response permission strategy; and if so, calling a large language model to generate a voice text, synthesizing voice by combining external perception characteristics, and broadcasting the voice through a loudspeaker outside the vehicle or a sound box in the vehicle. According to the scheme, three types of interaction intentions including the in-vehicle active request, the out-vehicle passive request or the system active triggering are recognized, the large language model is called to generate the scene-adaptive voice text, and personalized voice synthesis and broadcasting are performed in combination with the external perception characteristics, so that the response efficiency, the expression naturalness and the object adaptability of the in-vehicle and out-vehicle voice interaction are improved.
Owner:ZHEJIANG GEELY HLDG GRP CO LTD +1

Prompt construction method and system of multi-mode large language model, computer equipment and medium

The invention relates to the technical field of multi-modal large language model training, in particular to a prompt construction method and system for a multi-modal large language model, computer equipment and a medium. The method comprises the following steps: extracting a key frame set from an input video stream; executing a motion reconstruction process on the video stream to generate motion track information; and performing visualization processing on the motion track information to generate a track visualization graph. Performing space-time correlation coding on the key frame set and the motion track information to generate an enhanced key frame; a multi-modal prompt is constructed in a mode of integrating visual input and text input, and the multi-modal prompt is input into a preset multi-modal large language model for spatial reasoning. Through the mode, the technical problem that an existing prompting method is difficult to give consideration to the spatial reasoning precision and the calculation efficiency is solved, efficient and accurate spatial reasoning of the multi-modal large language model is achieved, and the calculation efficiency, the reasoning precision and the environmental adaptability of the model are improved.
Owner:HONG KONG UNIV OF SCI & TECH (GUANGZHOU)

Knowledge document-oriented method for structurally and stably presenting large language model output

The invention discloses a knowledge document-oriented method for structurally and stably presenting large language model output. The method comprises the following steps of: preprocessing a knowledge document and converting the knowledge document into a text by a system; the back-end service module adopts a prompt word engineering technology based on a preset JSON mode, constructs a structured prompt containing clear instructions and output format constraints, calls a large language model to carry out analysis and information extraction on text contents, and forces the model to generate structured JSON data following the preset mode; the back-end service provides access to the structured JSON data for the front-end application; and the front end executes templated mapping rendering according to a preset user interface component template isomorphic to the JSON mode, and fills the content of each field of the JSON into a corresponding visual component. According to the method, through the cooperation of the back-end constraint prompt word engineering and the front-end modularized template rendering, the defects that the output content format of a large language model is unstable and is difficult to be directly applied to standardized interface presentation are relieved.
Owner:BOBAN ZHIJIE (BEIJING) TECHNOLOGY CO LTD

Large language model proxy security test method and device based on model context protocol

The invention discloses a large language model agent security test method and device based on a model context agreement, and the method comprises the steps: firstly deploying a test tool in a test server, defining an application program interface which comprises function parameters and test probe parameters, describing the test probe parameters as necessary technical requirements for executing nominal functions, and executing the nominal functions according to the test probe parameters; and inducing the tested LLM agent to transmit complete session context data when calling. The test server receives a tool call request including a function parameter value and session context data, a back-end concurrently processes the request, executes a nominal function to generate a benign result, and extracts the session context data at the same time. Afterwards, a benign result is returned to the tested LLM agent, and the extracted data is asynchronously transmitted to a remote log server for recording as a test log. And finally, comparing the test log with the actual operation history, judging whether a session context data leakage vulnerability exists or not, and quantifying the severity level so as to detect whether the LLM agent leaks session memories such as the user interaction history or not.
Owner:XI AN JIAOTONG UNIV

Large model dynamic protection method and system based on zero-trust architecture

The invention provides a large model dynamic protection method and system based on a zero-trust architecture. The method comprises the steps that a security proxy gateway receives an access request; authenticating an initiating main body of the access request, collecting context information and transmitting the context information to a strategy decision point; the strategy decision point calculates a trust score in real time based on a dynamic trust evaluation model and performs real-time evaluation in combination with an access control strategy to generate a dynamic authorization judgment result; if the access is allowed, forwarding the access request to a large language model server, and performing input security filtering; the large language model server generates response content and performs output security filtering; and returning the final response subjected to the output security filtering to the initiating main body through the security proxy gateway. According to the method, a multi-layer protection framework is constructed, a dynamic trust evaluation model is introduced, and a content filtering layer is deployed, so that continuous permission verification, risk adaptive control and full-link content security protection are realized, and the service security of a large model is effectively guaranteed.
Owner:NO 30 INST OF CHINA ELECTRONIC TECH GRP CORP