35 results about "Interaction framework" patented technology

The invention relates to a weak supervision video anomaly detection method for text interaction context features. The method comprises a global local context framework, a cross-modal interaction framework and a feature fusion and multi-instance learning framework. Obtaining an abnormal detection video data set and a corresponding text dictionary, and dividing the abnormal detection video data set and the corresponding text dictionary into positive / abnormal video data sets; normal / abnormal context video features are generated in a global local context framework through I3D features obtained by the normal / different video data sets through an I3D feature extraction network; meanwhile, a video data set and a corresponding text dictionary are trained from a CLIP model, video features and text features are generated, and text guide video features are generated in a cross-modal interaction framework; and the normal / abnormal context video features and the text guidance video features are correspondingly fused in a feature fusion and multi-instance learning framework to obtain a weak supervision video anomaly detection result. The performance of weak supervision video anomaly detection can be remarkably improved, and the application scene of the method is widened.

The invention belongs to the technical field of motion control of robots, and relates to an LLM human-computer interaction framework-based AUV motion control method and system, and the method comprises the steps: obtaining a target AUV, and constructing a target AUV simulation platform based on ROS2; the LLM and the NMPC are integrated in the ROS2 to form an LLM man-machine interaction framework; acquiring a user instruction, converting the user instruction into a control instruction by the LLM man-machine interaction framework, and transmitting the control instruction to the target AUV simulation platform; the AUV in the AUV simulation platform executes a control instruction, generates trajectory tracking and displays the control instruction, a large language model is applied to the field of underwater robots, a man-machine interaction framework is combined with a traditional NMPC control method, control over a bottom-layer NMPC is achieved by means of high-level decision from LLM, and through an interactive display interface of Rviz2, the AUV simulation platform can be applied to the field of underwater robots. And the control flexibility of the AUV in a complex experiment scene is effectively improved.

The invention provides a flexible distributed energy network multi-agent collaborative optimization method based on MARL, and the method comprises the steps: building a distributed energy network multi-agent interaction framework which comprises a local energy transaction model for distinguishing an intelligent soft switching path, a network cost model based on a power transmission distribution factor, and a power distribution network operator and user income model; a physical system operation constraint model is established and covers intelligent soft switch operation characteristics, network topology radial constraint, energy storage charging and discharging dynamic states, controllable distributed power supply and photovoltaic output limitation, translational load scheduling and DistFlow power flow constraint. And finally, constructing a collaborative optimization mechanism based on a multi-agent near-end strategy optimization algorithm, modeling the system into a partially observable Markov decision process, adopting a centralized training distributed execution framework, and guaranteeing transaction security and data privacy through block chain evidence storage, RSA encryption and an intelligent contract. The system economy and the operation flexibility are improved.

The invention belongs to the field of man-machine interaction, and particularly discloses an intelligent AI dialogue interaction method and system based on an RICH normal form, and the method comprises the steps: obtaining current dialogue context data and a user input signal, processing the data through an intention recognition model, and obtaining a user intention vector and a scene label; according to the user intention vector and the scene label, acquiring hierarchical state information in a preset multilayer interaction framework, and if the matching degree of the vector and the current hierarchical state is lower than a preset threshold value, activating an information sharing mechanism to obtain an updated shared knowledge graph; through the shared knowledge graph, a dynamic scene adaptation module is fused, a potential switching path and intention extension of a conversation are determined, and an optimized interaction path sequence is obtained; the objective of the invention is to solve the problems of inaccurate intention recognition, stiff interaction path switching and incoherent response logic of a traditional dialogue system in a complex scene in the prior art.

The invention relates to a method for docking an external system in a supply chain system, and the method comprises the steps: designing a unified external system interaction framework based on message driving: defining a BaseHandler base class, and integrating getParams parameter analysis, sendRequest request sending and handleResponse response processing logic; external system configuration is stored through the relational database; the business system generates a standardized message and pushes the standardized message to a corresponding MQ theme; and monitoring Topic by the unified consumption service, querying configuration of an enabled state according to the actionCode in the message, dynamically instantiating and specifying a processor subclass, and sequentially executing parameter analysis, external request calling and response processing flows, so as to realize decoupling and dynamic expansion capabilities of business logic and external interaction. According to the method, through unified management configuration and logic, the code repetition rate is reduced, and the maintenance cost is reduced; dynamic configuration supports a quick start-stop docking function and adapts to business change requirements; the standardized process enables the code structure to be clear, and team cooperation and subsequent expansion are facilitated.

The invention provides a hand-eye collaborative interaction system based on eye movement tracking and a pressure ring, which is characterized in that two different modal inputs are subjected to a coarse positioning-fine operation interaction framework with complementary advantages, and specifically, large-range coarse glancing and positioning are completed by using natural and rapid characteristics of eye movement tracking; small-range fine pointing and confirmation are completed by means of the stable and controllable characteristics of the ring pressure sensing device, and the cooperative workflow of'coarse positioning-fine operation 'combining the advantages of rapidness and coarse-grained positioning of eye movement tracking with continuous and high-precision manual control provided by the finger-wearing device is adopted. On the premise that physiological and cognitive burdens of a user are not obviously increased, rapid and stable selection and control of long-distance and high-density virtual objects in the head-mounted display device can be achieved, and coordinate-level precision can be achieved.

The application relates to the technical field of digital twinning, in particular to a highway carbon emission full life cycle accounting system based on digital twinning, which comprises a front-end interaction framework and a back-end accounting framework, the front-end interaction framework and the back-end accounting framework realize data bidirectional interaction through network communication, the front-end interaction framework is used for realizing the human-computer interaction functions of digital twinning visual display, carbon emission accounting data operation and platform management instruction issuing. The application realizes accurate carbon emission full life cycle accounting, solves the pain point of the incomprehensive traditional accounting range, realizes the deep fusion of digital twinning and carbon emission data, improves the visualized control level, realizes multi-module collaborative linkage, improves the accounting efficiency and the control standardization, provides standardized data support, guarantees the consistency of the accounting accuracy, simultaneously, helps the green development of highways, and the system can realize the input of carbon reduction measure data and the accurate accounting of carbon reduction amount.

The invention discloses an LLM-based learner agent simulation learning and forgetting method and a feedback interaction framework with a teacher agent. The method mainly comprises the following steps: S1, a memory mechanism of a learner: mainly obtaining knowledge from a teacher agent through dialogue interaction and summarizing question answering experience of the learner, and storing the knowledge and the experience in a specific vector database in a two-tuple form to simulate memory so as to simulate the learning ability of the learner; s2, according to a forgetting mechanism, after a teacher message is received, a related vector database is retrieved through a vector similarity method, unforgotten memory is obtained and serves as supplement of an original message, the forgotten memory is filled into a prompt template to form a prompt, and the prompt is transmitted into a Monte Carlo Tree Of Though (MCTOT) module. The feedback evaluation is to analyze the evaluation and score of the teacher on the answer, and the score replaces the evaluation and score of the self-evaluation of the learner. Through the steps, the ability of the learner to solve the problem can be improved along with experience accumulation, and staged forgetting can be carried out to better fit the performance of the real learner.

The invention provides a BIM modeling and real-time rendering method and device based on parallel computing, and belongs to the technical field of building information models.The method comprises the steps that a visual interaction framework is built based on an open source rendering engine; based on an open source modeling engine, BIM parametric modeling algorithm interfaces such as sweeping, lofting and Boolean operation are packaged; according to professional modeling and visualization requirements, the BIM modeling and rendering process is decomposed into a plurality of task units, a parallel computing task execution queue is constructed, and dynamic allocation of computing resources is achieved; according to the task execution queue, parallel computing is carried out in batches, and a packaged BIM parametric modeling algorithm interface is called to carry out modeling operation; performing Mesh triangularization processing on the generated BIM model result to generate a Mesh data structure suitable for real-time rendering; hooking the generated Mesh data structure to a scene tree of a visual interaction framework for real-time rendering; and after all tasks in the parallel computing queue are executed, parameter setting after rendering is completed according to user configuration.

This application provides a method and apparatus for processing an interaction event. The method may be performed by a unified interaction framework. Behavior identification of the interaction event is performed by the unified interaction framework, and a widget may directly execute the interaction behavior identified by the unified interaction framework. The widget does not need to adapt to an interactive device. When a software system needs to adapt to a newly added interactive device, a developer can perform adaptation development on the unified interaction framework once to adapt the newly added interactive device for a plurality of widgets.

The invention provides an interactive visual post-processing module design method and device.The method comprises the steps that a data structure for recording calculation results is constructed, and the data structure comprises a calculation result class in aggregation relation with a calculation task class and a calculation result data class in aggregation relation with the calculation result class; the calculation result class is used for realizing interaction and management of calculation results, and the calculation result data class is used for recording calculation data of different calculation steps or iteration steps; constructing a visual model class and a visual grid class in a combination relationship with the visual model class, wherein the visual grid class is matched with the finite element model; the calculation result data class and the visual model class are in an aggregation relationship; and constructing a visual post-processing interaction framework, and constructing association between the visual model class and the visual post-processing interaction framework, so as to realize visual interaction operation on a calculation result. According to the invention, the expandability and maintainability of the module are improved on the premise of ensuring the user interaction experience.

The present invention relates to a method for biological tissue recognition and processing based on task consistency, comprising: constructing a network model based on a unified task-aware consistency interaction framework, the network model including a backbone network for extracting multi-scale image features, a main branch, and at least two additional branches; wherein, the backbone network is based on the encoder-decoder architecture of a deep learning model; the main branch and the at least two additional branches are in a parallel relationship, wherein the tasks executed by the additional branches have strong correlation and weak correlation with the main task respectively; using training sample data to train the network model to obtain a biological tissue recognition and processing training model; using the biological tissue recognition and processing training model to recognize and process biological tissue image data. This method enhances the dynamic ability of deep learning, benefits both supervised and semi-supervised scenarios, and makes it highly effective and robust in dealing with complex challenges of medical image segmentation.

The present invention discloses a multi-modal entity recognition method based on a large language model. S1, construct a multi-modal input data set; S2, determine and apply preprocessing methods for different modal input data through an adaptive modal interaction framework; S3, input the preprocessed multi-modal input data set into a large language model with a modal perception mechanism; S4, generate an optimized fused feature representation; S5, perform cross-modal entity recognition based on the fused feature representation; S6, automatically match the recognized entities with the existing knowledge graph through a dynamic mapping algorithm. If the entity is a new entity, according to the result of the adaptive modal interaction, automatically generate new nodes and update the existing knowledge graph node relationships; S7, by continuously inputting new multi-modal data, the system automatically adjusts and optimizes the modal perception mechanism and the adaptive modal interaction framework of the large language model to dynamically expand the knowledge graph. The present invention improves the accuracy of complex entity recognition.

The invention relates to a lightweight interaction framework implementation method based on a UE5 engine, which provides a global object this.ue5 through a Vue.js plug-in, encapsulates an API (Application Program Interface) calling interface of a UE5 engine function, and is used for supporting a front end to call the UE5 function through a standardized instruction set, and the standardized instruction set comprises instruction types of scene loading, object control and animation triggering. The development efficiency can be remarkably improved, a front-end framework is deeply integrated, a global object this.ue5 is provided through a Vue.js plug-in, an API calling interface of a UE5 engine function is packaged, automatic generation of front-end and rear-end protocols is achieved, and 80% of interface repeated coding workload is reduced. Through a dynamic instruction extension mechanism, instruction hot update is realized based on a yaml configuration file, an engine does not need to be recompiled when functions are newly added, and the development iteration period is shortened by 60%.

The present invention belongs to the technical field of robot motion control and relates to an AUV motion control method and system based on an LLM human-computer interaction framework. The method comprises: acquiring a target AUV and constructing a target AUV simulation platform based on ROS2; integrating LLM and NMPC in ROS2 to form an LLM human-computer interaction framework; acquiring user instructions, and the LLM human-computer interaction framework converting the user instructions into control instructions, and transmitting the control instructions to the target AUV simulation platform; the AUV in the AUV simulation platform executes the control instructions, generates trajectory tracking and displays it. The present invention applies a large language model to the field of underwater robots, combines the human-computer interaction framework with the traditional NMPC control method, utilizes high-level decisions from LLM to realize control of the underlying NMPC, and effectively improves the control flexibility of the AUV in complex experimental scenarios through the interactive display interface of Rviz2.

The embodiment of the invention provides a live broadcast interface display method, which is used for a client, the client is configured with a continuous click interaction framework, and the continuous click interaction framework comprises a plurality of continuous click bar management modules and a public logic processing module; comprising the following steps: receiving continuous click interaction information for a target live broadcast room, wherein the continuous click interaction information comprises an interaction type and continuous click data; according to the interaction type, importing the consecutive click data into a target consecutive click management module in a plurality of consecutive click management modules; and under the condition that the target continuous click bar exists, the continuous click data are transmitted to a public logic processing module through a target continuous click bar management module, so that the continuous click special effect of the target continuous click bar is updated through the public logic processing module. According to the technical scheme provided by the embodiment of the invention, the continuous-click interaction special effect can be displayed on the user interface through the continuous-click interaction framework, and when other functions of the user interface need to join the continuous-click interaction, the continuous-click interaction framework can be quickly developed and online.

The present invention discloses a method for collaborative multimodal interaction perception between humans and drone swarms using gestures, hand shapes, and voice. The method comprises the following steps: Step 1: constructing a speech recognition model for interaction between humans and drone swarms; Step 2: constructing a gesture recognition model for interaction between humans and drone swarms; Step 3: constructing an interaction framework for autonomous recognition of cluster formation collaborative control based on dual models of voice and gesture, inputting information on three interaction channels (hand shape, posture, and voice) to obtain gesture and voice command actions, and adopting a channel fusion implementation mechanism based on dual-channel switching to achieve multimodal interaction in the interaction framework. The present invention realizes multimodal interaction between humans and drone swarms by constructing an interaction framework for autonomous recognition of cluster formation collaborative control based on dual models of voice and gesture and channel fusion based on dual-channel switching. This method can accurately and efficiently achieve intelligent collaborative multimodal perception of humans and drone swarms through language and gestures.

An interactive system is presented for interacting with a user through biosignals and sensor data. The system includes a user input system, an interaction framework, a dynamic user interface (UI), a processor, and a memory. The interaction framework processes biosignal inputs, sensor data, and current context data to provide a context estimation, which may be used to classify user attention and determine the state of selection targets. The dynamic UI receives the UI configuration and LLM / GenAI suggestions or instructions, and provides output to the user. The system can also include additional features, such as presenting selection targets in the idle state, analyzing biosignal inputs and context data to determine user attention, and performing actions based on selection states.

The invention discloses an intelligent interactive collaborative operation system and method based on an AI large model technology, and the system comprises an AI interactive framework module which provides a unified task entrance which supports receiving user input and actively deducing user intentions to provide dynamic suggestions, the suggested contents are derived from input texts, local storage documents, recent access webpages, common service systems and command line tool histories, and meanwhile, content starting and state operations are supported to be directly executed in a window through a shortcut key; the dynamic task disassembling module is used for analyzing a natural language or multimedia instruction into structured sub-tasks based on a large language model and a multi-modal model; and the automatic state synchronization module is used for receiving and storing the execution state of the sub-task in real time through the persistent connection channel.

The present invention relates to a method for implementing a lightweight interaction framework based on the UE5 engine. It provides a global object this.$ue5 through a Vue.js plugin, encapsulates the API call interfaces of the UE5 engine functions, and is used to support the front-end to call UE5 functions through a standardized instruction set. The standardized instruction set includes instruction types for scene loading, object control, and animation triggering. It can significantly improve development efficiency. Deep integration of the front-end framework: By providing a global object this.$ue5 through a Vue.js plugin and encapsulating the API call interfaces of the UE5 engine functions, automatic generation of front-end and back-end protocols is achieved, reducing the workload of interface duplicate coding by 80%. Through a dynamic instruction extension mechanism, instruction hot updates are realized based on a yaml configuration file. When new functions are added, there is no need to recompile the engine, and the development iteration cycle is shortened by 60%.

The invention discloses a Java database connection JDBC connection configuration checking method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring an interactive framework type between a Java virtual machine environment corresponding to a to-be-checked object and a database, and determining a target framework object according to the interactive framework type; obtaining a data source object corresponding to the target framework object, and analyzing the data source object to obtain a corresponding Java database connection uniform resource locator (JDBC) URL; and checking the JDBC URL based on a pre-configured checking strategy to generate a corresponding connection configuration checking result. According to the method and the device, the effectiveness of the JDBC URL configuration can be automatically checked.

The invention relates to a cockpit 3D-HMI multi-screen collaborative interaction method, system, medium and equipment, and relates to the technical field of automobile man-machine interaction. The method comprises the following steps: constructing a 3D-HMI dynamic scene map based on a preset resource library and a vehicle real-time state; based on the 3D-HMI dynamic scene map, performing differential rendering on a plurality of display terminals at different positions in the vehicle cabin to obtain display content of each display terminal; in response to a user interaction instruction received on any display terminal, updating the 3D-HMI dynamic scene map; and synchronously updating the display content on the corresponding display terminal based on the updated 3D-HMI dynamic scene map. Compared with the prior art, the immersion feeling of all passengers in the cabin is greatly enhanced, and the operation intuition and safety are improved; the space in the vehicle is converted into a continuous entertainment and information platform through the multi-screen collaborative 3D HMI, a bottom layer interaction framework is provided for future vehicle-mounted application, and high expandability is achieved.

The invention provides an on-orbit service autonomous task planning method based on large and small model cooperation, and belongs to the technical field of spacecraft on-orbit service intelligent task planning. The problems of lengthy decision chain, high response delay and poor adaptability in current on-orbit service task planning are solved. The method comprises the following steps: constructing a hierarchical intelligent decision-making architecture for large and small model collaboration, wherein the hierarchical intelligent decision-making architecture is used for realizing deep collaboration of large model semantic understanding and small model library calculation; a heterogeneous model interaction framework based on a model context protocol (MCP) is developed and used for data circulation of a large model and a small model; and constructing a general small model library for on-orbit service task planning, wherein the general small model library is used for on-orbit service autonomous task planning. The method is mainly used for aerospace tasks.

This invention discloses an AI interaction system based on a large language model, comprising the following modules: a multimodal input processing unit for collecting and encoding speech, text, images, and structured information to generate interaction state vectors; a target parsing and weight calculation unit for determining multidimensional interaction targets and calculating weight coefficients based on state vectors, task types, and user feedback; an actor subnetwork and a comment subnetwork for generating action policies and state-action value functions, respectively; a dynamic target mapping layer for adjusting the value function relationship between the two networks; a counterfactual trajectory generation unit for generating candidate counterfactual trajectories; a counterfactual coupling layer for calculating reward differences and generating gating signals; and a parameter joint update unit for synchronously optimizing model parameters. This invention significantly improves the policy generalization ability, learning stability, and intelligent decision-making level of the AI ​​interaction system by introducing a dynamic target mapping and counterfactual coupling mechanism into the large language model interaction framework.

Managing memory when processing a large language model (LLM) using a multi-turn interaction framework can be difficult as the LLM can produce significantly more key-value (KV) pairs than can be stored in a processor's memory. The multi-turn framework allows the LLM to process information more efficiently using the KV pairs. The KV pairs can be cached, such as in a KV cache. Policies can be used to identify KV pairs that should remain in the cache, KV pairs that can be moved to a more distant cache, or KV pairs that can be discarded. These policies can assist in managing the memory so the most valuable KV pairs for LLM processing efficiency remain in the processor's local cache memory. More distant cache can be memory locations outside of the processor, or in memory stacks connected via a communication bus.

The invention relates to the technical field of unmanned clusters, and designs an unmanned cluster collaborative navigation method and system based on a distributed cluster self-organizing model. The method comprises the following steps: 1, acquiring a task environment, and preprocessing the task environment; 2, expressing the unmanned cluster platform by adopting characteristics; step 3, carrying out cluster inside and outside interaction framework criterion design based on a Boedes ring on the unmanned cluster platform expressed by the characteristics in the step 2, so that each unmanned platform adjusts the motion state of the unmanned platform according to the motion state of the surrounding platform and the external environment information sensed by a carried sensor; 4, on the basis of the interaction framework criterion designed in the step 3, each unmanned aerial vehicle uses a model prediction controller carried by the unmanned aerial vehicle to calculate the control input of the next moment on line, and the flight state is updated by using a fourth-order Runge-Kutta method; and each unmanned aerial vehicle performs online updating on the flight state of the surrounding unmanned platform by using a sensor carried by the unmanned aerial vehicle and returns to execute the step 3 until the unmanned aerial vehicle arrives at the target point. The method and the device are used for solving the problem of realizing a collaborative navigation behavior of an overall level of a cluster under the condition of reducing communication dependency.

The present invention discloses a coordinated control method for a multi-energy fusion system for a smart community. First, an interactive framework for the multi-energy fusion system of a smart community is constructed. Second, an output model of the photovoltaic power generation system and the wind power generation system, a model of the energy storage equipment, and a model of the power load are constructed within the smart community. Next, a model for the virtual energy storage control capability of electric private cars within the smart community is established. Finally, based on the interactive framework and model of the multi-energy fusion system of a smart community, a method for controlling and collaboratively optimizing the operation of the multi-energy fusion system of a smart community is proposed. The present invention fully utilizes the virtual energy storage control capability of electric private cars to participate in the research of the multi-energy fusion control strategy of the smart community, maximizes the enthusiasm of users to participate in system control, and can effectively realize the multi-energy fusion control and collaborative optimization operation of the smart community.

An interactive system is presented for interacting with a user through biosignals and sensor data. The system includes a user input system, an interaction framework, a dynamic user interface (UI), a processor, and a memory. The interaction framework processes biosignal inputs, sensor data, and current context data to provide a context estimation, which may be used to classify user attention and determine the state of selection targets. The dynamic UI receives the UI configuration and LLM / GenAI suggestions or instructions, and provides output to the user. The system can also include additional features, such as presenting selection targets in the idle state, analyzing biosignal inputs and context data to determine user attention, and performing actions based on selection states.

The invention relates to the technical field of internet big data and information security, designs a mechanical transmission equipment fault diagnosis agent interaction framework and an interaction method by combining the interaction capability of wide area data and agents, and realizes fault diagnosis of mechanical transmission equipment. A fixed classifier synthesized based on an equiangular tight frame is adopted as a classifier of a global diagnosis model, classifier deviation caused by inconsistent data label distribution is relieved, and parameters of a feature extractor and a projection layer in the global diagnosis model are trained, so that the stability of a decision boundary of the global diagnosis model in a fault category imbalance scene is improved, and the fault diagnosis accuracy is improved. And furthermore, by designing a self-adaptive optimization strategy aiming at the local mechanical transmission equipment operation data, the diagnosis and identification precision of the intelligent agent on the mechanical transmission equipment fault based on the local private local operation data by means of the global diagnosis model can be effectively improved.