35 results about "Natural interaction" patented technology

The invention relates to the technical field of electric digital data processing, in particular to an artificial intelligence algorithm-driven digital human language understanding and corresponding reaction interaction method. The method comprises the following steps: collecting historical language data, text data, action instruction data and physical environment information of a user to obtain structured training data; vectorizing the text data to obtain a text semantic vector; environment feature vectors are extracted from the physical environment information; splicing the text semantic vector and the environment feature vector to obtain an environment enhanced text vector; and constructing a pre-training language model based on the environment enhanced text vector, and performing secondary pre-training to generate an optimized language model parameter. According to the invention, by fusing the user language and the physical environment information and combining with the multi-module intelligent component, context perception, efficient understanding and multi-modal natural interaction of the digital human in a complex scene are realized, and the intelligence, adaptability and user experience of the system are remarkably improved.

The invention relates to the technical field of mixed reality and artificial intelligence fusion application, in particular to a mixed reality intelligent inspection method and system based on multi-modal fusion. According to the technical scheme, the method comprises the following steps: synchronously acquiring a user view image and voice stream data through mixed reality equipment; performing target detection on the view image by adopting an improved YOLOv10 model, and outputting a candidate target area and a confidence score # imgabs0 # thereof; voice input is transferred through a Whisper voice recognition model, and a semantic vector # imgabs1 # is generated by using a BERT semantic embedding model. According to the invention, through a multi-mode cooperative sensing mechanism of fusing image, voice and space gazing information, in combination with context semantic optimization and lightweight terminal deployment, intelligent target identification and natural interaction with high precision, real-time performance and strong robustness are realized in a complex industrial inspection scene.

The invention discloses a domestic PC terminal cross-modal natural interaction method and system fused with generative AI, and relates to the technical field of man-machine interaction, and the method comprises the steps: obtaining multi-modal data, and carrying out the preprocessing and feature extraction; constructing a modal adapter, mapping the extracted multi-modal features to a shared semantic space, and generating a multi-modal semantic vector; calculating a dynamic weight coefficient of each mode according to the current scene features; fusing the multi-modal semantic vectors to generate cross-modal fusion features, and carrying out ambiguity resolution processing on the fusion features; inputting the disambiguated fusion features into a lightweight domestic generative AI model, carrying out intention understanding, and generating a multi-modal response; and the generated multi-mode response is adapted to domestic PC terminal hardware and an operating system, and natural interaction is completed. According to the method, intention understanding and response generation are performed by introducing the generative AI domestic large model, so that the accuracy and naturalness of cross-modal interaction of the domestic PC terminal are improved.

The invention relates to the technical field of electric power training, and discloses an electric power training system based on a virtual reality technology, which comprises a hardware interaction module, a data and model module, a core simulation module, an intelligent evaluation module and an application presentation module, the hardware interaction module comprises a VR head-mounted display device, a VR handle controller, a selectable force feedback glove and a selectable universal action platform, and is used for providing immersive experience and natural interaction means for a user; the data and model module comprises a high-precision three-dimensional model library, a physical attribute database, an operation instruction and knowledge graph library, a case library and a fault library, absolute safety and cost reduction and efficiency improvement are achieved, students perform all operations in a highly realistic virtual environment, risks such as personal electric shock, high-altitude falling, equipment damage and the like are completely avoided, and the safety of the students is improved. And one set of system can be repeatedly used without consuming real materials, so that the training cost is greatly reduced.

The application discloses a kind of robot joint control method and system based on motion capture device, belong to robot control technical field.The existing robot joint control scheme, lack effective human-computer cooperation and natural interaction mechanism, leading to robot flexibility is poor, cannot naturally express the intention of operator.A kind of robot joint control method based on motion capture device of the application, by constructing motion capture module, data processing module, coordinate system transformation module, solving optimization module, the rotation posture data of human joint collected using motion capture device is converted into the target joint control information required for robot execution, realize robot joint control based on motion capture device, so that human action can be mapped to robot efficiently and accurately, so that robot flexibility is good, can effectively handle sudden action or complex collaborative operation, so that the intention of operator can be naturally expressed, realize human-computer cooperation and natural interaction.

The invention discloses a lightweight natural interaction control method and system for industrial simulation, and belongs to the technical field of industrial simulation. According to an existing industrial simulation technology, the operation barrier of mobile equipment is high, the interaction mode threshold is high, and the simulation operation requirement of a user in the moving process cannot be met. According to the lightweight natural interaction control method for industrial simulation, the simulation capability registration module, the lightweight interaction unit and the simulation instruction analysis unit are constructed, so that a user does not need to spend a lot of time to learn industrial simulation operation logic and directly describe simulation requirements in an oral manner, and an industrial simulation result can be obtained; therefore, the problems that an existing industrial simulation technology is high in operation barrier and low in interaction efficiency can be effectively solved, the technical threshold for operating professional simulation software on mobile equipment is effectively lowered, the simulation technology can be used in a mobile scene, the simulation operation requirement of a user in the moving process can be met, and the user experience is improved. And the simulation technology can be popularized conveniently.

Provided are a control device, a control method, and a control program capable of realizing natural interaction between a human and a robot like between humans. A control device according to an embodiment includes a slip detection unit (15, 104) and a control unit (10). The slip detection unit detects a slip of a target object gripped by a grip portion. The control unit controls a gripping force with which the grip portion grips the target object based on the slip detected by the slip detection unit. The control unit estimates an external force applied to the target object gripped by the grip portion based on the slip detected by the slip detection unit, and controls the gripping force based on the estimated external force.

The invention discloses an interaction method and device, electronic equipment and a storage medium, and belongs to the technical field of computers. The method includes displaying, at a display component, a computer-generated three-dimensional environment; presenting an interactive object in the three-dimensional environment; obtaining interaction information input by a user through an input device; inputting the interaction information to an interaction manager; when the interaction manager judges that the interaction information meets the current interaction scene, a target interaction object corresponding to the interaction information is determined; and when the target interaction object receives the interaction information, executing an operation corresponding to the interaction information on the target interaction object. According to the invention, rich and natural interaction experience can be provided for the user.

The invention belongs to the field of tourism recommendation, and provides a tourism recommendation method and system based on multi-modal data, and the method comprises the steps: constructing a multi-modal tourism causality cognitive map; natural interaction information input by a user is collected, and tourism intention variables and implicit preference features of the user are extracted through an intention recognition module; performing forward causal reasoning in the causal cognitive map based on the tourism intention variable, determining a target scenic spot sequence meeting the intention of the user, and performing path feasibility evaluation; according to the target scenic spot sequence, executing an anti-fact reasoning operation in a cognitive map, identifying adverse factors and reasons thereof in a potential path, and performing interpretation enhancement and risk avoidance correction on a recommended path; and a recommendation result is output, the result comprises a recommendation sequence of multiple paths, and each path is attached with user intention interpretation, a causal path diagram, a risk analysis diagram and an accessibility weight score.

The application discloses a kind of man-machine natural interaction method and system based on embodied intelligence, and the application relates to information interaction communication technical field, solve the technical problem that the segment processing of user continuous action or complex instruction lacks dynamic adaptability, the present application is segmented based on motion trajectory turning point and voice pause dynamically, support personalized adjustment, improve instruction segmentation accuracy, solve continuous action semantic segmentation ambiguity, introduce dynamic time regulation+cosine similarity multilayer matching algorithm, preferentially screen the same position pre-stored instruction, and support combination instruction reorganization, improve the efficiency of abnormal instruction repair, reduce the secondary input burden of user, improve the fluency of interaction, simultaneously build "recognition-verification-reorganization-execution" closed loop, by similarity mean evaluation preferred combination instruction, and support incremental learning, improve system adaptability.

The invention discloses an AR (Augmented Reality) multi-modal fusion natural interaction method and device, a medium and a product, and relates to the technical field of AR, the method comprises the following steps: constructing an implicit context interaction intention knowledge base; obtaining interaction modal information in a set time period; calling an AR gesture deep interaction intention, an AR real object deep interaction intention and an AR touch deep interaction intention respectively corresponding to the interaction modal information from an implicit context interaction intention knowledge base to obtain an AR interaction intention set; obtaining a real interaction intention based on the AR interaction intention set; determining whether the real interaction intention meets a trust degree requirement or not, and obtaining an executable real interaction intention; and the AR interaction system completes an AR implement interaction task based on the executable real interaction intention. According to the method and the device, efficient, accurate, stable and robust reasoning of an executable real interaction intention in AR body interaction can be realized, and the naturalness and experience of the AR body interaction are enhanced.

The application discloses a hand detection method based on a RetinaHand light infrared image, and comprises the following steps: S100, generating a hand region image by using a hand detection network based on RetinaHand; and S200, performing enhancement processing on the generated hand region image. The method has the characteristics of short delay, accurate hand detection positioning and real-time generation support, and can be widely used in natural interaction in the fields of intelligent vehicles, intelligent homes and robots.

This invention relates to the field of intelligent interaction for children's growth, and discloses a growth understanding and intelligent feedback system and method based on children's natural interaction. The system includes acquiring multimodal basic data samples and converting them into behavioral feature vectors of a unified dimension; calculating the similarity parameter between the behavioral feature vectors and the basic cognitive schema set; constructing a staged cognitive schema model by combining age-specific weight constraints and time decay variables; extracting the schema feature vectors in the activated state to generate growth analysis results; performing a split mapping on the growth analysis results to convert them into summary information and interaction strategy parameters; and dynamically adjusting the interactive operation logic of the intelligent agent according to the closed-loop interaction strategy parameters. This invention avoids data distortion through a low-intervention data acquisition architecture, improves the accuracy of growth analysis and supports continuous understanding; eliminates exploration noise through staged cognitive schema modeling, improves the stability and interpretability of the results, and enhances the adaptability and continuity of the intelligent agent's interaction.

This invention discloses a 3D virtual digital human interaction system and method, relating to the fields of human-computer interaction and artificial intelligence. It addresses the problem of traditional methods relying on preset templates, lacking natural and personalized expression, and resulting in poor interaction adaptability. The system obtains the current behavioral parameters of the virtual digital human and the user's interaction intent, calculates basic action logic, and initially matches the action model range according to the interaction type. During the interaction phase, it detects the signal strength of the input device, sets strength conditions to extract the data stream, establishes initial response logic, collects dynamic behavior, and constructs a random forest model to predict behavioral trends. It combines the prediction results with the model for adaptability classification; if the adaptability is low, it collects interaction samples through an auxiliary channel for visualization simulation, and filters suitable samples and models. Based on the simulation results, it matches the interaction scene, replaces the initial scene, and executes actions, achieving intelligent response and natural interaction of the virtual digital human.

The invention discloses a teleoperation control system fusing multi-modal natural interaction and intelligent decision. The teleoperation control system comprises a teleoperation control box, a multi-modal natural interaction module, an intelligent decision fusion module and a human-computer interaction UI interface, the multi-mode natural interaction module designs natural mapping of eye movement control, gesture control and robot control quantity, and supports handle control, rocker control and voice control at the same time; and the intelligent decision fusion module adaptively adjusts the fusion weight of the corresponding mode based on the stability and use frequency of each interaction mode. According to the invention, an operation mode close to nature is realized through the multi-modal natural interaction module, and a self-adaptive weight adjustment mechanism of the intelligent decision fusion module is combined, so that the stability and accuracy of teleoperation control are remarkably improved, and the fatigue of operators is effectively reduced. Meanwhile, the system has a rapid deployment capability, and has high integration and excellent operation experience.

The invention discloses an interaction system and method of a 3D virtual digital human, relates to the technical field of man-machine interaction and artificial intelligence, and is used for solving the problem of poor interaction adaptability caused by dependence on a preset template and lack of natural and personalized expression in a traditional method. Preliminarily matching an action model range according to an interaction type; in the interaction stage, the signal intensity of input equipment is detected, an intensity condition is set to extract a data stream, initial response logic is established, dynamic behaviors are collected, and a random forest model is constructed to predict a behavior trend; matching classification is carried out by combining the prediction result and the model, if the matching is low, interaction samples are collected through an auxiliary channel to carry out visualized simulation, and matching samples and models are screened; and matching an interaction scene based on a simulation result, replacing the initial scene and then executing an action, thereby realizing intelligent response and natural interaction of the virtual digital human.

The invention discloses an implicit knowledge explicit multichannel man-machine collaborative natural interaction method, which comprises the following steps of: (1) analyzing collaborative task sensing nodes taking an operator as a center to obtain a sensing information flow model in a man-machine-ring in a collaborative scene, and researching an explicit and implicit knowledge conversion chain in a man-machine collaborative combat process; step (2), constructing an SECI model of knowledge conversion, representing that there is explicit and implicit knowledge in a collaborative combat man-machine loop closed-loop system, and obtaining knowledge conversion node importance weight under a man-machine collaborative task; step (3), according to knowledge conversion node importance weights, carrying out an eye movement physiological evaluation experiment to obtain a behavior cognitive map fusing multiple perception channels, and explicitly expressing tacit knowledge in multichannel interaction behaviors; and step (4), outputting a multi-channel man-machine collaborative natural interaction method according to the behavior cognitive map. The method provides reference for man-machine collaborative interaction design of a multi-channel interaction system by mining behavior interaction implicit knowledge of an operator.

The present invention belongs to the technical field of interface recommendation, and provides an adaptive user interface recommendation method and system based on foot interaction. By using the foot length difference of users, the sizes of elements in the interaction interface are designed; the theme style and element shape in the interaction interface are recommended by using the deep learning field perception decomposition machine method; at the same time, an interface library with different layouts is constructed by using interface design rules, and the element layout in the interaction interface is recommended by using the reinforcement learning multi-armed bandit UCB1 algorithm. Through the personalized design of elements and layouts in the interaction interface, the possible misoperations that may occur during user movement are reduced or avoided, the accuracy of the user's natural interaction with the interface using the foot is improved, the efficiency of human-computer interaction is increased, and the user's needs are realized.

An object of a system according to an embodiment is to provide a place for matching and natural interaction in consideration of compatibility between employees.SOLUTION: A system includes a profile analysis part, a matching part, and an event planning part. A profile analysis part analyzes the profile, working method and sense of values of the employee. The matching unit matches co-workers who are compatible in character based on the data analyzed by the profile analysis unit. The event planning unit plans casual coffee break, lunch, and after-work events in order to provide a place where the employees matched by the matching unit are naturally connected to each other.SELECTED DRAWING: Figure 1

The application discloses an intelligent agent-based multimodal interaction intelligence application implementation method and device and a terminal, relates to the technical field of human-computer interaction, and comprises the following steps: controlling starting of an interactive intelligence application corresponding to a playing item, and identifying a player user; performing multimodal interaction input identification: capturing the player user's body movement, gesture, expression and position in real time, and identifying an entity prop, so as to convert a physical space where the player user is located into a game scene; receiving the player user's voice answer, instruction and natural interaction; performing real-time judgment and processing on the multimodal input of the player user, and controlling corresponding feedback and display in the converted physical space game scene according to the player user's operation on different playing items; continuously tracking the success rate of the player user's operation on different playing items, and dynamically adjusting the corresponding game difficulty. The application has the advantages of protecting visual health, supporting diversified interaction input, promoting multi-person social interaction, and adaptively adjusting game difficulty.

The invention discloses a man-machine natural interaction method and a man-machine natural interaction system based on intellectualization, relates to the technical field of information interaction communication, and solves the technical problem that segmentation processing of continuous actions or complex instructions of a user lacks dynamic adaptability. Individual adjustment is supported, instruction segmentation accuracy is improved, continuous action semantic segmentation ambiguity is solved, a multi-layer matching algorithm of dynamic time warping and cosine similarity is introduced, pre-stored instructions at the same position are screened preferentially, combination instruction recombination is supported, abnormal instruction repair efficiency is improved, secondary input burden of a user is reduced, interaction fluency is improved, and user experience is improved. And meanwhile, a closed loop of identification-verification-recombination-execution is constructed, the optimal combination instruction is evaluated through a similarity mean value, incremental learning is supported, and the adaptability of the system is improved.

The invention belongs to the technical field of acoustic virtual environment and natural man-machine interaction, and discloses an acoustic virtual environment intelligent construction and natural interaction system and method. In order to solve the problems that in the prior art, a virtual acoustic environment is insufficient in sense of reality, acoustic interaction response lags behind, and multi-modal acoustic fusion is stiff, four invention points including a dynamic space acoustic modeling algorithm, a context perception acoustic interaction model, a multi-modal acoustic natural fusion mechanism and a global cooperative processing framework are provided. The method comprises two innovative algorithm formulas of space acoustic parameter dynamic adjustment and acoustic interaction instruction generation. By constructing a modeling-interaction-fusion global collaborative closed loop, the reality sense of a virtual acoustic environment is remarkably improved, the acoustic interaction response timeliness is greatly enhanced, the multi-modal acoustic fusion naturalness is remarkably improved, and the method is suitable for various acoustic virtual scenes such as virtual reality, remote collaboration and intelligent interaction. And the intelligent level of acoustic virtual environment construction and interaction is comprehensively improved.

The invention discloses an interactive digital twinning enhancement modeling method. The method comprises the following steps: constructing a digital twinning model which is dynamically updated in real time; building a natural interaction interface system comprising a voice interaction module, a gesture recognition module and a tactile feedback module; a simulation calculation result is fed back in a three-dimensional visualization mode, meanwhile, virtual physical attributes of the operation object are output through a tactile feedback module, and optimized parameters are synchronized to a physical entity control system according to a user confirmation instruction. The method has the beneficial effects that according to the scheme, through data layer optimization, natural interaction, precise simulation, visual feedback and self-optimization closed loop, the interactivity, accuracy and practicability of the digital twin model are comprehensively improved, and powerful technical support is provided for optimization decision making in a complex scene.

A system is provided.SOLUTION: The system includes a means for enabling a user to input ideal appearance and personality setting information, a generation means for generating a virtual partner on the basis of the input appearance and personality setting information, and a communication means for storing a conversation history with the user and providing natural exchange.SELECTED DRAWING: Figure 1

The invention relates to an electric quantity data analysis code self-generation method and device based on natural interaction, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps that a natural language instruction used for describing electric quantity data analysis processing is acquired, and the natural language instruction is converted into a word segmentation sequence; performing semantic mapping from the power data knowledge base according to the word segmentation sequence to obtain a structured semantic mapping object corresponding to the natural language instruction; performing semantic role label labeling on each segmented word in the segmented word sequence to obtain a labeling sequence corresponding to the segmented word sequence; generating a logic tree according to each segmented word, the respective attention weight of each segmented word and the respective semantic role label of each segmented word in the labeling sequence; based on the logic tree and the structured semantic mapping object, an executable query code is generated, and the executable query code is used for executing electric quantity data analysis processing. By adopting the method, the processing efficiency of electric quantity data analysis can be improved.

This invention belongs to the field of intelligent natural interaction and visual picking control for robotic arms. Specifically, it is a method for collaborative robotic arm voice natural interaction and target recognition, localization, and picking control based on binocular depth vision perception, fusion of multimodal large models, and multi-agent collaborative decision-making. The method defines standardized tool / action units; the main agent understands voice commands and routes them to the action planning and RAG question-answering agents; the action planning agent generates a structured tool call sequence with unified parameter specifications and describes the action sequence and data dependencies using a directed graph; the system triggers binocular acquisition and calls a visual detection algorithm library, fuses depth to achieve 3D localization and grasping pose generation, performs grasping and placement after reachability and safety checks, and replans based on feedback backtracking to form a closed-loop picking process.

The invention provides an Internet of Things semantic control method based on large language model tool calling, and the method specifically comprises the following steps: S1, defining an Internet of Things equipment control API as a tool set which can be understood by a large language model, and registering the tool set in the pre-trained large language model; and S2, calling the Internet of Things equipment control tool according to a user instruction based on the pre-trained large language model. According to the proposal of the invention, a big language model tool calling method is adopted, semantic understanding and analysis are performed on prompt words input by a user, and corresponding Internet of Things equipment is called to control APIs or functions, so that the purpose of being closer to natural interaction of human languages is achieved.

The invention discloses an industrial man-machine natural interaction and task triggering method and system based on intention recognition. The method comprises the steps that an industrial field multi-mode interactive interface is constructed, and multiple input modes of voice, gestures and texts are supported; designing an intention recognition model based on deep learning, and accurately analyzing a user operation intention; establishing a mapping mechanism from an intention to a manufacturing task, and automatically triggering a corresponding business process; context awareness and adaptive optimization of the interaction process are realized; and constructing an interaction quality evaluation and continuous improvement framework. The system comprises a multi-mode interaction module, an intention recognition module, a task mapping module, a context awareness module and an evaluation optimization module. The problems that a traditional industrial system is complex in interaction and high in operation threshold are solved, and intelligent natural interaction and automatic task triggering in an industrial scene are achieved.

This invention relates to the field of robot interaction technology, specifically providing a multimodal interaction method based on robot behavior recognition, including robot initialization, dynamic behavior modeling, intent parsing, multimodal decision-making, and proactive service execution. The method involves real-time acquisition of user body movements using the robot's multi-axis sensors, combined with environmental state data to construct a spatiotemporal behavior model; parsing the action intent using an improved spatiotemporal graph convolutional network (ST-GCN) to generate dynamic behavioral semantic encoding; and integrating dialogue context and behavioral semantic data into a large multimodal model to generate proactive interaction strategies that conform to the user's behavioral intent. This invention solves the problem of traditional interaction systems' lack of understanding of body language, and is particularly suitable for scenarios such as children's education and rehabilitation training, achieving natural interaction where "action is command."