31 results about "Natural interaction" patented technology

The invention relates to the technical field of electric digital data processing, in particular to an artificial intelligence algorithm-driven digital human language understanding and corresponding reaction interaction method. The method comprises the following steps: collecting historical language data, text data, action instruction data and physical environment information of a user to obtain structured training data; vectorizing the text data to obtain a text semantic vector; environment feature vectors are extracted from the physical environment information; splicing the text semantic vector and the environment feature vector to obtain an environment enhanced text vector; and constructing a pre-training language model based on the environment enhanced text vector, and performing secondary pre-training to generate an optimized language model parameter. According to the invention, by fusing the user language and the physical environment information and combining with the multi-module intelligent component, context perception, efficient understanding and multi-modal natural interaction of the digital human in a complex scene are realized, and the intelligence, adaptability and user experience of the system are remarkably improved.

The invention relates to the technical field of mixed reality and artificial intelligence fusion application, in particular to a mixed reality intelligent inspection method and system based on multi-modal fusion. According to the technical scheme, the method comprises the following steps: synchronously acquiring a user view image and voice stream data through mixed reality equipment; performing target detection on the view image by adopting an improved YOLOv10 model, and outputting a candidate target area and a confidence score # imgabs0 # thereof; voice input is transferred through a Whisper voice recognition model, and a semantic vector # imgabs1 # is generated by using a BERT semantic embedding model. According to the invention, through a multi-mode cooperative sensing mechanism of fusing image, voice and space gazing information, in combination with context semantic optimization and lightweight terminal deployment, intelligent target identification and natural interaction with high precision, real-time performance and strong robustness are realized in a complex industrial inspection scene.

The invention discloses a domestic PC terminal cross-modal natural interaction method and system fused with generative AI, and relates to the technical field of man-machine interaction, and the method comprises the steps: obtaining multi-modal data, and carrying out the preprocessing and feature extraction; constructing a modal adapter, mapping the extracted multi-modal features to a shared semantic space, and generating a multi-modal semantic vector; calculating a dynamic weight coefficient of each mode according to the current scene features; fusing the multi-modal semantic vectors to generate cross-modal fusion features, and carrying out ambiguity resolution processing on the fusion features; inputting the disambiguated fusion features into a lightweight domestic generative AI model, carrying out intention understanding, and generating a multi-modal response; and the generated multi-mode response is adapted to domestic PC terminal hardware and an operating system, and natural interaction is completed. According to the method, intention understanding and response generation are performed by introducing the generative AI domestic large model, so that the accuracy and naturalness of cross-modal interaction of the domestic PC terminal are improved.

The invention relates to the technical field of electric power training, and discloses an electric power training system based on a virtual reality technology, which comprises a hardware interaction module, a data and model module, a core simulation module, an intelligent evaluation module and an application presentation module, the hardware interaction module comprises a VR head-mounted display device, a VR handle controller, a selectable force feedback glove and a selectable universal action platform, and is used for providing immersive experience and natural interaction means for a user; the data and model module comprises a high-precision three-dimensional model library, a physical attribute database, an operation instruction and knowledge graph library, a case library and a fault library, absolute safety and cost reduction and efficiency improvement are achieved, students perform all operations in a highly realistic virtual environment, risks such as personal electric shock, high-altitude falling, equipment damage and the like are completely avoided, and the safety of the students is improved. And one set of system can be repeatedly used without consuming real materials, so that the training cost is greatly reduced.

The application discloses a kind of robot joint control method and system based on motion capture device, belong to robot control technical field.The existing robot joint control scheme, lack effective human-computer cooperation and natural interaction mechanism, leading to robot flexibility is poor, cannot naturally express the intention of operator.A kind of robot joint control method based on motion capture device of the application, by constructing motion capture module, data processing module, coordinate system transformation module, solving optimization module, the rotation posture data of human joint collected using motion capture device is converted into the target joint control information required for robot execution, realize robot joint control based on motion capture device, so that human action can be mapped to robot efficiently and accurately, so that robot flexibility is good, can effectively handle sudden action or complex collaborative operation, so that the intention of operator can be naturally expressed, realize human-computer cooperation and natural interaction.

The invention discloses a lightweight natural interaction control method and system for industrial simulation, and belongs to the technical field of industrial simulation. According to an existing industrial simulation technology, the operation barrier of mobile equipment is high, the interaction mode threshold is high, and the simulation operation requirement of a user in the moving process cannot be met. According to the lightweight natural interaction control method for industrial simulation, the simulation capability registration module, the lightweight interaction unit and the simulation instruction analysis unit are constructed, so that a user does not need to spend a lot of time to learn industrial simulation operation logic and directly describe simulation requirements in an oral manner, and an industrial simulation result can be obtained; therefore, the problems that an existing industrial simulation technology is high in operation barrier and low in interaction efficiency can be effectively solved, the technical threshold for operating professional simulation software on mobile equipment is effectively lowered, the simulation technology can be used in a mobile scene, the simulation operation requirement of a user in the moving process can be met, and the user experience is improved. And the simulation technology can be popularized conveniently.

The invention belongs to the field of tourism recommendation, and provides a tourism recommendation method and system based on multi-modal data, and the method comprises the steps: constructing a multi-modal tourism causality cognitive map; natural interaction information input by a user is collected, and tourism intention variables and implicit preference features of the user are extracted through an intention recognition module; performing forward causal reasoning in the causal cognitive map based on the tourism intention variable, determining a target scenic spot sequence meeting the intention of the user, and performing path feasibility evaluation; according to the target scenic spot sequence, executing an anti-fact reasoning operation in a cognitive map, identifying adverse factors and reasons thereof in a potential path, and performing interpretation enhancement and risk avoidance correction on a recommended path; and a recommendation result is output, the result comprises a recommendation sequence of multiple paths, and each path is attached with user intention interpretation, a causal path diagram, a risk analysis diagram and an accessibility weight score.

The application discloses a kind of man-machine natural interaction method and system based on embodied intelligence, and the application relates to information interaction communication technical field, solve the technical problem that the segment processing of user continuous action or complex instruction lacks dynamic adaptability, the present application is segmented based on motion trajectory turning point and voice pause dynamically, support personalized adjustment, improve instruction segmentation accuracy, solve continuous action semantic segmentation ambiguity, introduce dynamic time regulation+cosine similarity multilayer matching algorithm, preferentially screen the same position pre-stored instruction, and support combination instruction reorganization, improve the efficiency of abnormal instruction repair, reduce the secondary input burden of user, improve the fluency of interaction, simultaneously build "recognition-verification-reorganization-execution" closed loop, by similarity mean evaluation preferred combination instruction, and support incremental learning, improve system adaptability.

The invention discloses an AR (Augmented Reality) multi-modal fusion natural interaction method and device, a medium and a product, and relates to the technical field of AR, the method comprises the following steps: constructing an implicit context interaction intention knowledge base; obtaining interaction modal information in a set time period; calling an AR gesture deep interaction intention, an AR real object deep interaction intention and an AR touch deep interaction intention respectively corresponding to the interaction modal information from an implicit context interaction intention knowledge base to obtain an AR interaction intention set; obtaining a real interaction intention based on the AR interaction intention set; determining whether the real interaction intention meets a trust degree requirement or not, and obtaining an executable real interaction intention; and the AR interaction system completes an AR implement interaction task based on the executable real interaction intention. According to the method and the device, efficient, accurate, stable and robust reasoning of an executable real interaction intention in AR body interaction can be realized, and the naturalness and experience of the AR body interaction are enhanced.

The application discloses a hand detection method based on a RetinaHand light infrared image, and comprises the following steps: S100, generating a hand region image by using a hand detection network based on RetinaHand; and S200, performing enhancement processing on the generated hand region image. The method has the characteristics of short delay, accurate hand detection positioning and real-time generation support, and can be widely used in natural interaction in the fields of intelligent vehicles, intelligent homes and robots.

This invention relates to the field of intelligent interaction for children's growth, and discloses a growth understanding and intelligent feedback system and method based on children's natural interaction. The system includes acquiring multimodal basic data samples and converting them into behavioral feature vectors of a unified dimension; calculating the similarity parameter between the behavioral feature vectors and the basic cognitive schema set; constructing a staged cognitive schema model by combining age-specific weight constraints and time decay variables; extracting the schema feature vectors in the activated state to generate growth analysis results; performing a split mapping on the growth analysis results to convert them into summary information and interaction strategy parameters; and dynamically adjusting the interactive operation logic of the intelligent agent according to the closed-loop interaction strategy parameters. This invention avoids data distortion through a low-intervention data acquisition architecture, improves the accuracy of growth analysis and supports continuous understanding; eliminates exploration noise through staged cognitive schema modeling, improves the stability and interpretability of the results, and enhances the adaptability and continuity of the intelligent agent's interaction.

This invention discloses a 3D virtual digital human interaction system and method, relating to the fields of human-computer interaction and artificial intelligence. It addresses the problem of traditional methods relying on preset templates, lacking natural and personalized expression, and resulting in poor interaction adaptability. The system obtains the current behavioral parameters of the virtual digital human and the user's interaction intent, calculates basic action logic, and initially matches the action model range according to the interaction type. During the interaction phase, it detects the signal strength of the input device, sets strength conditions to extract the data stream, establishes initial response logic, collects dynamic behavior, and constructs a random forest model to predict behavioral trends. It combines the prediction results with the model for adaptability classification; if the adaptability is low, it collects interaction samples through an auxiliary channel for visualization simulation, and filters suitable samples and models. Based on the simulation results, it matches the interaction scene, replaces the initial scene, and executes actions, achieving intelligent response and natural interaction of the virtual digital human.

The invention discloses a teleoperation control system fusing multi-modal natural interaction and intelligent decision. The teleoperation control system comprises a teleoperation control box, a multi-modal natural interaction module, an intelligent decision fusion module and a human-computer interaction UI interface, the multi-mode natural interaction module designs natural mapping of eye movement control, gesture control and robot control quantity, and supports handle control, rocker control and voice control at the same time; and the intelligent decision fusion module adaptively adjusts the fusion weight of the corresponding mode based on the stability and use frequency of each interaction mode. According to the invention, an operation mode close to nature is realized through the multi-modal natural interaction module, and a self-adaptive weight adjustment mechanism of the intelligent decision fusion module is combined, so that the stability and accuracy of teleoperation control are remarkably improved, and the fatigue of operators is effectively reduced. Meanwhile, the system has a rapid deployment capability, and has high integration and excellent operation experience.

The invention discloses an interaction system and method of a 3D virtual digital human, relates to the technical field of man-machine interaction and artificial intelligence, and is used for solving the problem of poor interaction adaptability caused by dependence on a preset template and lack of natural and personalized expression in a traditional method. Preliminarily matching an action model range according to an interaction type; in the interaction stage, the signal intensity of input equipment is detected, an intensity condition is set to extract a data stream, initial response logic is established, dynamic behaviors are collected, and a random forest model is constructed to predict a behavior trend; matching classification is carried out by combining the prediction result and the model, if the matching is low, interaction samples are collected through an auxiliary channel to carry out visualized simulation, and matching samples and models are screened; and matching an interaction scene based on a simulation result, replacing the initial scene and then executing an action, thereby realizing intelligent response and natural interaction of the virtual digital human.

An object of a system according to an embodiment is to provide a place for matching and natural interaction in consideration of compatibility between employees.SOLUTION: A system includes a profile analysis part, a matching part, and an event planning part. A profile analysis part analyzes the profile, working method and sense of values of the employee. The matching unit matches co-workers who are compatible in character based on the data analyzed by the profile analysis unit. The event planning unit plans casual coffee break, lunch, and after-work events in order to provide a place where the employees matched by the matching unit are naturally connected to each other.SELECTED DRAWING: Figure 1

The application discloses an intelligent agent-based multimodal interaction intelligence application implementation method and device and a terminal, relates to the technical field of human-computer interaction, and comprises the following steps: controlling starting of an interactive intelligence application corresponding to a playing item, and identifying a player user; performing multimodal interaction input identification: capturing the player user's body movement, gesture, expression and position in real time, and identifying an entity prop, so as to convert a physical space where the player user is located into a game scene; receiving the player user's voice answer, instruction and natural interaction; performing real-time judgment and processing on the multimodal input of the player user, and controlling corresponding feedback and display in the converted physical space game scene according to the player user's operation on different playing items; continuously tracking the success rate of the player user's operation on different playing items, and dynamically adjusting the corresponding game difficulty. The application has the advantages of protecting visual health, supporting diversified interaction input, promoting multi-person social interaction, and adaptively adjusting game difficulty.

The invention discloses a man-machine natural interaction method and a man-machine natural interaction system based on intellectualization, relates to the technical field of information interaction communication, and solves the technical problem that segmentation processing of continuous actions or complex instructions of a user lacks dynamic adaptability. Individual adjustment is supported, instruction segmentation accuracy is improved, continuous action semantic segmentation ambiguity is solved, a multi-layer matching algorithm of dynamic time warping and cosine similarity is introduced, pre-stored instructions at the same position are screened preferentially, combination instruction recombination is supported, abnormal instruction repair efficiency is improved, secondary input burden of a user is reduced, interaction fluency is improved, and user experience is improved. And meanwhile, a closed loop of identification-verification-recombination-execution is constructed, the optimal combination instruction is evaluated through a similarity mean value, incremental learning is supported, and the adaptability of the system is improved.

The invention belongs to the technical field of acoustic virtual environment and natural man-machine interaction, and discloses an acoustic virtual environment intelligent construction and natural interaction system and method. In order to solve the problems that in the prior art, a virtual acoustic environment is insufficient in sense of reality, acoustic interaction response lags behind, and multi-modal acoustic fusion is stiff, four invention points including a dynamic space acoustic modeling algorithm, a context perception acoustic interaction model, a multi-modal acoustic natural fusion mechanism and a global cooperative processing framework are provided. The method comprises two innovative algorithm formulas of space acoustic parameter dynamic adjustment and acoustic interaction instruction generation. By constructing a modeling-interaction-fusion global collaborative closed loop, the reality sense of a virtual acoustic environment is remarkably improved, the acoustic interaction response timeliness is greatly enhanced, the multi-modal acoustic fusion naturalness is remarkably improved, and the method is suitable for various acoustic virtual scenes such as virtual reality, remote collaboration and intelligent interaction. And the intelligent level of acoustic virtual environment construction and interaction is comprehensively improved.

The invention discloses an interactive digital twinning enhancement modeling method. The method comprises the following steps: constructing a digital twinning model which is dynamically updated in real time; building a natural interaction interface system comprising a voice interaction module, a gesture recognition module and a tactile feedback module; a simulation calculation result is fed back in a three-dimensional visualization mode, meanwhile, virtual physical attributes of the operation object are output through a tactile feedback module, and optimized parameters are synchronized to a physical entity control system according to a user confirmation instruction. The method has the beneficial effects that according to the scheme, through data layer optimization, natural interaction, precise simulation, visual feedback and self-optimization closed loop, the interactivity, accuracy and practicability of the digital twin model are comprehensively improved, and powerful technical support is provided for optimization decision making in a complex scene.

A system is provided.SOLUTION: The system includes a means for enabling a user to input ideal appearance and personality setting information, a generation means for generating a virtual partner on the basis of the input appearance and personality setting information, and a communication means for storing a conversation history with the user and providing natural exchange.SELECTED DRAWING: Figure 1

The invention relates to an electric quantity data analysis code self-generation method and device based on natural interaction, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps that a natural language instruction used for describing electric quantity data analysis processing is acquired, and the natural language instruction is converted into a word segmentation sequence; performing semantic mapping from the power data knowledge base according to the word segmentation sequence to obtain a structured semantic mapping object corresponding to the natural language instruction; performing semantic role label labeling on each segmented word in the segmented word sequence to obtain a labeling sequence corresponding to the segmented word sequence; generating a logic tree according to each segmented word, the respective attention weight of each segmented word and the respective semantic role label of each segmented word in the labeling sequence; based on the logic tree and the structured semantic mapping object, an executable query code is generated, and the executable query code is used for executing electric quantity data analysis processing. By adopting the method, the processing efficiency of electric quantity data analysis can be improved.

This invention belongs to the field of intelligent natural interaction and visual picking control for robotic arms. Specifically, it is a method for collaborative robotic arm voice natural interaction and target recognition, localization, and picking control based on binocular depth vision perception, fusion of multimodal large models, and multi-agent collaborative decision-making. The method defines standardized tool / action units; the main agent understands voice commands and routes them to the action planning and RAG question-answering agents; the action planning agent generates a structured tool call sequence with unified parameter specifications and describes the action sequence and data dependencies using a directed graph; the system triggers binocular acquisition and calls a visual detection algorithm library, fuses depth to achieve 3D localization and grasping pose generation, performs grasping and placement after reachability and safety checks, and replans based on feedback backtracking to form a closed-loop picking process.

The invention provides an Internet of Things semantic control method based on large language model tool calling, and the method specifically comprises the following steps: S1, defining an Internet of Things equipment control API as a tool set which can be understood by a large language model, and registering the tool set in the pre-trained large language model; and S2, calling the Internet of Things equipment control tool according to a user instruction based on the pre-trained large language model. According to the proposal of the invention, a big language model tool calling method is adopted, semantic understanding and analysis are performed on prompt words input by a user, and corresponding Internet of Things equipment is called to control APIs or functions, so that the purpose of being closer to natural interaction of human languages is achieved.

The invention discloses an industrial man-machine natural interaction and task triggering method and system based on intention recognition. The method comprises the steps that an industrial field multi-mode interactive interface is constructed, and multiple input modes of voice, gestures and texts are supported; designing an intention recognition model based on deep learning, and accurately analyzing a user operation intention; establishing a mapping mechanism from an intention to a manufacturing task, and automatically triggering a corresponding business process; context awareness and adaptive optimization of the interaction process are realized; and constructing an interaction quality evaluation and continuous improvement framework. The system comprises a multi-mode interaction module, an intention recognition module, a task mapping module, a context awareness module and an evaluation optimization module. The problems that a traditional industrial system is complex in interaction and high in operation threshold are solved, and intelligent natural interaction and automatic task triggering in an industrial scene are achieved.

This invention relates to the field of robot interaction technology, specifically providing a multimodal interaction method based on robot behavior recognition, including robot initialization, dynamic behavior modeling, intent parsing, multimodal decision-making, and proactive service execution. The method involves real-time acquisition of user body movements using the robot's multi-axis sensors, combined with environmental state data to construct a spatiotemporal behavior model; parsing the action intent using an improved spatiotemporal graph convolutional network (ST-GCN) to generate dynamic behavioral semantic encoding; and integrating dialogue context and behavioral semantic data into a large multimodal model to generate proactive interaction strategies that conform to the user's behavioral intent. This invention solves the problem of traditional interaction systems' lack of understanding of body language, and is particularly suitable for scenarios such as children's education and rehabilitation training, achieving natural interaction where "action is command."

The invention relates to the technical field of man-machine interaction, and discloses an anaphora resolution method based on a spatial position relation, computer equipment and a storage medium, and the method comprises the steps: extracting spatial position relation features and visual description features in query information, and carrying out semantic search in combination with a preset spatial position relation database, the problems that an existing vehicle-mounted voice assistant depends on accurate vocabularies, lacks space understanding capacity and is weak in anaphora resolution capacity during man-machine interaction are effectively solved, accurate recognition and understanding of natural language description of a user are achieved, the adaptability, accuracy and interaction efficiency of the vehicle-mounted voice assistant in a complex query scene are remarkably improved, and the user experience is improved. And more intelligent, convenient and natural interaction experience is provided for the user, so that the practicability and user satisfaction of the vehicle-mounted voice assistant are greatly improved, and the vehicle-mounted voice assistant has remarkable practical value and wide application prospect.

The invention discloses an interaction method and device, a storage medium, equipment and a program product. The method comprises the steps that it is determined that a hand is switched from a first hand shape to a second hand shape; and in response to release of the second hand shape at the target pose, triggering a target function corresponding to the target pose. According to the invention, the switching of the hand of the user from one specific hand type (the first hand type) to the other hand type (the second hand type) is identified, and the second hand type is released under the specific target pose; according to the method, a natural interaction mode that different functions are triggered by switching different hand types and poses with one hand of a user is achieved, the interaction process is simplified, the user does not need to memorize complex instructions or operation sequences, function triggering can be achieved only through changes of the hand poses, and the user experience is improved. The memory burden of the user is reduced, and the interaction efficiency is improved.

The invention provides a robot teleoperation natural interaction method and device based on XR, and relates to the technical field of XR. natural action data of an operator are captured in real time and subjected to attitude analysis through an XR interaction interface, and action characteristic parameters are obtained; building a robot digital twin model and a virtual operation environment based on a virtual reality engine, mapping the action characteristic parameters into control instructions of the robot, transmitting the control instructions to the physical robot, and driving the physical robot to execute corresponding actions; state data of the physical robot are transmitted back to the XR interaction interface in real time and presented to an operator through multi-mode feedback; and performing virtual rehearsal and collision detection on the execution track of the control instruction to generate a risk prompt. Therefore, through unified registration and information fusion of a three-dimensional space, direct manipulation of natural mapping, low-delay communication, multi-modal feedback and a predictive safety mechanism, efficient cooperation of an operator and a robot is achieved, and operation precision, real-time performance and safety are improved.

The invention relates to the technical field of data processing, in particular to a multi-modal perception and bionic action coordinated pet interaction device. Comprising the steps that user touch behavior data are collected through a sensor, track features and pressure features are extracted, and a preliminary behavior feature set is generated; by fusing multi-dimensional data, the system determines the intention category of user touch, analyzes user interaction style preference, and obtains personalized response adjustment parameters; according to the adjustment parameters, the equipment dynamically updates a reaction mode and generates optimized interaction response logic; a natural interaction sequence is formed by analyzing the matching degree and updating intention prediction data, an equipment output signal is generated, and consistency is verified; if the consistency is confirmed, the dynamic adjustment mechanism is fed back to the parameter updating process, and final bionic action output is generated. The problems that existing pet equipment is inaccurate in response and unnatural in interaction are solved, and higher-precision and personalized bionic pet interaction is achieved.