134 results about "Embodied intelligence" patented technology

The invention provides a full-autonomous cell operation system based on body intelligence, and relates to the field of cell operation, the system comprises a system hardware architecture and a system software architecture; the system hardware architecture comprises a micromanipulation execution system, a precision positioning system, a multi-mode sensing system and an intelligent calculation and control center. The micromanipulation execution system is used for cell environment interaction, the precision positioning system is used for sample bearing and environment control, the multi-modal sensing system is used for providing key information, and the intelligent calculation and control center is used for operation decision making; the system software architecture comprises a sensing layer, a planning layer, an execution layer and an interaction layer. According to the application, cell recognition, tool selection, path planning, operation execution and result feedback can be automatically completed without manual intervention; through tool autonomous switching and strategy dynamic adjustment, various operations such as cell clamping, injection, controllable deformation and micro-assembly can be completed, and the problems that traditional cell operation depends on manual work, efficiency is low, and consistency is poor are solved.

The invention discloses a target-level three-dimensional point cloud cross-modal semantic retrieval method and device and electronic equipment. The target-level three-dimensional point cloud cross-modal semantic retrieval method comprises the steps that independent three-dimensional target point clouds are segmented from three-dimensional scene point clouds, and unique identifiers are given to the independent three-dimensional target point clouds; projecting each target point cloud to three orthogonal two-dimensional observation planes to generate a multi-view composite image; a pre-trained vision-language basic model is adopted to code and fuse the synthesized image, and a unified multi-modal feature vector is generated; a vector database in which the feature vectors are associated with their identifiers is constructed. And encoding a natural language query text into a text query vector by using the model, calculating the semantic similarity between the text query vector and the feature vector in the database, and positioning and returning the corresponding three-dimensional target point cloud. According to the method, accurate and efficient cross-modal retrieval from a natural language to a three-dimensional point cloud target is realized, the problem that a traditional method is difficult to support semantic fine-grained retrieval of the three-dimensional target point cloud is solved, and a key technical support is provided for training data management in the fields of intelligence and the like.

The invention discloses an intelligent experimental training system with a body based on a multi-modal large model, and the system comprises the following modules: a multi-modal sensing module collects visual, voice, motion and force sense data, and generates a state vector through synchronization and fusion; the semantic evaluation module is used for performing semantic matching and identifying knowledge blanks by using a multi-modal large model; the causal generation module generates a causal query after blank detection and forms a hypothetical experiment proposition; the tool body execution module executes an intervention action and collects multi-mode feedback; the effect verification module calculates causal effects before and after intervention and judges significance; the graph updating module generates new knowledge nodes and updates the causal relationship graph when verification is established; and the teaching management module performs causal chain visualization and result feedback according to the updated atlas. According to the method, a self intervention feedback mechanism is introduced into a multi-modal perception, semantic understanding and decision execution closed loop, so that the knowledge evolution and teaching intelligence level is remarkably improved.

The invention relates to the technical field of ownership intelligence, in particular to a ownership intelligence fuzzy language instruction recognition method based on scene risk guidance. In order to solve the problem that a fuzzy language instruction recognition method in the prior art easily causes understanding deviation and potential safety hazards, the invention provides an intelligent fuzzy language instruction recognition method based on scene risk guidance, which comprises the following steps of: 1) constructing a scene risk map according to a basic risk and a distance risk; 2) identifying an instruction type; 3) instruction analysis and risk assessment; and 4) instruction rewriting and object identification replacement. According to the method, a risk perception mechanism is introduced, the high-risk behavior execution probability caused by semantic deviation of the intelligent robot is effectively reduced, the understanding accuracy and interaction safety of the intelligent robot in a real scene are improved, and the method is suitable for a complex application environment with high requirements for task reliability and behavior controllability of the intelligent robot.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as personal intelligence, financial science and technology, medical health and the like, and discloses an action instruction sequence generation method, device, equipment and medium. Generating an attention weight based on the visual feature vector and the tactile feature vector, fusing the visual feature vector and the tactile feature vector to generate a fusion feature vector, and constructing a reinforcement learning model state space containing environment state information and the fusion feature vector; a sequence of action instructions is generated by a reward function in response to a task target approach event or an obstacle collision event based on the state space. According to the method, through combination of multi-modal information fusion and reinforcement learning state space construction, the perception ability and action decision-making ability of the intelligent agent in a complex environment are improved, and autonomy, flexibility and stability of task execution are remarkably enhanced.

The invention discloses a task processing method and an intelligent device, and relates to the technical field of intelligent devices, and the method comprises the steps: obtaining multi-modal data which comprises task information of a to-be-processed task in a real-time interaction scene of a physical entity and an environment; performing feature extraction and feature alignment processing on different modal data in the multi-modal data to obtain a first feature after feature alignment; performing cross-modal data retrieval on a knowledge base based on the first feature to obtain target retrieval data; performing feature fusion on different modal data features in the first features to obtain second features; and making a decision based on the target retrieval data and the second feature by using a decision model to obtain an action instruction sequence, and executing the to-be-processed task based on the action instruction sequence.

The invention discloses a dynamic obstacle avoidance and path optimization method based on a body intelligence robot, and relates to the technical field of robot path planning, and the method comprises the following steps: collecting map data and task constraint data; performing path planning analysis on the map data and the task constraint data based on a set path optimization algorithm to generate initial planning path data; according to the intelligent robot dynamic obstacle avoidance and path optimization method, a robot actual driving path prediction model is constructed, and a target function is constructed for the deviation between the robot actual driving path prediction model and the real-time path planning data of the next time period to carry out minimization analysis; and the real-time path planning data of the next time period is optimized, so that the robot runs according to the optimized real-time path planning data to obtain an actual running path which fits the original real-time path planning data, and the running error of the robot is reduced.

The invention provides a teaching data conversion method and device oriented to intelligence and electronic equipment, and relates to the technical field of intelligence. The teaching data conversion method for the intelligent teaching system comprises the steps of obtaining to-be-converted teaching data in a first data format; reading metadata information of the to-be-converted teaching data declared in the configuration file; calling a target interface method aiming at the first data format according to the metadata information by utilizing a conversion engine, and obtaining feature data of the body agent during execution of the target task from the to-be-converted teaching data; wherein the conversion engine is a class which is defined by a parent class and is used for converting teaching data in a non-second format into teaching data in a second format; and obtaining the to-be-converted teaching data in the second format according to the feature data by using an aggregation method provided by the conversion engine. According to the method, standardized conversion of the intelligent teaching data with different sources can be realized efficiently at low cost.

The invention discloses a pipeline detection method and system based on an intelligent pipeline robot with a body, and relates to the technical field of pipeline detection. The method comprises the following steps: acquiring pose, environment and pipe wall sensing data acquired by a multi-mode sensor; a pipeline three-dimensional digital map is constructed, and real-time self-positioning is achieved; identifying a pipe wall defect based on the pipe wall sensing data, generating a defect data packet, and binding the defect data packet with a map space position for storage; when an adjacent robot is detected, constructing a temporary communication link; exchanging detection information containing a map abstract and task state information with an adjacent robot through the link; updating the three-dimensional digital map by using the map abstract of the adjacent robot; performing task negotiation by combining the task state information and the updated map; and adjusting the target operation area and the motion path according to the negotiation result. According to the invention, full-automatic high-precision detection and intelligent operation and maintenance decision-making in a complex pipeline environment can be realized.

The invention provides a robot evaluation method, a storage medium, electronic equipment and a program product, and relates to the technical field of intelligent equipment. The robot evaluation method comprises the steps of determining a weight corresponding to the degree of freedom of motion of a robot based on a to-be-executed task of the robot; based on the upper limit of the movement speed of the target joint, the weight corresponding to the target joint is determined, and the target joint comprises joints related to the to-be-executed task; based on the weight corresponding to the motion freedom degree of the robot and the weight corresponding to the target joint, operability evaluation information of the robot under the current configuration is calculated, and the operability evaluation information represents the ability of the robot to complete the to-be-executed task with the current configuration.

The invention relates to the technical field of intelligent equipment. A trajectory diffusion imitation learning method based on single human video demonstration comprises the steps that a demonstration video is analyzed to obtain an operation body posture motion trajectory and an object pose motion trajectory, and the demonstration video is a video of a human or machine operation object; respectively carrying out interpolation calculation on the positions and postures of the operation body posture motion trail and the object pose motion trail to obtain a training data set; using the training data set to train a diffusion strategy model to obtain a target diffusion strategy model; and during deployment, the real-time pose of an end effector of the robot serves as observation input of the target diffusion type strategy model, an operation body pose movement track and an object pose movement track in the deployment stage are formed, and a joint control instruction of a target operation body is obtained. And the task success rate is kept, and the practicability and the expandability of the imitation learning method are improved at the same time.

The invention relates to the field of artificial intelligence and self intelligence, and discloses a cross-space, cross-task and cross-ontology learning large model and reasoning method, the method comprises the following steps: a pre-training multi-modal large language model as a backbone network is used for processing visual and text input and generating a response; the intention bridging interface is used for compressing a hidden state output by the multi-mode large language model into semantic intention representation with a fixed length; the action strategy head is used for generating a continuous action sequence based on the semantic intention; the state encoder is used for encoding the body sensing state of the robot; and the motion encoder / decoder is used for embedding and reconstructing the motion. According to the method, cross-space migration, cross-task learning and cross-ontology generalization are realized in a single model, and the performance and generalization ability of digital space reasoning and physical space control are improved.

The invention discloses a magnetic attraction intelligent connection type autonomous operation and maintenance experiment node system, and the system comprises a sensing and modeling layer which is used for constructing a thinking digital twinning environment which is synchronous with a physics laboratory in real time, is centimeter-level in precision, can be interacted with the physics laboratory, and is rich in semantic information; the cognition and decision-making layer is used for upgrading simple and rule-based reactive behaviors into complex, logic-based and prospective active decisions; the collaboration and network layer is used for connecting all the distributed cognitive experiment nodes into an organic super organism capable of emerging'swarm intelligence '; and the interaction and operation and maintenance layer is used for seamlessly and immersively transmitting remote experts to a physical site by introducing generative artificial intelligence and own intelligence, so that the remote experts become a high-level commander and a lifelong learning partner of the whole cognitive experiment node system. By means of the magnetic attraction intelligent linkage type autonomous operation and maintenance experiment node system, the problems that current experiment equipment is lack of cognition on the physical environment, blank in complex logical reasoning, stagnant in adaptive capacity to dynamic tasks and the like can be solved, and the intelligent and systematic operation capacity of the experiment equipment is remarkably improved.

The invention provides a training method and device of a body intelligent model based on a multi-modal autoregression model, and relates to the technical field of intelligent robots, and the method comprises the steps: carrying out the feature extraction processing and feature fusion processing of an image feature set collected by a robot, and obtaining the overall visual feature information and a visual information feature set; performing text visual feature fusion processing on the visual information feature set and the instruction text to obtain visual text fusion features, and determining angle information features based on the overall visual feature information and the instruction text; and sending the visual text fusion feature and the angle information feature to a preset large language model to predict the action of the robot, training a body-equipped intelligent model of the robot according to a prediction result, and determining a target body-equipped intelligent model. According to the method, the object angle and environment sensing capability of the intelligent model can be remarkably improved, and the accuracy of model action prediction is further improved.

The invention discloses a body world model of a Sim2Real-Agent efficient video generation technology, and particularly relates to the technical field of video processing, and the body world model comprises a video generation physical modeling module which is used for ensuring that a generated video is physically consistent with reality and can keep stable physical behaviors in a short time sequence and a long time sequence. The data set fine-tuning management module optimizes the adaptability of the model in a specific scene through staged pre-training and fine-tuning strategies, greatly reduces the dependence on high-quality annotation data, improves the learning ability of the model, and improves the learning efficiency of the model through efficient hardware resource scheduling and reasoning optimization by the resource management reasoning optimization module. Compared with the prior art, the simulation training method has the advantages that the simulation training cost is greatly reduced by the Sim2Real-Agent, the calculation resource consumption in the training and reasoning process is reduced, the real-time response capability of the system in the actual deployment is improved, the simulation training cost is greatly reduced by the Sim2Real-Agent through the technical innovation, the physical consistency, the time sequence coherence and the multi-modal control precision are improved, and the application of the intelligent robot in the fields of robots, automatic driving and the like is promoted.

The invention relates to the technical field of body intelligence, in particular to a body intelligence attack detection method based on a constraint function, mainly solves the technical problems of insufficient granularity, poor real-time performance and insufficient robustness of the existing attack detection method, and comprises the following steps: S1, selecting key points; s2, generating a constraint function; s3, analyzing the images where the key points are located by adopting a credible multi-modal model, and generating task fingerprints; s4, comparing the constraint function with the task fingerprints, and calculating an output difference; semantic equivalence measurement is introduced; a tolerance value is introduced, if the output difference exceeds a threshold value, it is judged that the constraint function is attacked, and the system stops task execution. According to the method, the detection accuracy exceeding 92% can be realized in various attack scenes; the problems of false report and missing report caused by insufficient rule coverage or large language model output diversity can be avoided, and the robustness is high; and the execution safety and the real-time performance of the body-equipped intelligent agent in a complex and open environment can be obviously improved.

The application discloses a kind of based on embodiment intelligence's self-adapting rocker arm type carrying robot control method and system, belong to robot control field, the method includes: by being carried on robot car body and rocker arm mechanism on multi-modal sensor array obtains first perception dataset and second state dataset, extracts historical configuration matrix and exports feedforward configuration instruction to realize feedforward pre-adjustment;During obstacle crossing and load period, generate and execute the non-symmetrical compensation instruction of driving two sides rocker arm to carry out asymmetric telescoping and lifting;In distress period, according to the attribute of physical root node, match the corresponding quasi-physical escape sequence from state transition matrix and execute escape action;In edge control node, running reinforcement learning network module to output original speed instruction vector.The application guarantees the control completeness when facing new terrain, improves the accuracy of compensation action.

The present invention relates to an embodied intelligence agent based on physical intelligence and active perception, and a control method therefor. The method controls the embodied intelligence agent on the basis of a multimodal embodiment large model, and the multimodal embodiment large model comprises a data processing module, a perception and planning module, a physical intelligence module, an information transmission module, an environment interaction module, and a motion control execution module. The method comprises: the perception and planning module receives data processed by the data processing module and, according to the data, outputs perception information and a task planning instruction; the physical intelligence module receives the perception information and the task planning instruction, and outputs a motion action planning instruction; the environment interaction module collects environment information in real time and compares same with the perception information; and the motion control module receives the motion action planning instruction and incorporates the comparison result to perform action correction so as to implement control of the embodied intelligence agent. Compared with the prior art, the present invention has the advantages of high generalisation performance and high task execution accuracy.

This application relates to the field of information technology and discloses a method, device, medium, and product for acquiring embodied intelligence data. The method includes: collecting artificial demonstration data through an exoskeleton system, the artificial demonstration data including video stream data, motion data, and force data; importing the video stream data into a first large language-visual model, segmenting it into multiple independent task segments, and generating descriptions for the independent task segments; importing the descriptions of the independent task segments into the large language model to generate task annotations based on natural language descriptions; and exporting the independent task segments, the task annotations, and the corresponding motion data and force data to generate embodied intelligence data. This establishes a low-cost, high-efficiency, and scalable method for acquiring and processing embodied intelligence data, achieving automated conversion from human operation data to high-quality VLA training data.

The present application relates to the field of electric digital data processing, and in particular to a method for sensing and decision-making of a door lock interaction system based on embodied intelligence, which comprises collecting sensing data, labeling an intention label and an environmental noise label corresponding to each sensing data to obtain labeled sensing data; constructing a sensing and decision-making model and training the sensing and decision-making model to obtain a trained sensing and decision-making model; collecting new sensing data, inputting the voiceprint data, the relative distance data between the person and the door lock, and the door lock state data in the new sensing data into the trained sensing and decision-making model after aligning them to a unified reference according to a time stamp to obtain an intention probability distribution, selecting the intention with the highest probability as a decision result and triggering a corresponding door lock control instruction. The existing sensing and decision-making method of the door lock interaction system based on embodied intelligence has the problem of poor user experience. The method provided by the present application can provide good user experience.

The invention relates to the technical field of body sensing, in particular to a body intelligent super-flexible wire harness sensing system, which comprises a fiber response acquisition module, a state monitoring and correction module, a feature fusion module, a topological field reconstruction module and a control instruction generation module. According to the method, accurate extraction of the fiber response points is achieved through comparison of differential operation and the reference displacement, the bending position and angle change can be recognized in time and corrected in the threshold exceeding state, noise in signal transmission keeps stable after being weakened through a numerical method, and the stability of the fiber response points is improved. Further performing weighted integration on the deformation characteristics in multi-scale fusion to enable touch and deformation information to form continuous mapping under the same perception distribution, and finally positioning an action center through a probability model and outputting a topological mapping image, so that accurate capture and spatial position identification of an external force peak value can be realized in a complex environment, and the accuracy of the external force peak value is improved. And the sensitivity of the flexible wire harness to fine mechanical changes and the integrity of global perception are improved.

The application relates to the field of natural language processing and decision intelligence, and particularly relates to a large language model reasoning method based on time series difference learning and rule enhancement, which is widely applied to automatic planning, intelligent question answering, embodied intelligence and the like. The method comprises task trajectory sampling, domain knowledge induction, domain rule extraction and large language model reasoning based on rule enhancement. For test task data, the most relevant historical task is matched based on vector retrieval, and a domain rule set corresponding to the task is obtained. The rule is rewritten in natural language by the large language model itself, so that the rule is more interpretable and adaptable. Finally, the optimized rule set is integrated into the large language model reasoning prompt text, so as to optimize the reasoning quality and stability.

The invention relates to the technical field of robot sensing and intelligent, in particular to a robot self-adaptive sensing method and system. The method comprises the following steps: in response to a task instruction, analyzing to obtain a perception modal demand corresponding to task semantics and an execution stage; dynamically adjusting configuration strategies of various modal sensors in a task execution process according to the sensing modal requirements; monitoring the confidence of each sensor data stream in real time; and when it is monitored that the confidence coefficient of the main sensor data flow is lower than a first threshold value, dynamically adjusting fusion priorities of different sensor data in the perception decision. According to the method, fine scheduling of sensing resources is realized, the environment robustness of a robot sensing system is greatly improved, and meanwhile, the decision accuracy and generalization ability are effectively improved.

The application provides a body-embodied intelligent data processing method and device, relates to the field of body-embodied intelligence, and the method comprises the following steps: acquiring a task reference video of a body-embodied intelligent agent; performing multi-modal feature extraction on the task reference video to obtain visual semantic features of the task reference video and body posture features of a task execution object; fusing the visual semantic features and the body posture features to obtain fused features; performing task recognition on the task reference video and behavior intention recognition on the task execution object according to the fused features to obtain recognition results; and determining effective video segments of the task reference video according to the recognition results, wherein the effective video segments are used for verifying and / or referencing the operation of the body-embodied intelligent agent in a task execution process. The application realizes automatic processing of body-embodied intelligent data through multi-modal technology, improves data processing efficiency and accuracy, and saves labor costs.

The application relates to the field of embodied intelligence and autonomous navigation, in particular to a wheelchair navigation method and device, equipment and medium, wherein the method comprises the following steps: acquiring a current environment image, a navigation instruction and a historical environment image of a wheelchair; extracting an environment structure feature of the current environment image, and determining whether a target environment is an indoor environment or an outdoor environment based on the environment structure feature; when the target environment is the indoor environment, generating a bird's-eye view feature map according to the current environment image, and combining the navigation instruction to navigate; when the target environment is the outdoor environment, predicting a three-dimensional point cloud, a relative pose and a driving trajectory based on the current environment image and the historical environment image, and combining the navigation instruction to navigate. Therefore, the problems of poor unified representation capability of perception, mapping and decision, insufficient spatial geometric expression and real-time performance, and lack of continuous environment updating and memory capability in the related art are solved.

The application provides a three-dimensional object token-based closed-loop task execution method, a robot and a medium, and relates to the field of embodied intelligence. The method comprises determining a task to be executed and a task sub-target based on an input instruction; constructing a three-dimensional object token related to the task sub-target based on environment observation information of an environment in which the robot is located; inputting the task sub-target, the three-dimensional object token and robot state information into a visual language action model to generate an action instruction for completing the task sub-target; driving the robot to perform a corresponding action based on the action instruction; updating the three-dimensional object token based on environment observation information after the action is performed; verifying whether the task sub-target is completed based on the updated three-dimensional object token; and in the case where it is verified that the task sub-target is not completed, triggering re-perception, resuming execution, action regeneration or task re-planning, which is conducive to ensuring the consistency of object identity, spatial position and physical relationship of the embodied intelligent agent in a long-term mobile operation task.

The invention provides an industrial-manufacturing-oriented method for constructing a deviation-rectifying body execution large model, and belongs to the technical field of body intelligence and robot control. The invention provides a multi-modal physical consistency monitoring-causal anti-fact attribution-lightweight online correction-continuous self-supervised evolution architecture. The method comprises the following steps: constructing a physical consistency monitoring mechanism based on a submerged space diffusion world model, and capturing a tiny execution deviation by using a maximum mean value difference; a traceability-intervention-prediction structured causal reasoning framework is introduced to accurately position a deviation root, and a low-rank adaptation module is adopted to generate real-time residual compensation; and accumulation and cross-task migration of correction experience are realized through hierarchical continuous learning and a self-prompting generalization mechanism. According to the method, the capability from passive response to active correction is improved, and the method has the advantages of high-fidelity physical perception, millisecond-level real-time compensation, catastrophic forgetting resistance, zero sample generalization and the like, and can be widely applied to industrial scenes such as precision assembly and flexible material operation.

The application belongs to the technical field of embodied intelligence, and discloses a robot action generation method based on continuous state decomposition and related devices; wherein the robot action generation method based on continuous state decomposition comprises the following steps: acquiring a language instruction of a user and a scene observation of a robot; analyzing the language instruction through a large language model to obtain a key target state value; generating an action through an actor model based on the language instruction, the scene observation and the key target state value, to obtain a robot action. 2 The application can solve the problem of difficult mapping between language instructions and target actions in current robot operation, can effectively process continuous states, can generate accurate robot operation actions in new states, and improves the generalization ability and operation precision of robots in diversified tasks.

The application provides an entropy-aware robot control method and system based on expert demonstration, and relates to the field of embodied intelligence technology, which comprises obtaining a task instruction input by a user, observation data and ontology perception data of a robot at a current time; cleaning and feature extracting the data to obtain multiple features, and retrieving a reference video in an expert reference video database; encoding and splicing the multiple features to generate a multi-modal feature representation; feature extracting the reference video to obtain reference video features; processing the multi-modal feature representation and the reference video features through a skill generation model to obtain a discrete skill codebook index probability; calculating an information entropy of the probability, and determining a sampling candidate number of a current candidate skill based on the information entropy; generating a skill token sequence based on the discrete skill codebook index probability and the sampling candidate number; and finally decoding the skill token sequence to obtain a robot action sequence. The control method provided by the application can guarantee accurate control of a robot in a complex environment.

The invention discloses an intelligent VLA model language instruction noise filtering and robustness improving method, and aims to solve the problem that performance is reduced due to the fact that a VLA model is sensitive to natural language instructions containing irrelevant contexts. The method comprises the following steps: an instruction analysis module based on a large language model is used for receiving a natural language instruction containing noise; the noise detection and filtering module identifies and removes irrelevant contexts through semantic analysis; and the instruction reconstruction module is used for converting the filtered core instruction into a standardized instruction which can be executed by the VLA model. By constructing a processing flow of'user instruction-noise filtering-standardized instruction ', a noise-containing instruction is converted into a simple and clear executable instruction, and the robustness of a VLA model to language change is improved. The method is suitable for instruction preprocessing of the VLA model in an intelligent system, and the task execution success rate of the model in a real man-machine interaction scene can be remarkably improved.