Industrial multi-modal intelligent interaction method and system based on large language model agent

By aligning and fusing multimodal data to generate physical state vectors, calculating anchor confidence in real time, and weighting the hybrid lexical probability distribution, the semantic drift problem between language reasoning paths and physical states in industrial settings by large language model agents is solved, achieving safe autonomous degradation and avoiding safety incidents.

CN122196146BActive Publication Date: 2026-07-24TIANJIN HONGHUANG TECH CO LTD
2 Cites 0 Cited by

Patent Information

Application Number
CN202610672461.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-05-15
Publication Date
2026-07-24
Estimated Expiration
2046-05-15

AI Technical Summary

Technical Problem

In existing technologies, large language model agents experience continuous semantic drift between the language reasoning path and the physical state of the industrial site during word-by-word generation. Post-verification cannot intervene in the completed reasoning process, leading to frequent safety incidents. Furthermore, when physical constraint signals fail, safety degradation cannot be completed without interrupting the operation of the industrial site.

Method used

By aligning the pose data stream of autonomous mobile robots, visual image frames, cargo load readings, and production line alarm status codes with timestamps, a multimodal acquisition dataset is constructed. The features of each modality are then projected onto the hidden layer dimension space of a large language model and weighted and fused to generate a physical state vector. The natural language scheduling instructions of the operators are jointly encoded with the physical state vector. The confidence of stepwise anchor points is calculated in real time and the probability distribution of mixed lexical terms is weighted to construct an uncertain semantic vector to generate safety degradation instructions.

Benefits of technology

It achieves real-time constraints on the physical state of the industrial site during the generation of large language model agents, ensuring that the language reasoning process is consistent with the physical state. It can perform autonomous safety degradation without interrupting the operation of the industrial site, thus avoiding safety accidents.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application relates to the technical field of data processing, and discloses an industrial multi-modal intelligent interaction method and system based on a large language model agent. The method comprises the following steps: fusing industrial field multi-modal collection data sets into a physical state vector, jointly encoding the physical state vector and a natural language scheduling instruction to obtain a multi-modal interaction context, modulating a word element probability distribution in real time in a word element generation process by using a step-by-step anchor point confidence, and injecting an uncertainty semantic vector to drive safe degradation instruction generation when the confidence continuously decreases. The application solves the problems in the prior art that, in the word element generation process of a large language model agent, semantic drift between language reasoning paths and industrial field physical states is continuously accumulated, post-checking cannot intervene in the completed reasoning process, and when a physical constraint signal fails, the system cannot complete safe degradation without interrupting the operation of the industrial field.
Need to check novelty before this filing date? Find Prior Art