The invention relates to the technical field of
industrial internet,
artificial intelligence and human-computer interaction, in particular to an
industrial internet software intelligent customer service
interaction method based on multi-
modal deep fusion. According to the method, an intelligent customer
service system integrating
natural language processing, a voice technology,
computer vision and
augmented reality (AR) is constructed aiming at the characteristics of complex operation, multiple user groups, high problem speciality and the like in an industrial scene. The core of the method is that operation questions, interface states and potential faults of engineers in
software use are accurately understood through multi-
modal input; an accurate solution is generated in combination with the
knowledge graph of the
industrial software and the real-
time data of the equipment; a 3D virtual expert image is driven, and immersive and scene-based guidance is provided by fusing voice explanation, interface marking guidance, AR superposition demonstration and operation demonstration videos. The method has the working condition self-adaptive capability, can identify the professional level (such as a green hand / expert mode) of a user, and adjusts the explanation depth; and cross-region
collaboration is supported, a plurality of users are allowed to synchronously share a guidance picture of a virtual expert, and efficient remote
collaboration troubleshooting is realized. The method solves the technical problems that traditional
industrial software is slow in customer service response, abstract in guidance, difficult to process
complex problems on site and the like, and operation and maintenance efficiency and user experience are remarkably improved.