The invention relates to the technical field of
artificial intelligence, can be applied to business
system platforms of smart home,
medical health and the like, and discloses an
artificial intelligence-based fault identification and
processing method, which comprises the following steps of: obtaining multi-
modal information of task execution of an intelligent
robot through image and audio information, constructing a task
scene graph based on the image and audio information and generating an audio event tag, identifying an image
key frame and creating key scene text description, analyzing the key scene text description and the audio event tag by using a large
language model, identifying a subtask completion condition and detecting a fault, and if no fault exists, executing the next step; if yes, fault detection is carried out on a
robot task execution result, if a fault is found, error recognition is carried out in combination with task description and a task plan, and according to a
fault analysis or error recognition result, a
fault correction task is generated to be executed by the intelligent
robot. According to the method, the state information of the intelligent robot is converted into the text description which is easy to understand, so that the fault reason is positioned and corrected.