This application discloses a target element localization method and apparatus based on interface structure
perception, relating to the fields of
artificial intelligence and human-computer interaction. By fusing multiple
inference results, it achieves collaborative optimization based on explicit structural
inference and multi-source information, which not only significantly improves the localization accuracy under complex interface layouts and complex
natural language commands, but also enhances the
interpretability and generalization ability of the method, providing more accurate target element localization capabilities for
graphical user interface agents. The method includes: in response to a target element localization request, acquiring interface input data and
natural language commands; constructing an interface relationship
graph based on the interface input data; performing multi-
type inference on element objects in the interface relationship graph to obtain multi-
type inference results for the element objects; selecting at least two
inference results from the multi-
type inference results for fusion, and determining the localization result of the target element based on the fusion result.