Mobile agent information processing system based on prior rules and multimodal large model

By combining prior rules and multimodal large models with a mobile intelligent agent information processing system, the problem of low recognition and operation efficiency of multimodal large models in graphical user interfaces is solved, precise positioning and function recognition of icons are achieved, task paths are optimized, customized operations are adapted to specific applications, and response speed and execution efficiency are improved in high-frequency interaction scenarios.

CN120161973BActive Publication Date: 2025-09-23HEFEI UNIV OF TECH
2 Cites 0 Cited by

Patent Information

Application Number
CN202510266979.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-03-06
Publication Date
2025-09-23
Estimated Expiration
2045-03-06

AI Technical Summary

Technical Problem

When processing graphical user interfaces, existing large multimodal models are unable to accurately identify and generate the coordinates and functional descriptions of interactive elements, resulting in increased misoperations and resource consumption. They are unable to adapt to complex pages and multi-tasking scenarios, and lack deep optimization for specific applications, resulting in low execution efficiency.

Method used

A mobile intelligent agent information processing system based on prior rules and multimodal large models is adopted. Through the icon positioning and recognition module, execution efficiency improvement module, specific application operation optimization module and navigation task data collection and update module, combined with deep learning and interface data analysis, it can achieve precise icon positioning and function recognition, optimize task paths, and self-learn to adapt to interface changes.

Benefits of technology

It achieves accurate positioning and function identification of icons, reduces computing resource consumption, improves the stability and efficiency of task execution, adapts to customized operations for specific applications, and ensures fast response and accuracy in high-frequency interaction scenarios.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The present invention discloses a mobile terminal intelligent body information processing system based on prior rules and multimodal large models, which relates to the field of artificial intelligence technology, including an icon accurate positioning and recognition module, which accurately positions and recognizes icons in the interface through different levels of recognition methods, and infers the functions of the icons; an execution efficiency improvement module, which improves the execution speed by task disassembly and optimization path design; an operation optimization module for specific applications, which is used to record the position information, size and corresponding icon function of each interface element, build a navigation path library for specific applications, and optimize the execution path of periodic tasks; a navigation and task data collection and update module, which is responsible for collecting and updating the dynamic data of navigation paths and task execution, and continuously performing self-learning and optimization. Therefore, the above method can achieve accurate icon positioning and function recognition, improve execution efficiency and reduce computing resource consumption, and at the same time be able to cope with interface changes and continuously optimize.
Need to check novelty before this filing date? Find Prior Art