1 results about "Free interface" patented technology

The application discloses an API-free interface automatic control implementation method and system based on visual understanding, relates to the technical field of interface automatic control, and comprises the following steps: performing gradient partition and multi-scale adaptive filtering processing on an original RGB interface image, performing interface element identification and semantic processing on an obtained enhanced interface image, and obtaining feature anchor points of each interface element and an interface semantic layout tree of the enhanced interface image; performing path planning processing and abnormality prediction on a user operation track based on the interface semantic layout tree and the feature anchor points; performing simulated input operation based on an optimal execution path, and performing dynamic self-recovery according to an operation execution state and in combination with an abnormality processing scheme; checking an operation execution result of the simulated input operation, and updating a local evolution knowledge base according to a checking result. The application is used for solving the problems of low interface automatic control precision, poor robustness and no self-recovery of abnormality in the prior art without relying on API.