Deep Application Crawling for Mobile App Data Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies lack the ability to extract and utilize content from mobile and desktop applications for search engines, advertisement targeting, and recommendation services, as conventional Web crawlers only index web content and not application data.
Innovation Solution
The deep application crawling technique extracts both static and dynamic data from applications, using binary code analysis and navigation scripts to capture and store data for use in search engines, advertisement engines, and recommendation engines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional Web crawlers are used to index content, then web content can be indexed and searched, but application data inside mobile and desktop applications cannot be extracted or utilized
Solution Approach 1:
The patent introduces an intermediary application crawling system that bridges the gap between conventional web crawlers and application data. This intermediary system includes components such as the application binary analyzer, navigation script generator, and data extractor that specifically target application packages (APK files) and their contents, enabling search engines to access and index application data without requiring changes to the applications themselves or the core web crawler architecture
Solution Approach 2:
The patent segments the application data extraction process into distinct components: static data extraction from application binaries, dynamic data extraction through simulated user interactions, and structured organization of extracted content. This segmentation allows each component to be optimized independently and enables comprehensive coverage of different data types within applications
2Productivity
If application data is extracted and indexed, then search results and advertisement targeting improve, but the complexity of data extraction and processing increases
Solution Approach 1:
The patent applies preliminary action by pre-analyzing application binaries to generate navigation scripts that automatically guide the crawling process through application interfaces. These scripts are generated before actual data extraction occurs, enabling systematic navigation through application screens and interactions without requiring complex real-time decision-making during the crawling process
Solution Approach 2:
The patent uses copying by creating virtual representations of user interactions through navigation scripts. Instead of requiring physical manual testing of each application, the system generates and executes scripted sequences that replicate user behavior, thereby extracting dynamic content efficiently while reducing the need for complex automated testing infrastructure
Data Source
AI summary
The deep application crawling technique described herein crawls one or more applications, commonly referred to as “apps”, in order to extract information inside of them. This can involve crawling and extracting static data that are embedded within apps or resource files that are associated with the apps. The technique can also crawl and extract dynamic data that apps download from the Internet or display to the user on demand, in order to extract data. This extracted static and/or data can then be used by another application or an engine to perform various functions. For example, the technique can use the extracted data to provide search results in response to a user query entered into a search engine. Alternately, the extracted static and/or dynamic data can be used by an advertisement engine to select application-specific advertisements. Or the data can be used by a recommendation engine to make recommendations for goods/services.