Automated Geospatial Web Service Discovery and Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for searching and discovering Web Mapping Services (WMS) on the internet are largely manual and inefficient, relying on internet search engines, which makes it difficult to find and utilize available WMS servers that adhere to Open Geospatial Consortium standards.
Innovation Solution
A computer-based method and system that uses search engines and web crawling to identify and validate HTTP URLs that comply with a predetermined XML schema, such as the Open Geospatial Consortium's WMS Capabilities document, and integrates these services into a geospatial portal, providing a single point of access to geospatial information systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual searching methods are used to find WMS servers, then users can locate some WMS services, but the process is time-consuming and inefficient
Solution Approach 1:
The system enables automated self-discovery of WMS servers through web crawlers that autonomously search the internet for WMS Capabilities documents without requiring manual user intervention. The crawler automatically identifies, retrieves, and validates WMS service information from various web sources.
Solution Approach 2:
The manual mechanical process of searching for WMS servers is replaced with an automated electronic web crawler system that uses search engine APIs and automated browsing to discover and validate WMS services, significantly improving efficiency and reducing time loss.
2Reliability
If web crawling is performed on all identified URLs to validate WMS services, then comprehensive WMS discovery is achieved, but system complexity and processing load increase
Solution Approach 1:
The validation process is segmented into distinct modular components: URL identification, Capabilities document retrieval, XML schema validation, and service registration. Each component handles a specific aspect of the validation process, making the overall system more manageable and maintainable while ensuring comprehensive validation.
Solution Approach 2:
The system uses an intermediary validation layer that checks WMS Capabilities documents against the OGC XML schema before registering services. This intermediary validation mechanism ensures reliability by filtering out non-compliant services without requiring complex custom validation logic throughout the entire system.
3Productivity
If automated web crawling and validation is implemented, then WMS discovery efficiency improves, but energy consumption and computational resources increase
Solution Approach 1:
The web crawler operates periodically rather than continuously, scheduling discovery tasks at intervals and using caching mechanisms to store previously validated WMS service information. This periodic operation reduces energy consumption by avoiding redundant crawling and validation of the same URLs while maintaining high discovery productivity.
Solution Approach 2:
The system performs preliminary validation checks and uses search engine APIs to filter potential WMS URLs before initiating full web crawling. This preliminary action reduces the number of URLs that require intensive crawling and validation, thereby reducing overall energy consumption while maintaining high discovery efficiency.
Data Source
AI summary
A method and computer system for identifying internet web pages containing documents that comply with a predetermined XML schema. The method includes searching the internet with a search engine for web pages using initial search terms and identifying a first set of HTTP URLs, web crawling at least the first set of HTTP URLs to identify additional HTTP URLs, appending a query to the identified URLs, and evaluating the responses to the query to determine which responses comply with the predetermined XML schema. The XML schema can be a Web Mapping Services schema. The system can store responses that comply with the XML schema in a database of servers, periodically check the database for validity, and convert the map requests for map servers in the database to a GIDB Portal Interface API.