Website Crawling Input Value Prediction for Reliable Form Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to set appropriate input values for input items during website crawling, leading to potential errors that disrupt the process.
Innovation Solution
A system utilizing a trained model to set input values by processing webpage information, enabling accurate input value determination based on model output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If previously prepared input values are set for input items during website crawling, then the crawling process can proceed automatically, but unexpected errors occur due to inappropriate input values for different websites
Solution Approach 1:
The system performs preliminary actions by analyzing the destination webpage content before finalizing input values. It acquires the destination webpage, analyzes its content structure, and determines appropriate input values in advance, ensuring they are suitable for the specific website context before submission occurs.
Solution Approach 2:
The system implements feedback by analyzing the content of the destination webpage after form submission and using this information to determine appropriate input values for subsequent crawling steps. The destination page content provides feedback about what input values are appropriate, creating a closed-loop system that adapts to different websites.
2Reliability
If manual follow-up is performed to check and correct input values, then crawling reliability improves, but processing time and complexity increase
Solution Approach 1:
The system performs self-service by automatically analyzing destination webpage content and determining appropriate input values without human intervention. The crawling system itself checks and sets input values based on the content it retrieves, eliminating the need for manual follow-up while maintaining reliability.
Solution Approach 2:
The system replaces manual mechanical operations with automated content analysis. Instead of manual inspection and correction of input values, the system uses automated processing of destination webpage content to intelligently determine appropriate input values, substituting human effort with computational analysis.
3Ease of manufacture
If previously prepared input values are used for all websites, then setup time is reduced, but adaptability to different website requirements decreases
Solution Approach 1:
The system transitions from static pre-prepared input values to dynamic, context-dependent value determination. Input values are no longer fixed in advance but are dynamically determined based on the actual content of each destination webpage, allowing the system to adapt to different website requirements while maintaining ease of operation.
Solution Approach 2:
The system changes the parameters used to determine input values from fixed pre-prepared values to content-based dynamic values. By analyzing the actual content of destination webpages and adjusting input values accordingly, the system adapts to different website characteristics without requiring extensive manual configuration.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A website inspection server 10 according to an embodiment of the present invention assists in setting appropriate input values for input items during the crawling of a website. The server 10 provides, to a user who operates a user terminal 30 communicatively connected to the server 10 through a communication network 20, a website inspection service in which a website 22 specified by the user is inspected. The server 10 identifies input items in a webpage and sets input values for the input items on the basis of information outputted from a trained model during the crawling of the website 22. Thus, use of input values suitable for various websites is promoted compared to when previously prepared input values are used.