3 results about "Deep Web" patented technology

This invention discloses a deep web crawler method and system based on multi-level recursion, belonging to the field of computer and artificial intelligence technology. The method includes data collection and preprocessing, deep learning model selection and training, real-time monitoring and detection, system integration and performance optimization. Data collection and preprocessing involves collecting data and cleaning and denoising it, as well as feature extraction and transformation. Deep learning model selection and training involves choosing different models or hybrid models based on the data type. Real-time monitoring and detection involves deploying the trained model to a real network environment and performing real-time monitoring and detection, monitoring the IP addresses of access requests, especially identifying a large number of requests from the same IP address or a range of IP addresses, which suggests web crawler activity. This invention can distinguish between normal users and web crawlers, helping websites and service providers effectively prevent malicious web crawlers and protect their data and resources from adverse effects.

The application discloses a mineral resource prediction method and device based on a deep network agent, and comprises the following steps: surveying a deposit through an exploration component, obtaining exploration data of the deposit, and constructing a mineralization numerical calculation model of the deposit based on the exploration data; outputting a mineralization condition parameter calculation result through the mineralization numerical calculation model, and training a deep neural network model; after resampling model parameters, performing forward calculation on the mineralization condition parameters through the trained deep neural network agent model; generating a data set according to the forward calculation data combined with the exploration data, training a machine learning model based on the data set, and obtaining a trained machine learning model; outputting a preliminary prediction result through the trained machine learning model, detecting the preliminary prediction result through a prediction index, and outputting a prediction result. The mineralization data is obtained through the forward calculation, the most possible mineralization condition and mineralization situation are deduced through the mineralization data, and the prediction accuracy of the prediction model is improved.

The invention relates to the technical field of data collection, in particular to a deep network news data collection method and device, equipment, a medium and a product, and the method comprises the steps: collecting surface news data under each news column of different news platforms; performing first-stage clustering processing on the surface news data in a mode of calculating data density to obtain a first clustering result, and performing second-stage clustering processing on the first clustering result to obtain a second clustering result; classifying the surface news data into each category of the second clustering result, respectively extracting keywords in each category, and generating a keyword list of each category; and based on the keywords in the keyword list, collecting deep network news data under each category. Through the first-stage clustering processing and the second-stage clustering processing, cross-type event clustering and same-type event clustering are carried out respectively, overlapping and fuzziness between categories are reduced, and the collection integrity and diversity of deep network news data are effectively improved.