Open source data processing method and system based on adversarial samples

Abstract

The invention discloses an open source data processing method and system based on adversarial samples. The method comprises the following steps of: splitting open source data information X into a plurality of inseparable minimum units, forming a picture set A, generating an adversarial sample picture set D according to the picture set A and an identification model B, enabling the difference between the picture d and the picture a to be smaller than a preset threshold delta, and finally splicing the adversarial sample picture set D to generate open source data information X' for network front-end display. According to the technical scheme, on the premise that normal reading and open source data use of a common user are not affected, a web crawler is hard to correctly analyze information inthe data even if the web crawler can capture the data, and the cracking difficulty and cost are improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to an open source data processing method and system based on adversarial examples. Background technique [0002] With the rapid development of information technology today, it is difficult for users to protect their data from being easily obtained and used by others. Companies and individuals generally display their data in an open source manner, such as through web pages, but due to the existence of web crawlers, attackers can easily use web crawlers to obtain open source data of companies and individuals. [0003] A web crawler is a program or script that automatically grabs open source data on the World Wide Web according to certain rules. People can use web crawlers to obtain the data information they need in batches instead of manually obtaining it, which saves manpower and material resources. For some important data, such as data that is valuable to the company or ...

AI Technical Summary

Problems solved by technology

[0004] The existing anti-crawler strategies are as follows: (1) Prohibition of ip, User-Agent or cookies: the operation and maintenance personnel of the web page find the recent abnormal access ip and User-Agent or cookies by analyzing the logs, and prohibit them through the blacklist mechanism Access; (2) Verification code verification: When a user visits too many times, the request will automatically jump to a verification code page, and the website can only be accessed after entering the correct verification code; (3) javascript rendering or ajax asynchronous transmission: web page developers put important information in the web page but do not write it into html tags, and the browser will automatically render 标签中的js代码将信息展现在浏览器当中，而爬虫是不具备执行js代码的能力，所以无法将js事件产生的信息读取出来，访问网页的时候服务器将网页框架返回给客户端，在与客户端交互的过程中通过异步ajax技术传输数据包到客户端，呈现在网页上，爬虫直接抓取的话信息为空；(4)提高数据识别难度：一些经常更新的重要数据可以改变其表现形式，例如将文本信息分割成一系列的图片，即使网络爬虫获取到相关的数据，后期也难以分析和处理

[0005] Although the above anti-crawler strategy can reduce the probability of crawlers grabbing data to a certain extent, it also has a greater impact on normal users. For example, the blacklist setting of the first solution is not good, and it is easy to accidentally injure normal users

Images