Apparatus and method for verifying similarity of illegal bypass site

Abstract

The present invention relates to an apparatus and a method for verifying the similarity of an illegal bypass site. According to the present invention, the apparatus for verifying the similarity of an illegal bypass site comprises: a learning unit for learning a domain change pattern for at least one illegal site from a URL change history of the illegal site for each time on the basis of on machine learning and constructing a domain tracking model; a data input unit for receiving an input of an existing URL of an illegal site to be tracked; a tracking unit for applying the existing URL to the pre-learned domain tracking model to derive and provide, as a prediction result, at least one bypass URL expected to be changed from the existing URL; and a verification unit for verifying a domain tracking result by determining whether the bypass URL and the existing URL correspond to the same site.

Description

Device and method for verifying the similarity of illegal bypass sites

[0001] The present invention relates to a device and method for verifying the similarity of an illegal bypass site, and more particularly, to a device and method for verifying the similarity of an illegal bypass site that can detect a modified URL by tracking a bypass URL address for an overseas illegal site and verify whether the modified URL is the same site as the existing URL.

[0002] Recently, illegal copyright infringement sites that use overseas servers or cloud services, such as webtoons, streaming, and torrents, have become a social issue.

[0003] In the case of overseas cloud services, there are limitations to applying domestic laws and actively investigating them, so blocking domestic inflow is becoming a more realistic alternative than direct investigation.

[0004] However, most illegal sites immediately open a bypass site and continue providing services when the URL is blocked.

[0005] Blocking access to illegal overseas websites in Korea requires review by the Korea Communications Standards Commission, a process that takes at least three months. Furthermore, if a blocked site bypasses the service and resumes service, it takes approximately three to seven days to re-verify the site and re-block it.

[0006] Recently, illegal sites automatically change their URL addresses every 3 to 5 days, making blocking very ineffective.

[0007] In the early days, URL changes were made by sequentially increasing the numbers one by one, such as copytoon to copytoon1, copytoon2, copytoon3, etc. Recently, it has evolved to a method of jumping numbers, such as copytoon34 to copytoon100.

[0008] Therefore, for effective URL blocking, a technology is needed that can actively track bypass sites and obtain the next URL.

[0009] The technology underlying the present invention is disclosed in Korean Patent Publication No. 10-2016-0028709 (published on March 14, 2016).

[0010] The purpose of the present invention is to provide a device and method for verifying the similarity of an illegal bypass site, which can detect a modified URL by tracking a bypass URL address for an overseas illegal site and verify whether the modified URL is the same site as the original URL.

[0011] The present invention relates to a device for verifying the similarity of an illegal bypass site, comprising: a learning unit for learning a domain change pattern of at least one illegal site based on machine learning from a time-based URL change history of the illegal site and constructing a domain tracking model; a data input unit for receiving an existing URL of the illegal site to be tracked; a tracking unit for applying the existing URL to the previously learned domain tracking model to derive and provide at least one bypass URL expected to be changed from the existing URL as a prediction result; and a verification unit for verifying a domain tracking result by determining whether there is site identity between the bypass URL and the existing URL.

[0012] In addition, the verification unit can determine whether the site is identical between the detour URL and the existing URL by comparing at least one of the characteristic information extracted from the URL purchase point and the URL access page between the detour URL and the existing URL.

[0013] Additionally, the above characteristic information may include at least one of a logo image, an advertisement image, and a hash value detected on the URL access page.

[0014] In addition, the verification unit may determine that the sites between the bypass URL and the existing URL are the same if the time of purchase of the URL between the bypass URL and the existing URL is the same or if the time of purchase of the bypass URL is within a set time from the time of site blocking of the existing URL or if the characteristic information detected on the URL access page matches.

[0015] In addition, the tracking unit provides a list of predicted detour URLs by sorting the plurality of detour URLs in descending order of probability based on the probability values ​​for each of the plurality of detour URLs derived by the domain tracking model, and the verification unit performs a site similarity verification between the highest priority detour URL and the existing URL, and if the sites are determined not to be the same, selects the next priority detour URL and performs a site similarity verification with the existing URL.

[0016] And, the present invention provides a method for verifying similarity performed by a similarity verification device for an illegal bypass site, comprising the steps of: learning a domain change pattern for at least one illegal site based on machine learning from a time-based URL change history for the illegal site and constructing a domain tracking model; receiving an existing URL of the illegal site to be tracked as an input; applying the existing URL to the previously learned domain tracking model to derive and provide at least one bypass URL expected to be changed from the existing URL as a prediction result; and verifying a domain tracking result by determining whether there is site identity between the bypass URL and the existing URL.

[0017] According to the present invention, a bypass URL address for an illegal site can be predicted based on artificial intelligence, and the expected next URL address after blocking can be actively tracked and secured.

[0018] In addition, the present invention can secure the reliability of the domain tracking model by additionally verifying whether the tracked URL address is the same site as the existing URL address, and enables efficient tracking and rapid blocking of illegal copyright infringement sites that continuously change their domains.

[0019] In particular, the present invention can effectively recognize, track, and block overseas illegal copyright infringement sites that illegally distribute copyrighted material through bypass sites by periodically changing URL addresses through overseas servers, thereby enabling a rapid response to copyright infringement and activating the copyright distribution ecosystem.

[0020] FIG. 1 is a diagram showing the configuration of a similarity verification device for an illegal bypass site according to an embodiment of the present invention.

[0021] Figure 2 is a diagram exemplifying the domain change pattern of an illegal site.

[0022] Figures 3a and 3b are diagrams showing an example of an identity verification method using hash value calculation.

[0023] Figure 4 is a drawing showing an example of an identity verification method using an advertisement or logo.

[0024] Fig. 5 is a drawing explaining a similarity verification method of an illegal bypass site using the device of Fig. 1.

[0025] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily practice the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0026] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the cases where the parts are "directly connected" but also the cases where the parts are "electrically connected" with other elements intervening. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes other components, unless otherwise stated.

[0027] The present invention relates to a similarity verification technology for illegal bypass sites, and proposes a technique that predicts bypass URLs for overseas illegal sites based on artificial intelligence, actively tracks and secures the next URL address expected to be newly opened after blocking, and verifies whether it is identical.

[0028] FIG. 1 is a diagram showing the configuration of a similarity verification device for an illegal bypass site according to an embodiment of the present invention.

[0029] As shown in Fig. 1, the similarity verification device (100) of an illegal bypass site according to an embodiment of the present invention includes a data collection unit (110), a learning unit (120), a data input unit (130), a tracking unit (140), and may further include a verification unit (150). Here, the operation of each unit (110 to 150) and the data flow between each unit may be controlled by a control unit (not shown).

[0030] The data collection unit (110) collects the hourly URL change history for at least one illegal site.

[0031] In the embodiment of the present invention, an illegal overseas site may include an illegal copyright infringement site that provides illegal services such as webtoons, dramas, and movies by placing servers overseas or using cloud services, and an illegal site related to entertainment such as games and gambling.

[0032] This data collection unit (110) can collect URL change history data over time for multiple illegal websites by receiving it through user terminals, etc. The collected data is used for machine learning.

[0033] The learning unit (120) analyzes the data collected through the data collection unit (110) based on machine learning, and trains a domain tracking model to predict the next detour URL expected to be opened based on the current URL input.

[0034] Specifically, the learning unit (120) can learn the domain change pattern of an illegal site by analyzing the relationship between the URL before and after the change through machine learning from the past URL change history collected for the illegal site, and build a domain tracking model.

[0035] Domain change patterns of illegal sites can vary. In an embodiment of the present invention, domain change patterns analyzed through machine learning may include at least one of the following: whether a parameter of interest within the URL text changes, the parameter change pattern, the parameter change cycle, whether there is a change in location between different parameters of interest, the location change pattern, and the site characteristics of the URL.

[0036] Here, the parameter of interest can include at least one of the following: a number, a letter, or a top-level domain (TLD) within the URL text. Furthermore, the site's character relates to the type of service content provided by the site associated with the URL, which can be categorized as entertainment, gambling, games, webtoons, dramas, etc. Since preferred domain change methods may vary depending on the site's character, the site's character can also be included in the learning pattern.

[0037] When the parameter of interest is a number, the domain change pattern may include a numerical increase pattern, a numerical decrease pattern, a numerical change cycle, a fixed numerical increase or decrease range, a numerical increase or decrease range that varies over time, etc. when the parameter of interest is a number.

[0038] Figure 2 is a diagram illustrating domain change patterns of illegal websites. Figure 2 illustrates ten representative examples.

[0039] First, in case 1, it is a pattern that increases the number included in the URL domain by 1, and the URL after the change corresponds to the URL before the change with the number increased by 1 (before change: agit248.com, after change: agit249.com).

[0040] Number 2 shows a domain change method that removes the last part of the URL's top-level domain (TLD), corresponding to the deletion of the address ending with the slash after "com." Number 3 is a variation of number 1, with the numbers increasing by 2.

[0041] Pattern 4 is a change in the TLD alone, where the top-level domain (TLD) changes from "org" to "cc" (before: tvhot.org, after: tvhot.cC). Pattern 5 corresponds to a change pattern where the numbers increase significantly, and Pattern 6 shows a pattern where the number of repeated numbers within the URL increases in a cluster.

[0042] Number 7 is a case where two URLs are simultaneously preempted. If one of them is blocked, the other one is used to bypass the service. Number 8 corresponds to a pattern where the positions of the numbers and letters in the URL change, and the numbers themselves are also changed.

[0043] Number 9 refers to services using similar domains under the same business name, which also applies to the simultaneous use of two domains. Number 10 also applies to cases involving two or more complex patterns, combining numerical changes with TLD changes.

[0044] Since it's virtually impossible to know which domain change patterns will be applied to illegal sites, this embodiment allows for learning by reflecting all possible patterns. Furthermore, by collecting and analyzing recent domain change history, the model can be designed to prioritize patterns with a high probability of reflecting recent trends.

[0045] The learning unit (120) can continuously train and update the domain tracking model by assigning higher weights to recently collected data sets within a set period. This can improve the performance and reliability of the domain tracking model by better reflecting recent address modification trends and patterns.

[0046] In an embodiment of the present invention, the learning unit (120) can learn a domain tracking model based on a deep learning algorithm, and can use various known machine learning algorithms. For example, a deep neural network (DNN), a recurrent neural network (RNN), a logistic regression (LR) algorithm, etc. can be utilized.

[0047] The learning unit (120) can individually learn the URL change history data for each illegal site and build multiple domain tracking models that are applied to each site, but it can also learn the URL change history data for all illegal sites and build a single domain tracking model that is commonly applied to all sites.

[0048] Once the model is completed through learning, it can provide a predicted URL address that is expected to change in response to input of existing URL information about the illegal site being tracked.

[0049] To this end, the data input unit (130) receives the existing URL of any illegal site to be tracked and transmits it to the tracking unit (140). Here, the existing URL may correspond to the URL information for the illegal site prior to the address change.

[0050] The tracking unit (140) can apply an existing URL to a domain tracking model learned based on artificial intelligence, and derive at least one detour URL expected to change from the existing URL as a prediction result, thereby providing a list of expected detour URLs.

[0051] In this way, according to an embodiment of the present invention, efficiency can be maximized by using a learning-type bypass site tracking algorithm through the introduction of artificial intelligence.

[0052] Here, the tracking unit (140) can provide a list by sorting the multiple detour URLs in order of highest probability value based on the probability values ​​for each of the multiple detour URLs derived by the domain tracking model.

[0053] In addition, the tracking unit (140) can filter and provide a list of the top N reliable detour URLs whose probability values ​​are greater than a set value based on the probability values ​​for each of the multiple detour URLs derived by the domain tracking model.

[0054] Additionally, the tracking unit (140) can purchase and preemptively occupy a selected portion of the detour URLs within the list of anticipated detour URLs before the corresponding illegal site occupies them. Here, the selected portion of the list may correspond to detour URLs that are currently unoccupied but are expected to be occupied in the future, and URLs with a high probability of being ranked high within the list of anticipated detour URLs may be primarily targeted.

[0055] In an embodiment of the present invention, a process of verifying whether the detected bypass URL is actually the same site as the existing URL can be performed.

[0056] To this end, the verification unit (150) verifies the domain tracking result by determining whether there is site identity between the tracked bypass URL and the existing URL.

[0057] Here, the verification unit (150) can perform a site similarity verification between the highest priority detour URL and the existing URL within the list of predicted detour URLs sorted in descending order of probability value, and terminate the similarity verification process if the sites are identical. However, if the highest priority detour URL and the existing URL are determined to be different sites, the next highest priority detour URL can be selected to perform a site similarity verification with the existing URL, and the process can be repeated until the URLs are confirmed to be identical.

[0058] In an embodiment of the present invention, the verification unit (150) can determine whether the site is identical between the detour URL and the existing URL by comparing at least one of the characteristic information extracted from the URL purchase time and the URL access page between the detour URL and the existing URL.

[0059] Here, the feature information may include at least one of a logo image, an advertisement image, and a hash value detected on the URL access page.

[0060] Typically, advertising placements may remain in a fixed location, or they may change with each site visit. While minor changes may occur, such as the addition of new ads after a URL bypass is created, or the discontinuation of ads whose contracts have expired, if the same ad banner appears on the access page, it will have the same subscription code and hash value. Furthermore, logo images are always provided in the same location and as identical files, ensuring the overall site layout and appearance remain consistent. Considering this, in the embodiment of the present invention, the verification unit (150) extracts logos, advertising images, and hash values ​​displayed on the access page as feature information, and can determine whether sites are identical based on the comparison and consistency of the feature information.

[0061] Here, the verification unit (150) can detect relevant feature information about advertisements or logo images from the access page for the URL based on a pre-trained deep learning algorithm. At this time, a hash value calculation tool or feature recognition algorithm, such as a CNN, can be utilized to detect relevant features on the access screen.

[0062] A specific example of site identity verification through comparison of feature information is as follows.

[0063] Figures 3a and 3b are diagrams exemplifying an identity verification method using hash value calculation. Figure 3a shows a hash value calculation method using a hash value calculation program (HashTab tool). Hash comparison is possible by right-clicking on a file requiring hash value verification > Properties > File Hash. At this time, the file identity can be determined using the MD5 hash value. The file to be detected as a hash value may correspond to a logo or advertisement image file displayed on the connected screen. Figure 3b shows a hash value calculation method using an online tool provided on the web. Identity verification is possible by dropping a file requiring hash value verification on the web, calculating the hash value, and comparing the results.

[0064] Figure 4 is a diagram exemplifying a method of verifying identity using advertisements or logos. The upper figure in Figure 4 shows the access screen via a conventional URL address, and the lower figure shows the access screen via a detour URL address. The logo in the upper left corner of the screen remains the same, while the advertisements are arranged slightly differently. If the feature similarity between the logo images exceeds a threshold, the logos are considered to match, and the two sites can be determined to be the same. If the feature similarity between the provided advertisement banner images exceeds a threshold, the advertisements are considered to match, and the two sites can be determined to be the same. In this case, if all advertisement banners present on the access screen are checked and a preset number of matching advertisement banners are found, the sites can be determined to be the same.

[0065] In addition, the verification unit (150) may determine that the sites between the bypass URL and the existing URL are the same if the time of purchase of the URL between the bypass URL and the existing URL is the same or if the time of purchase of the bypass URL is within a set time from the time of site blocking of the existing URL.

[0066] Operators of illegal websites typically purchase multiple similar URL addresses in advance to prepare for site blocking. Therefore, if two URLs are purchased at the same time, they can be considered the same site. Furthermore, operators tend to repurchase similar URLs after a site is blocked. Therefore, if the predicted bypass URL was purchased and created within a certain period of time (e.g., within three days) of the original URL being blocked, it can be considered the same site.

[0067] In addition, the verification unit (150) can determine that the sites between the detour URL and the original URL are the same if the characteristic information detected on the URL access page matches with each other.

[0068] Of course, in order to increase the accuracy of identity verification, if the conditions that the time of purchase of the URL between the detour URL and the original URL is the same or the time of purchase of the detour URL is within a set time from the time of site blocking of the original URL, and the characteristic information detected on the URL access page matches each other are all satisfied, the site between the detour URL and the original URL can be determined to be the same.

[0069] Additionally, the verification unit (150) can exploit the large amount of traffic flowing into illegal sites and domain change patterns to filter out cases where third-party advertising sites completely unrelated to the site appear in domain tracking. Furthermore, if a site announces a detour on its previous site, the verification unit (150) can verify whether the site is actually providing services or is being used as a site to announce detour addresses. In this way, the verification unit (150) can filter third-party advertising sites and distinguish between sites that announce detour addresses.

[0070] The similarity verification device (100) according to an embodiment of the present invention may be a server or device for predicting and verifying, monitoring, and monitoring the bypass address of an illegal website in real time. It may also correspond to a web- or app-based application implemented on a user terminal. In this case, the terminal may be connected to the similarity verification device (100) through a network while the application is running, thereby receiving bypass address tracking and verification services.

[0071] Fig. 5 is a drawing explaining a similarity verification method of an illegal bypass site using the device of Fig. 1.

[0072] First, the data collection unit (110) collects the hourly URL change history for at least one illegal website (S510). Then, the collected data is transmitted to the learning unit (120).

[0073] The learning unit (120) builds a domain tracking model by artificial intelligence analyzing the relationship between URLs before and after changes included in the hourly URL change history (S520).

[0074] This learning unit (120) can use the hourly URL change history of illegal sites as learning data and, through an artificial intelligence algorithm, create a domain tracking model to predict at least one next expected detour URL based on the currently input URL.

[0075] The learning unit (120) can train the model through deep learning, and the reliability of the model can be increased through learning big data on the URL change history of one or more illegal sites.

[0076] After the model is built through learning, you can predict the URL address expected to change in the future by inputting only the existing URL information of the illegal site to be tracked into the model.

[0077] To this end, the data input unit (130) receives the existing URL of the illegal site to be tracked (S530).

[0078] Then, the tracking unit (140) applies the existing URL of the illegal site to be tracked to the pre-learned domain tracking model, and derives at least one detour URL expected to be changed from the existing URL as a prediction result (S540).

[0079] In addition, the tracking unit (140) provides a list of URLs expected to be bypassed for the illegal site to be tracked derived from the domain tracking model (S550).

[0080] Afterwards, the verification unit (150) determines whether the tracked bypass URL is the same site as the existing URL, and verifies the domain tracking result derived in step S540 (S560).

[0081] According to the present invention as described above, by predicting a bypass URL for an illegal overseas site based on artificial intelligence, the expected next URL address can be actively tracked and secured after blocking.

[0082] In addition, by additionally verifying whether the tracked URL address is the same site as the existing URL address, the reliability of the domain tracking model can be secured, and it enables efficient tracking and rapid blocking of illegal copyright infringement sites that continuously change their domains.

[0083] In addition, the present invention can effectively recognize, track, and block overseas illegal copyright infringement sites that illegally distribute copyrighted material through bypass sites by periodically changing URL addresses through overseas servers, thereby enabling a rapid response to copyright infringement and activating the copyright distribution ecosystem.

[0084] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the appended claims.

Claims

1. In a device for verifying the similarity of illegal bypass sites, A learning unit that learns a domain change pattern for at least one illegal site based on machine learning from the hourly URL change history for the illegal site and builds a domain tracking model; Data input section for entering the existing URL of the illegal site to be tracked; A tracking unit that applies the existing URL to the previously learned domain tracking model and provides at least one detour URL expected to be changed from the existing URL as a prediction result; and A similarity verification device for an illegal bypass site, including a verification unit that verifies the domain tracking result by determining whether the site is identical between the above-mentioned bypass URL and the above-mentioned existing URL.

2. In claim 1, The above verification department, A device for verifying the similarity of an illegal bypass site that determines whether the bypass URL and the existing URL are identical sites by comparing at least one of the characteristic information extracted from the URL purchase point and the URL access page between the bypass URL and the existing URL.

3. In claim 2, The above feature information is, A similarity verification device for illegal bypass sites that include at least one of a logo image, an advertisement image, and a hash value detected on a URL access page.

4. In claim 3, The above verification department, A device for verifying the similarity of illegal bypass sites that determines that the sites between the bypass URL and the existing URL are the same if the time of purchase of the URL between the above-mentioned bypass URL and the existing URL is the same, or if the time of purchase of the above-mentioned bypass URL is within a set time from the time of site blocking of the above-mentioned existing URL, or if the characteristic information detected on the above-mentioned URL access page matches, is the same.

5. In claim 1, The above tracking unit, Based on the probability values ​​of multiple detour URLs derived by the above domain tracking model, the multiple detour URLs are sorted in order of highest probability value to provide the list of predicted detour URLs. The above verification department, A device for verifying the similarity of an illegal bypass site by performing a site similarity verification between a top priority bypass URL and the above existing URL, and if it is determined that the sites are not identical, selecting a next priority bypass URL and performing a site similarity verification with the above existing URL.

6. In a similarity verification method performed by a similarity verification device of an illegal bypass site, A step of learning a domain change pattern for at least one illegal site based on machine learning from the hourly URL change history for the illegal site and building a domain tracking model; Step 1: Enter the existing URL of the illegal site to be tracked; A step of applying the above existing URL to the previously learned domain tracking model, and deriving and providing at least one detour URL expected to be changed from the above existing URL as a prediction result; and A method for verifying the similarity of an illegal bypass site, comprising a step of verifying the domain tracking result by determining whether there is site identity between the above-mentioned bypass URL and the above-mentioned existing URL.

7. In claim 6, The above verification steps are: A method for verifying the similarity of an illegal bypass site, which determines whether the bypass URL and the existing URL are identical sites by comparing at least one of the characteristic information extracted from the URL purchase point and the URL access page between the bypass URL and the existing URL.

8. In claim 7, The above feature information is, A method for verifying the similarity of an illegal bypass site that includes at least one of a logo image, an advertisement image, and a hash value detected on a URL access page.

9. In claim 8, The above verification steps are: A method for verifying the similarity of an illegal bypass site, which determines that the site between the bypass URL and the existing URL is the same if the time of purchase of the URL between the above-mentioned bypass URL and the existing URL is the same, or if the time of purchase of the above-mentioned bypass URL is within a set time from the time of site blocking of the above-mentioned existing URL, or if the characteristic information detected on the above-mentioned URL access page matches, is the same.

10. In claim 6, The steps for deriving the above bypass URL are: Based on the probability values ​​of multiple detour URLs derived by the above domain tracking model, the multiple detour URLs are sorted in order of highest probability value to provide the list of predicted detour URLs. The above verification steps are: A method for verifying the similarity of an illegal bypass site by performing a site similarity verification between a top priority bypass URL and the above existing URL, and if it is determined that the sites are not identical, selecting a next priority bypass URL and performing a site similarity verification with the above existing URL.

Images