A method for obtaining game edition numbers and approval numbers by using a crawler and OCR technology

Abstract

The application discloses a method for obtaining game edition numbers and examination and approval numbers by using a crawler and an OCR technology, and comprises the following steps: extracting a record game URL, establishing a game website crawler model, crawling a webpage to obtain game documents and new game URLs, updating a game crawling queue, determining game URLs to be crawled next time, repeatedly performing a crawling task to obtain more game documents, detecting whether the game documents contain game versions and examination and approval numbers, if not, logging into a game interface and taking a screenshot, extracting game information text by using an OCR technology, detecting whether the game information text contains game edition numbers and examination and approval numbers, and if not, directly inserting the game URL into an alarm list. By using the application, data can be obtained from a general protocol and data can also be obtained from a private protocol, so that the technical problem of a game supervision department in supervision is solved, the supervision range is expanded, and the supervision effect is improved.

Description

Technical Field

[0001] This invention relates to a data scraping technology related to the regulation of online games, and in particular to a method for obtaining game license numbers and approval numbers using web crawling and OCR technology. Background Technology

[0002] With the booming development of the internet industry in my country, online games have become increasingly diverse. The main users of online games are teenagers. Some criminals have taken advantage of this to insert unhealthy content into games, causing numerous negative impacts on young people. Therefore, the government has established a dedicated game regulatory department to supervise domestic game developers and operators.

[0003] Currently, the regulation of games in China is mainly carried out through game licenses and approval numbers. Domestic game developers and operators need to apply for a game license and obtain an approval number before they can release their games to users. However, in actual operation, some game developers and operators launch games before uploading the game license and approval number, and there are even frequent cases of misuse, fabrication, or lack of licenses and approval numbers.

[0004] In response to the aforementioned irregularities in the operation of domestic games, the national game regulatory authorities intend to proactively intervene and rectify the situation. However, the way game websites record game license numbers and approval numbers is inconsistent. Some store them in response documents, while others display them on the game interface. Furthermore, different game websites use different protocols, such as HTTP / HTTPS, while others use proprietary protocols. Conventional technical methods cannot simultaneously adapt to this complex situation. Therefore, using conventional technical means makes it difficult to obtain comprehensive data and achieve the desired regulatory effect. Summary of the Invention

[0005] Given the chaotic state of domestic game operations, in the actual game operation environment, game regulatory authorities lack truly reliable and effective means to proactively obtain game licenses or approval numbers from game operators. The purpose of this invention is to propose a method for obtaining game licenses and approval numbers using web crawling and OCR technologies. This method can obtain data not only from general protocols but also from proprietary protocols, thereby solving the technical problems faced by game regulatory authorities in supervision, expanding the scope of supervision, and improving the effectiveness of supervision.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A method for obtaining game license numbers and approval numbers using web crawling and OCR technology includes the following steps:

[0008] S1. Extract the registered game URLs from the regulatory database;

[0009] S2. Establish a game website crawler model and put the registered game URLs into the game crawling queue as the initial crawling targets;

[0010] S3. Schedule a crawling task to crawl pages and obtain game documents and new game URLs;

[0011] S4. Based on the filtering strategy, filter out links from the new game URLs that are irrelevant to the crawling target;

[0012] S5. Add the filtered new game URLs to the game crawling queue to be updated, and update the game crawling queue according to the update strategy.

[0013] S6. From the updated game team crawl list, determine the priority of the URLs according to the crawling strategy, and determine the next game URL to be crawled;

[0014] S7. Repeat steps S3 to S6 to obtain more game documents;

[0015] S8. Read the obtained game document and check if it contains the game version number and approval number. If not, continue to the next step.

[0016] S9. Read the game URL, log in to its game interface, and take a screenshot to obtain its game interface image;

[0017] S10. Use an OCR text recognition algorithm to perform text recognition processing on the obtained game interface image to obtain game information text;

[0018] S11. Read the game information text and check whether it contains the game version number and approval number. If it does not contain them, directly insert the game URL into the alarm list.

[0019] To further optimize the technical solution, the game website crawler model adopts a focused web crawler model. The structure of the focused web crawler model includes an initial URL set, a URL queue, a page crawling module, a page analysis module, a page database, a link filtering module, a content evaluation module, and a link evaluation module.

[0020] Further optimizing the technical solution, in step S4, the purpose of the filtering strategy is to filter out links that do not meet the target. The specific filtering method is as follows:

[0021] S41. Remove links that do not conform to the protocol, i.e., invalid links;

[0022] S42. According to the rules of the game platform, determine whether the link meets the requirements of the game platform, and remove the link that does not meet the rules of the game platform;

[0023] S43, Deduplicating links.

[0024] Further prioritizing the technical solution, in step S5, the update strategy specifically includes the following steps:

[0025] S51. For content that has not been crawled, add it directly to the crawling queue;

[0026] S52. For crawled content, first determine whether the link history is available. Add unavailable links directly to the queue and build an update model based on the historical update cycle to predict the update cycle. If the update cycle is exceeded, add the link to the queue.

[0027] To further optimize the technical solution, in step S6, the crawling strategy adopts any one of the following strategies:

[0028] Traffic crawling strategy: crawl links with higher visitor counts first;

[0029] Update cycle crawling strategy: crawl those with longer update cycles first;

[0030] Availability crawling strategy: crawl valid links first.

[0031] To further optimize the technical solution, in steps S8 and S11, the detection methods for the game version number and approval number are as follows:

[0032] a. Obtain the game name, approval number, game version number, publishing unit, operating unit, and adoption time published by the State Press and Publication Administration to establish a filing comparison database;

[0033] b. Compare the obtained text information with the filing comparison database to determine whether there is game information in the filing comparison database that completely matches the game name, publisher, and operator;

[0034] c. If they exist, then use regular expressions to obtain the approval number and game version number according to the naming rules of the approval number and game version number and the time when the country adopted them.

[0035] The beneficial effects of this invention are: this solution can obtain game license numbers and approval numbers not only from general protocols but also from proprietary protocols. This solution not only acquires game data quickly and accurately but also overcomes the limitation of current web crawler technologies in capturing game data from proprietary protocols. This solution has a wider data collection scope and obtains more comprehensive data, thus expanding the scope of supervision and ensuring the effectiveness of supervision while minimizing manual costs. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the process of obtaining game license numbers and approval numbers in this invention. Detailed Implementation

[0037] In China, game developers and operators need to apply for a game license number and obtain an approval number before they can legally release their games to users. Typically, before a game is launched, it needs to upload its game license number and approval number to a regulatory database. Therefore, the regulatory database stores the URLs of registered games.

[0038] like Figure 1 As shown, the specific implementation process of the present invention is as follows:

[0039] S1. Extract the registered game URLs from the regulatory database;

[0040] S2. Establish a game website crawler model and put the registered game URLs into the game crawling queue as the initial crawling targets;

[0041] The game website crawler model adopts the focused web crawler model. The structure of the focused web crawler model includes an initial URL set, a URL queue, a page crawling module, a page analysis module, a page database, a link filtering module, a content evaluation module, and a link evaluation module.

[0042] S3. Schedule a crawling task to crawl pages and obtain game documents and new game URLs;

[0043] S4. Based on the filtering strategy, filter out links from the new game URLs that are irrelevant to the crawling target;

[0044] The purpose of the filtering strategy is to filter out links that do not meet the target criteria. The specific filtering methods are as follows:

[0045] S41. Remove links that do not conform to the protocol, i.e., invalid links;

[0046] S42. According to the rules of the game platform, determine whether the link meets the requirements of the game platform, and remove the link that does not meet the rules of the game platform;

[0047] S43. Deduplicating links;

[0048] S5. Add the filtered new game URLs to the game crawling queue to be updated, and update the game crawling queue according to the update strategy.

[0049] The update strategy is as follows:

[0050] S51. For content that has not been crawled, add it directly to the crawling queue;

[0051] S52. For crawled content, first determine whether the link history is available. Add unavailable links directly to the queue and build an update model based on the historical update cycle to predict its update cycle. If the update cycle is exceeded, add it to the queue.

[0052] S6. From the updated game team crawl list, determine the priority of the URLs according to the crawling strategy, and determine the next game URL to be crawled;

[0053] The crawling strategy can adopt any of the following strategies:

[0054] Traffic crawling strategy: crawl links with higher visitor counts first;

[0055] Update cycle crawling strategy: crawl those with longer update cycles first;

[0056] Availability crawling strategy: crawl valid links first.

[0057] S7. Repeat steps S3 to S6 to obtain more game documents;

[0058] S8. Read the obtained game document and check if it contains the game version number and approval number. If not, continue to the next step.

[0059] S9. Read the game URL, log in to its game interface, and take a screenshot to obtain its game interface image;

[0060] S10. Use an OCR text recognition algorithm to perform text recognition processing on the obtained game interface image to obtain game information text;

[0061] S11. Read the game information text and check whether it contains the game version number and approval number. If it does not contain them, directly insert the game URL into the alarm list.

[0062] Furthermore, in steps S8 and S11, the specific methods for detecting the game version number and approval number are as follows:

[0063] a. Obtain the game name, approval number, game version number, publishing unit, operating unit, and adoption time published by the State Press and Publication Administration to establish a filing comparison database;

[0064] b. Compare the obtained text information with the filing comparison database to determine whether there is game information in the filing comparison database that completely matches the game name, publisher, and operator;

[0065] c. If they exist, then use regular expressions to obtain the approval number and game version number according to the naming rules of the approval number and game version number and the time when the country adopted them.

[0066] The technical solution of this invention integrates web crawler technology and OCR text recognition technology, which can maximize the advantages of speed and accuracy of the technology, and make up for the shortcomings of current crawler technology in obtaining non-HTTP / HTTPS protocol / channel data. It can expand the scope of supervision and ensure the effectiveness of supervision while minimizing manual costs.

[0067] The foregoing has shown and described the basic principles, main features, and advantages of this solution. Those skilled in the art should understand that this solution is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this solution. Various changes and modifications can be made to this solution without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed solution. The scope of protection of this solution is defined by the appended claims and their equivalents.

Claims

1. A method for obtaining game license numbers and approval numbers using web crawling and OCR technology, characterized in that, Specifically, the steps include the following: S1. Extract the registered game URLs from the regulatory database; S2. Establish a game website crawler model and put the registered game URLs into the game crawling queue as the initial crawling targets; S3. Schedule a crawling task to crawl pages and obtain game documents and new game URLs; S4. Based on the filtering strategy, filter out links from the new game URLs that are irrelevant to the crawling target; S5. Add the filtered new game URLs to the game crawling queue to be updated, and update the game crawling queue according to the update strategy. The update strategy specifically includes the following steps: S51. For content that has not been crawled, add it directly to the crawling queue; S52. For crawled content, first determine whether the link history is available. Add unavailable links directly to the queue and build an update model based on the historical update cycle to predict the update cycle. If the update cycle is exceeded, add the link to the queue. S6. From the updated game team crawl list, determine the priority of the URLs according to the crawling strategy, and determine the next game URL to be crawled; S7. Repeat steps S3 to S6 to obtain more game documents; S8. Read the obtained game document and check if it contains the game version number and approval number. If not, continue to the next step. S9. Read the game URL, log in to its game interface, and take a screenshot to obtain its game interface image; S10. Use an OCR text recognition algorithm to perform text recognition processing on the obtained game interface image to obtain game information text; S11. Read the game information text and check whether it contains the game version number and approval number. If it does not contain them, directly insert the game URL into the alarm list. The methods for detecting and identifying the game version number and approval document number mentioned in steps S8 and S11 are as follows: a. Obtain the game name, approval number, game version number, publishing unit, operating unit, and adoption time published by the State Press and Publication Administration to establish a filing comparison database; b. Compare the obtained text information with the filing comparison database to determine whether there is game information in the filing comparison database that completely matches the game name, publisher, and operator; c. If they exist, then use regular expressions to obtain the approval number and game version number according to the naming rules of the approval number and game version number and the time when the country adopted them.

2. The method for obtaining game license numbers and approval numbers using web crawler and OCR technology as described in claim 1, characterized in that, The game website crawler model adopts the focused web crawler model. The structure of the focused web crawler model includes an initial URL set, a URL queue, a page crawling module, a page analysis module, a page database, a link filtering module, a content evaluation module, and a link evaluation module.

3. The method for obtaining game license numbers and approval numbers using web crawler and OCR technology as described in claim 1, characterized in that, The purpose of the filtering strategy described in step S4 is to filter out links that do not meet the target. The specific filtering method is as follows: S41. Remove links that do not conform to the protocol, i.e., invalid links; S42. According to the rules of the game platform, determine whether the link meets the requirements of the game platform, and remove the link that does not meet the rules of the game platform; S43, Deduplicating links.

4. The method for obtaining game license numbers and approval numbers using web crawler and OCR technology as described in claim 1, characterized in that, The crawling strategy described in step S6 can be any of the following strategies: Traffic crawling strategy: crawl links with higher visitor counts first; Update cycle crawling strategy: crawl those with longer update cycles first; Availability crawling strategy: crawl valid links first.

Images