Network detection method and device, storage medium and electronic equipment

Abstract

The invention discloses a network detection method and device, a storage medium and electronic equipment. The method comprises the following steps: in response to an Internet surfing request of a user side, detecting whether a URL address in the Internet surfing request is a safe website or not through a URL list library detection mode; when it is detected that the URL address is a safe website, the internet surfing request is sent to a website server; receiving a response result sent by the website server for the internet surfing request, detecting the response result to obtain a detection result, the response result comprising at least one of the following: a webpage title and webpage content; the sample library is updated based on detection results, and the detection results comprise at least one of a webpage title detection result and a webpage content detection result. According to the method and the device, the problem that the accuracy of internet content detection is reduced in the prior art is solved.

Description

Background Art

[0002] Among the related technologies, although great progress has been made in Internet content security detection technology, there are still some shortcomings. The detection model based on artificial intelligence has the defect of low accuracy. In addition, the application of supervised deep learning technology faces the problem of data labeling, such as the lack of labeled data, uneven data distribution, and data imbalance compared with real information data, which reduces the accuracy of Internet content detection.

[0003] With respect to the problem of reduced accuracy in detecting online content in related technologies, no effective solution has been proposed so far. Summary of the invention

[0004] The main purpose of the present application is to provide a network detection method, device, storage medium and electronic device to solve the problem of reduced accuracy in detecting online content in related technologies.

[0005] In order to achieve the above-mentioned purpose, according to the first aspect of the present application, a network detection method is provided. The method includes: responding to an Internet access request from a user end, detecting whether the URL address in the Internet access request is a safe URL through a URL list library detection method; sending the Internet access request to a website server when the URL address is detected to be a safe URL; receiving a response result sent by the website server to the Internet access request, detecting the response result, and obtaining a detection result, wherein the response result includes at least one of the following: web page title, web page content; updating the sample library based on the detection result, wherein the detection result includes at least one of the following: web page title detection result, web page content detection result.

[0006] Further, in response to an Internet access request from a user terminal, a URL list library detection method is used to detect whether the URL address in the Internet access request is a safe URL, including: calling a pre-established URL list library, wherein the URL list library includes: a URL blacklist and a URL whitelist; searching whether the URL address in the Internet access request exists in the URL list library by a complete matching method; if it exists, determining whether the URL address is a blacklist address; if the URL address is a blacklist address, detecting that the URL address is not a safe URL; if the URL address is a whitelist address, detecting that the URL address is a safe URL and allowing access; if it does not exist, extracting the root address of the URL address, searching whether the root address exists in the URL list library by a prefix matching method; if it exists, determining whether the root address is a blacklist address; if the root address is a blacklist address, detecting that the URL address is not a safe URL.

[0007] Furthermore, the response result is a web page title, and the response result is tested to obtain a test result, including: extracting title information in the web page title and calculating a hash value of the title information; retrieving whether the hash value exists in a pre-established sensitive title library; if so, detecting that the web page title is an unsafe title; if not, testing the response result by vector retrieval to obtain a test result.

[0008] Furthermore, the response result is detected by vector retrieval to obtain the detection result, including: converting the web page title into a title vector by embedding a model; searching whether the title vector exists in the sensitive title library, if so, detecting that the web page title is an unsafe title, if not, calling the large model technology to perform inference detection on the web page title to obtain the detection result.

[0009] Furthermore, the response result is the web page content, and the response result is tested to obtain the test result, including: splitting the web page content into paragraphs to obtain multiple paragraphs; counting the number of occurrences of sensitive words in each paragraph based on a pre-established sensitive keyword library; sorting each paragraph according to the number of occurrences of sensitive words to obtain a sorting result, and testing the response result based on the sorting result to obtain a test result.

[0010] Furthermore, each paragraph is sorted according to the number of times the sensitive words appear to obtain a sorting result, including: when the number of times the sensitive words appear is non-zero, each paragraph is sorted according to the number of times the sensitive words appear to obtain a sorting result; when the number of times the sensitive words appear is zero, each paragraph is sorted according to the order in which they appear in the web page content to obtain a sorting result.

[0011] Furthermore, the response results are tested based on the sorting results to obtain the test results, including: assembling the target prompt words in sequence based on each paragraph in the sorting results; calling the large model engine to perform reasoning test on the target prompt words to obtain the test results.

[0012] In order to achieve the above-mentioned purpose, according to the second aspect of the present application, a network detection device is provided. The device includes: a response unit, which is used to respond to the Internet access request of the user end, and detect whether the URL address in the Internet access request is a safe URL through the URL list library detection method; a sending unit, which is used to send the Internet access request to the website server end when the URL address is detected to be a safe URL; a receiving unit, which is used to receive the response result sent by the website server end to the Internet access request, detect the response result, and obtain the detection result, wherein the response result includes at least one of the following: web page title, web page content; an updating unit, which is used to update the sample library based on the detection result, wherein the detection result includes at least one of the following: web page title detection result, web page content detection result.

[0013] According to a third aspect of an embodiment of the present application, an electronic device is provided, including: a processor; and a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned network detection methods is implemented.

[0014] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the network detection method according to any one of the above items is implemented.

[0015] Through this application, the following steps are adopted: in response to the Internet access request of the user end, the URL address in the Internet access request is detected by the URL list library detection method to see if it is a safe URL; when the URL address is detected to be a safe URL, the Internet access request is sent to the website server; the response result sent by the website server to the Internet access request is received, and the response result is detected to obtain the detection result, wherein the response result includes at least one of the following: web page title, web page content; the sample library is updated based on the detection result, wherein the detection result includes at least one of the following: web page title detection result, web page content detection result. Through this application, the problem of reduced accuracy of Internet content detection in related technologies is solved. This case uses three-level detection of URLs, web page titles, and web page content. Thereby achieving the effect of improving the accuracy of Internet content detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a flow chart of a network detection method provided according to an embodiment of the present application;

[0018] Figure 2 is a flowchart of address detection logic processing provided according to an embodiment of the present application;

[0019] Figure 3 is a flowchart of the title detection logic processing provided according to an embodiment of the present application;

[0020] Figure 4 is a flowchart of content detection logic processing provided according to an embodiment of the present application;

[0021] Figure 5 It is a schematic diagram of the overall architecture provided according to an embodiment of the present application;

[0022] Figure 6 is a schematic diagram of a network detection device provided according to an embodiment of the present application;

[0023] Figure 7 It is a schematic diagram of the network architecture of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0028] URL: The abbreviation of Uniform Resource Locator, which means "Uniform Resource Locator" in Chinese. It is used to uniquely identify and locate resources on the Internet.

[0029] Blacklist: In the field of network security, blacklists are often used to identify and block potential threats, such as malware, phishing websites, spammers, etc. By blacklisting known malicious entities, the system is able to prevent them from accessing network resources, sending spam, or performing other malicious activities.

[0030] Whitelist: is a term widely used in many fields, especially in network security, system management and database access control. Its basic definition is a clear list or collection of elements that are explicitly allowed or recognized by the system, platform or program. These elements may be user accounts, IP addresses, domain names, software programs or any other form of digital entities.

[0031] According to an embodiment of the present application, a network detection method is provided.

[0032] Figure 1 is a flow chart of a network detection method according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0033] Step S101, in response to a request for Internet access from a user terminal, detecting whether a URL address in the request for Internet access is a safe URL by using a URL list library detection method.

[0034] Exemplarily, when a user terminal initiates an Internet access request, that is, attempts to access a certain network resource, the present application detects the user's Internet access upstream and downstream in the Internet proxy mode. In the upstream stage, the URL list library detection method is used to detect whether the URL address in the Internet access request is a safe URL, which has the effect of improving security, enhancing user experience, and simplifying management processes.

[0035] Specifically, in response to an Internet access request from a user terminal, detecting whether a URL address in the Internet access request is a safe URL by means of a URL list library detection method can be implemented through the following steps: calling a pre-established URL list library, wherein the URL list library includes: a URL blacklist and a URL whitelist; searching whether the URL address in the Internet access request exists in the URL list library by means of a complete match; if so, determining whether the URL address is a blacklist address; if the URL address is a blacklist address, detecting that the URL address is not a safe URL; if the URL address is a whitelist address, detecting that the URL address is a safe URL and allowing access; if not, extracting the root address of the URL address; searching whether the root address exists in the URL list library by means of a prefix match; if so, determining whether the root address is a blacklist address; if the root address is a blacklist address, detecting that the URL address is not a safe URL.

[0036] For example, Figure 2As shown, for each URL address visited on the Internet, the URL address in the complete matching mode is first retrieved from the black and white URL list library. If it is a hit, the security of the URL is judged according to the black and white list attributes, and the detection ends. If it is not a hit, the root of the access URL address is extracted according to the regular expression, that is, the beginning address consisting of the protocol, host name and port number. Then, the root address of the access URL in the prefix matching mode is retrieved from the black and white list library. If it is a hit, the security of the URL is judged according to the black and white list attributes, and the detection ends. If it is not a hit, proceed to the next step of title detection.

[0037] Among them, the URL list library pre-established in this application includes a URL blacklist library and a URL whitelist library. This application establishes a whitelist URL address library for websites with safe web content, such as a library http: / / www.xx.cn / , etc., and at the same time establishes a blacklist library for URL addresses of bad web pages. The black and white list URL addresses can be collected by manual sorting and automatic scanning by the system. The URL address can be the root directory of the website, which is compared by prefix matching. For example, https: / / www.xx.cn / represents all URL addresses starting with https: / / www.xx.cn / . It can also be a specific single web page complete URL address, which is compared by complete matching, such as https: / / www.xx.cn / sp / zbhk / 202409 / t20240927_270208.shtml. Among them, the data storage structure design of the black and white URL list library is shown in Table 1 below:

[0038] Table 1

[0039]

[0040] In order to continuously accumulate URL blacklist and whitelist database data, this application supports automatic detection and saving of unsafe URLs submitted by users to the URL blacklist and white sample database. After the information in the sample database is manually reviewed by the administrator, if it is determined to be unsafe content, it can be added to the URL blacklist and white list database, which can significantly improve the security, protection effect and management efficiency of the system. Among them, the data storage structure design of the URL blacklist and white sample database is shown in Table 2 below:

[0041] Table 2

[0042]

[0043] Step S102: When it is detected that the URL address is a secure website, a request for accessing the Internet is sent to the website server.

[0044] When the URL address is detected to be a secure URL, this application sends the Internet access request to the website server, which can improve user experience, enhance security, and optimize network traffic.

[0045] Step S103, receiving a response result to the Internet access request sent by the website server, detecting the response result to obtain a detection result, wherein the response result includes at least one of the following: a web page title, and web page content.

[0046] The response result may be a web page title, web page content, response header, and redirection information, and the detection result is a security assessment result of the above response result. This application can not only improve security and user experience, but also optimize network traffic and resource utilization by receiving and detecting the response result sent by the website server.

[0047] Specifically, the response result is a web page title. The response result is tested to obtain a test result, which can be achieved through the following steps: extracting title information in the web page title and calculating a hash value of the title information; searching whether the hash value exists in a pre-established sensitive title library; if so, detecting that the web page title is an unsafe title; if not, testing the response result by vector retrieval to obtain a test result.

[0048] For example, Figure 3 As shown in the figure, for online responses, the security is mainly tested based on the content of the web page. Usually the title represents the central idea of ​​the web page content, and the title is generally short, so the title can be tested first. The topic can be extracted from the web page based on the HTML tags, usually in the " <title>< / title> " tag. In order to quickly compare the sensitive title library, the web page title is first hashed according to algorithms such as MD5 and SM3, and the titleHash field in the sensitive title library is retrieved according to the hash value. If a match is found, it means that the title is unsafe, and Internet access is blocked and a warning message is given. Otherwise, the web page title is converted into a vector through an embedding model such as BGE (BAAI General Embedding), and the response result is detected by vector retrieval to obtain the detection result.

[0049] Specifically, the response result is detected by vector retrieval to obtain the detection result, which can be achieved through the following steps: converting the web page title into a title vector by embedding a model; searching whether the title vector exists in the sensitive title library, if so, detecting that the web page title is an unsafe title, if not, calling the large model technology to perform inference detection on the web page title to obtain the detection result.

[0050] For example, continue as Figure 3As shown in the figure, the web page title is converted into a vector through an embedding model such as BGE (BAAI General Embedding), and the vector is searched with the titleVector field in the sensitive title library. If a hit is found, it means that the title is unsafe, and the web page title is inserted into the sensitive title library, blocking Internet access and giving a warning message. Otherwise, the large model is used to determine whether the title content is safe. The large model can deeply understand the complexity and diversity of language through pre-training on large-scale data sets, which makes it more accurate and efficient when processing natural language content.

[0051] Specifically, this application uses large model technology to perform content security detection, and calls the large model through the OneAPI or OpenAI interface to implement reasoning. If unsafe content is detected, the Internet access response is interrupted, and corresponding warning information is given, and the unsafe content is submitted to the URL address sample library and sensitive title library.

[0052] For example, based on the big model technology, the prompt words for title content security detection are written according to the following example:

[0053] start,

[0054] #Role

[0055] You are an advanced content review assistant responsible for analyzing web page titles and determining whether they contain objectionable content.

[0056] #Goals

[0057] Your goal is to identify and analyze titles to ensure they don’t contain the following types of harmful content:

[0058] -Violence and terrorism

[0059] - Pornographic and indecent content

[0060] -Discrimination and hate speech

[0061] - Politically sensitive content

[0062] - Misleading information and fake news

[0063] -Drugs and illegal substances

[0064] -Personal privacy and security

[0065] -Copyright and intellectual property infringement

[0066] -Other harmful content (such as extremism, cyberbullying, etc.)

[0067] #Skills

[0068] You have the following skills:

[0069] -Ability to understand and analyze article titles

[0070] - Identify potentially harmful content in titles

[0071] - Make judgments based on the sensitivity and potential impact of the content

[0072] - Provide clear and accurate classification results

[0073] #Attention

[0074] When analyzing headlines, keep in mind:

[0075] - Titles may contain metaphors or puns that require in-depth understanding of their meaning

[0076] -Some titles may require more context to make an accurate judgment

[0077] - Remain impartial and objective, without being influenced by personal bias

[0078] - Comply with relevant laws, regulations and ethical standards

[0079] #OutputFormat

[0080] For each title, please give your judgment in the following format:

[0081] - "Safe": means the title does not contain harmful content

[0082] - "Harmful": indicates that the title contains harmful content

[0083] - "Unsure": Indicates that the title is ambiguous or requires more context to make an accurate judgment

[0084] Now, please analyze the following title: ...

[0085] Finish.

[0086] This application uses hash values ​​to achieve accurate matching of titles, uses vector retrieval to achieve highly approximate semantic judgment, and uses large models to achieve logical reasoning of complex texts. The three-layer detection process from fast to slow ensures that the detection effect is greatly improved while ensuring that the performance does not drop significantly.

[0087] It should be noted that the sensitive title library pre-established in this application mainly refers to the establishment of a title library for web pages with unsafe content, because the title is usually a short sentence and many similar titles may appear. Therefore, it cannot be compared only by text consistency, but can be compared in combination with semantics. Vector retrieval can quickly find the vector most similar to the query vector in a large-scale data set, which is very critical for content detection systems that need to process large amounts of data. Vector retrieval services are usually faster than large model reasoning because they utilize optimized similarity calculations and index structures. The data storage structure design of the sensitive title library is shown in Table 3 below:

[0088] Table 3

[0089]

[0090] In some embodiments, the response result is web page content, and the response result is tested to obtain a test result, which can be obtained by the following steps: split the web page content into paragraphs to obtain multiple paragraphs; count the number of occurrences of sensitive words in each paragraph based on a pre-established sensitive keyword library; sort each paragraph according to the number of occurrences of sensitive words to obtain a sorting result, and test the response result based on the sorting result to obtain a test result.

[0091] It should be noted that webpages usually contain a lot of text, and the number of words may even reach tens of thousands. Because the large model requires a lot of computation, in order to strike a balance between processing performance and accuracy, the key detection paragraphs are first matched with keywords, and then other paragraphs are detected in turn. During the process, the detection can be interrupted and a warning can be given as long as unsafe content is detected.

[0092] Specifically, Figure 4 As shown, the web page content is split into paragraphs, each paragraph is identified, each paragraph is compared with the phrases in the sensitive keyword library in turn, the number of times the sensitive keywords appear in each paragraph is counted, and each paragraph is comprehensively ranked to obtain the ranking result.

[0093] Among them, sorting each paragraph according to the number of times the sensitive word appears to obtain the sorting result can be achieved by the following steps: when the number of times the sensitive word appears is non-zero, sorting each paragraph according to the number of times the sensitive word appears to obtain the sorting result; when the number of times the sensitive word appears is zero, sorting each paragraph according to the order in which each paragraph appears in the webpage content to obtain the sorting result;

[0094] Specifically, when the number of occurrences of sensitive words is non-zero, the number of occurrences of sensitive keywords is used as the priority of content paragraph detection from high to low. The more times, the more likely it is that unsafe content will appear, and the higher the priority of the detection order. For paragraphs without sensitive keywords, they are sorted according to the order in which they appear in the web page to obtain the sorting results.

[0095] This application splits the web page content into multiple paragraphs to make subsequent processing more modular. Each paragraph can independently perform sensitive word statistics and analysis, thereby improving the overall processing efficiency. Sorting the web page content by paragraphs and displaying it based on the number of occurrences of sensitive words can make the detection results more intuitive and easy to understand.

[0096] Next, the response results are tested based on the sorting results to obtain the test results, which can be achieved through the following steps: assembling the target prompt words in sequence based on each paragraph in the sorting results; calling the large model engine to perform inference detection on the target prompt words to obtain the test results.

[0097] For example, continue as Figure 4 As shown, according to the sorted paragraphs, each paragraph is assembled into a prompt word, and the large model engine is called for reasoning to determine whether the content of this paragraph is safe. Among them, the results of the large model response are divided into three types: unsafe content, safe content, and uncertain. When any paragraph is detected to have unsafe content, it can be determined that the web page content is unsafe, then the detection is interrupted and a warning is given, and the web page URL address is updated to the URL black and white sample library. If all paragraphs are not detected to have unsafe content, the web page response is forwarded to the user end.

[0098] Specifically, the sensitive keyword library pre-established in this application is a sensitive keyword library established for bad-type phrases, such as gambling-related sensitive keywords "casino", "gambling", "betting", "lottery", "gambling", "poker", etc. The data storage structure design of the sensitive keyword library is shown in Table 4 below:

[0099] Table 4

[0100]

[0101]

[0102] Step S104: updating the sample library based on the detection result, wherein the detection result includes at least one of the following: a web page title detection result, a web page content detection result.

[0103] Among them, the web page title detection result is the result of the security assessment of the web page title, and the web page content detection result is the result of the security assessment of the web page content. This application can fully apply the detection results through the combination of sample library and knowledge base, accumulate sample information and continuously update and improve the knowledge base, so that the entire system has self-learning capabilities.

[0104] Optional, Figure 5 This is a schematic diagram of the overall architecture of this application, such as Figure 5As shown in the figure, in the uplink stage, the URL address of the Internet access request initiated by the user is detected. If it is detected to be unsafe, the Internet access is interrupted and a prompt is given, otherwise the Internet access request is forwarded to the corresponding website server. In the downlink stage, for the response result of the Internet access request, the web page title is first detected, and then the web page content is detected. If the content is unsafe, the Internet access is interrupted and a prompt is given, otherwise the response result is forwarded to the user. Through the three-level detection of the URL, title and content, a balance between efficiency and accuracy is achieved from easy to difficult.

[0105] In summary, the network detection method provided in the embodiment of the present application detects whether the URL address in the Internet access request is a safe URL through the URL list library detection method; when the URL address is detected to be a safe URL, the Internet access request is sent to the website server; the response result sent by the website server to the Internet access request is received, and the response result is detected to obtain the detection result, wherein the response result includes at least one of the following: web page title, web page content; the sample library is updated based on the detection result, wherein the detection result includes at least one of the following: web page title detection result, web page content detection result. Through this application, the problem of reduced accuracy of Internet access content detection in related technologies is solved. This case uses three-level detection of URLs, web page titles, and web page content. Thereby achieving the effect of improving the accuracy of Internet access content detection.

[0106] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0107] The embodiment of the present application also provides a network detection device. It should be noted that the network detection device of the embodiment of the present application can be used to execute the network detection method provided by the embodiment of the present application. The network detection device provided by the embodiment of the present application is introduced below.

[0108] Figure 6 is a schematic diagram of a network detection device 600 according to an embodiment of the present application. Figure 6 As shown, the device includes: a responding unit 601, a sending unit 602, a receiving unit 603, and an updating unit 604.

[0109] Specifically, the response unit 601 is used to respond to the Internet access request of the user terminal and detect whether the URL address in the Internet access request is a safe website through the URL list library detection method;

[0110] The sending unit 602 is used to send the Internet access request to the website server when the URL address is detected to be a safe website;

[0111] The receiving unit 603 is used to receive the response result of the Internet access request sent by the website server, detect the response result, and obtain the detection result, wherein the response result includes at least one of the following: web page title, web page content;

[0112] The updating unit 604 is used to update the sample library based on the detection result, wherein the detection result includes at least one of the following: a web page title detection result and a web page content detection result.

[0113] In summary, the network detection device provided in the embodiment of the present application is used to respond to the Internet access request of the user terminal through the response unit 601, and detect whether the URL address in the Internet access request is a safe website through the URL list library detection method; the sending unit 602 is used to send the Internet access request to the website server when the URL address is detected to be a safe website; the receiving unit 603 is used to receive the response result sent by the website server to the Internet access request, detect the response result, and obtain the detection result, wherein the response result includes at least one of the following: web page title, web page content; the updating unit 604 is used to update the sample library based on the detection result, wherein the detection result includes at least one of the following: web page title detection result, web page content detection result, which solves the problem of reduced accuracy of Internet access content detection in related technologies. This case uses three-level detection of URL, web page title, and web page content. And thus achieves the effect of improving the accuracy of Internet access content detection.

[0114] Optionally, in the network detection device provided in the embodiment of the present application, in response to the Internet access request of the user terminal, the response unit includes: a calling subunit, which is used to call a pre-established URL list library, wherein the URL list library includes: a URL blacklist and a URL whitelist; a first retrieval subunit, which is used to retrieve whether the URL address in the Internet access request exists in the URL list library by a complete match; a first judgment subunit, which is used to determine whether the URL address is a blacklist address if it exists, and if the URL address is a blacklist address, detect that the URL address is not a safe URL, and if the URL address is a whitelist address, detect that the URL address is a safe URL and allow access; the first judgment subunit is used to extract the root address of the URL address if it does not exist, and retrieve whether the root address exists in the URL list library by a prefix match, and if it exists, determine whether the root address is a blacklist address, and if the root address is a blacklist address, detect that the URL address is not a safe URL.

[0115] Optionally, in the network detection device provided in the embodiment of the present application, the response result is a web page title, and the receiving unit includes: an extraction subunit, used to extract title information in the web page title and calculate a hash value of the title information; a second retrieval subunit, used to retrieve whether the hash value exists in a pre-established sensitive title library; if so, the web page title is detected as an unsafe title; if not, the response result is detected by vector retrieval to obtain a detection result.

[0116] Optionally, in the network detection device provided in the embodiment of the present application, the second retrieval sub-unit includes: a conversion sub-unit, used to convert the web page title into a title vector by means of an embedded model; a retrieval sub-unit, used to retrieve whether the title vector exists in the sensitive title library. If so, the web page title is detected as an unsafe title; if not, the large model technology is called to perform inference detection on the web page title to obtain a detection result.

[0117] Optionally, in the network detection device provided in the embodiment of the present application, the response result is the web page content, and the receiving unit includes: a splitting subunit, used to split the web page content into paragraphs to obtain multiple paragraphs; a counting subunit, used to count the number of occurrences of sensitive words in each paragraph based on a pre-established sensitive keyword library; a sorting subunit, used to sort each paragraph according to the number of occurrences of sensitive words to obtain a sorting result, and detect the response result based on the sorting result to obtain a detection result.

[0118] Optionally, in the network detection device provided in the embodiment of the present application, the sorting subunit includes: a first sorting module, which is used to sort each paragraph according to the number of times the sensitive word appears when the number of times the sensitive word appears is non-zero, and obtain a sorting result; a second sorting module, which is used to sort each paragraph according to the order in which each paragraph appears in the web page content when the number of times the sensitive word appears is zero, and obtain a sorting result.

[0119] Optionally, in the network detection device provided in the embodiment of the present application, the sorting subunit includes: an assembling module, which is used to assemble the target prompt words in sequence based on each paragraph in the sorting result; and a calling module, which is used to call the large model engine to perform inference detection on the target prompt words to obtain the detection results.

[0120] The network detection device includes a processor and a memory. The above-mentioned response unit 601, sending unit 602, receiving unit 603, updating unit 604, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0121] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and network detection can be performed by adjusting kernel parameters.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0123] In the exemplary embodiment of the present application, a computer storage medium capable of implementing the above method is also provided. A program product capable of implementing the above method of the present specification is stored thereon. In some possible embodiments, various aspects of the present application can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of this specification, such as, the following steps can be performed: in response to the Internet access request of the user end, the URL address in the Internet access request is detected by the URL list library detection method. Whether it is a safe website; in the case where the URL address is detected to be a safe website, the Internet access request is sent to the website service end; receiving the response result sent by the website service end for the Internet access request, detecting the response result, and obtaining the detection result, wherein the response result includes at least one of the following: web page title, web page content; updating the sample library based on the detection result, wherein the detection result includes at least one of the following: web page title detection result, web page content detection result.

[0124] In an optional implementation: calling a pre-established URL list library, wherein the URL list library includes: a URL blacklist and a URL whitelist; searching whether the URL address in the Internet access request exists in the URL list library by a complete matching method; if it exists, determining whether the URL address is a blacklist address; if the URL address is a blacklist address, detecting that the URL address is not a safe URL; if the URL address is a whitelist address, detecting that the URL address is a safe URL, and allowing access; if it does not exist, extracting the root address of the URL address, searching whether the root address exists in the URL list library by a prefix matching method; if it exists, determining whether the root address is a blacklist address; if the root address is a blacklist address, detecting that the URL address is not a safe URL.

[0125] In an optional implementation: extract the title information in the web page title, and calculate the hash value of the title information; search whether the hash value exists in a pre-established sensitive title library; if it exists, detect that the web page title is an unsafe title; if it does not exist, detect the response result by vector retrieval to obtain a detection result.

[0126] In an optional implementation: converting the web page title into a title vector by means of an embedding model; searching whether the title vector exists in a sensitive title library; if so, detecting that the web page title is an unsafe title; if not, calling the large model technology to perform inference detection on the web page title to obtain a detection result.

[0127] In an optional implementation: split the web page content into paragraphs to obtain multiple paragraphs; count the number of times sensitive words appear in each paragraph based on a pre-established sensitive keyword library; sort each paragraph according to the number of times sensitive words appear to obtain a sorting result, and detect the response result based on the sorting result to obtain a detection result.

[0128] In an optional implementation: when the number of occurrences of sensitive words is non-zero, each paragraph is sorted according to the number of occurrences of the sensitive words to obtain a sorting result; when the number of occurrences of sensitive words is zero, each paragraph is sorted according to the order in which each paragraph appears in the web page content to obtain a sorting result.

[0129] In an optional implementation: each paragraph in the sorting result is sequentially assembled into a target prompt word; and a large model engine is called to perform reasoning detection on the target prompt word to obtain a detection result.

[0130] In an optional embodiment, the embodiment of the present application may also include a program product for implementing the above method, which may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0131] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0132] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0133] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0134] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0135] In addition, in an exemplary embodiment of the present application, an electronic device capable of implementing the above method is also provided.

[0136] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0137] Refer to the following Figure 7 hereinafter describes an electronic device 700 according to this embodiment of the present application. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0138] like Figure 7As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), and a display unit 740.

[0139] The storage unit stores program code, and the program code can be executed by the processing unit 710, so that the processing unit 710 executes the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present application. For example, the processing unit 710 can execute the following steps: responding to the Internet access request of the user end, detecting whether the URL address in the Internet access request is a safe URL through the URL list library detection method; in the case where the URL address is detected to be a safe URL, sending the Internet access request to the website server; receiving the response result sent by the website server for the Internet access request, detecting the response result, and obtaining the detection result, wherein the response result includes at least one of the following: web page title, web page content; updating the sample library based on the detection result, wherein the detection result includes at least one of the following: web page title detection result, web page content detection result.

[0140] In an optional implementation: calling a pre-established URL list library, wherein the URL list library includes: a URL blacklist and a URL whitelist; searching whether the URL address in the Internet access request exists in the URL list library by a complete matching method; if it exists, determining whether the URL address is a blacklist address; if the URL address is a blacklist address, detecting that the URL address is not a safe URL; if the URL address is a whitelist address, detecting that the URL address is a safe URL, and allowing access; if it does not exist, extracting the root address of the URL address, searching whether the root address exists in the URL list library by a prefix matching method; if it exists, determining whether the root address is a blacklist address; if the root address is a blacklist address, detecting that the URL address is not a safe URL.

[0141] In an optional implementation: extract the title information in the web page title, and calculate the hash value of the title information; search whether the hash value exists in a pre-established sensitive title library; if it exists, detect that the web page title is an unsafe title; if it does not exist, detect the response result by vector retrieval to obtain a detection result.

[0142] In an optional implementation: converting the web page title into a title vector by means of an embedding model; searching whether the title vector exists in a sensitive title library; if so, detecting that the web page title is an unsafe title; if not, calling the large model technology to perform inference detection on the web page title to obtain a detection result.

[0143] In an optional implementation: split the web page content into paragraphs to obtain multiple paragraphs; count the number of times sensitive words appear in each paragraph based on a pre-established sensitive keyword library; sort each paragraph according to the number of times sensitive words appear to obtain a sorting result, and detect the response result based on the sorting result to obtain a detection result.

[0144] In an optional implementation: when the number of occurrences of sensitive words is non-zero, each paragraph is sorted according to the number of occurrences of the sensitive words to obtain a sorting result; when the number of occurrences of sensitive words is zero, each paragraph is sorted according to the order in which each paragraph appears in the web page content to obtain a sorting result.

[0145] In an optional implementation: each paragraph in the sorting result is sequentially assembled into a target prompt word; and a large model engine is called to perform reasoning detection on the target prompt word to obtain a detection result.

[0146] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

[0147] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0148] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0149] The electronic device 700 may also communicate with one or more external devices 800 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0150] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0151] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0152] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application. The specification and embodiments are to be regarded as exemplary only, and the true scope and spirit of the present application are indicated by the claims.

Claims

1. A network detection method, characterized in that: include: In response to a request from a user to access the Internet, detecting whether the URL address in the request to access the Internet is a safe URL by using a URL list library detection method; When the URL address is detected to be a secure website, the Internet access request is sent to the website server; Receiving a response result to the Internet access request sent by the website server, detecting the response result to obtain a detection result, wherein the response result includes at least one of the following: a web page title and web page content; The sample library is updated based on the detection result, wherein the detection result includes at least one of the following: a web page title detection result and a web page content detection result.

2. The method according to claim 1, characterized in that In response to a user's Internet access request, detecting whether a URL address in the Internet access request is a safe URL by using a URL list library detection method includes: Calling a pre-established URL list library, wherein the URL list library includes: a URL blacklist and a URL whitelist; Searching whether the URL address in the Internet access request exists in the URL list library by a complete matching method; If it exists, it is determined whether the URL address is a blacklist address. If the URL address is a blacklist address, it is detected that the URL address is not a safe URL. If the URL address is a whitelist address, it is detected that the URL address is a safe URL and access is allowed. If it does not exist, extract the root address of the URL address, and retrieve whether the root address exists in the URL list library by prefix matching. If it does exist, determine whether the root address is a blacklist address. If the root address is a blacklist address, detect that the URL address is not a safe URL.

3. The method according to claim 1, characterized in that The response result is the webpage title, and the response result is tested to obtain a test result, including: Extracting title information from the webpage title and calculating a hash value of the title information; Retrieve whether the hash value exists in a pre-established sensitive title library; if so, detect that the web page title is an unsafe title; if not, detect the response result by vector retrieval to obtain a detection result.

4. The method according to claim 3, characterized in that The response result is detected by vector retrieval to obtain a detection result, including: Convert the webpage title into a title vector by means of an embedding model; It is retrieved whether the title vector exists in the sensitive title library. If it exists, the web page title is detected as an unsafe title. If it does not exist, the large model technology is called to perform reasoning detection on the web page title to obtain a detection result.

5. The method according to claim 1, characterized in that The response result is the webpage content, and the response result is tested to obtain a test result, including: Splitting the webpage content into paragraphs to obtain multiple paragraphs; Count the number of occurrences of sensitive words in each paragraph based on the pre-established sensitive keyword library; Each paragraph is sorted according to the number of times the sensitive word appears to obtain a sorting result, and the response result is tested based on the sorting result to obtain a test result.

6. The method according to claim 5, characterized in that Sorting each paragraph according to the number of occurrences of the sensitive words to obtain a sorting result, including: When the number of occurrences of the sensitive word is non-zero, sorting each paragraph according to the number of occurrences of the sensitive word to obtain a sorting result; When the number of occurrences of the sensitive word is zero, each paragraph is sorted according to the order in which each paragraph appears in the webpage content to obtain a sorting result.

7. The method according to claim 5, characterized in that The response result is tested based on the sorting result to obtain a test result, including: Assembling target prompt words in sequence based on each paragraph in the sorting result; The large model engine is called to perform reasoning detection on the target prompt word to obtain a detection result.

8. A network detection device, characterized in that: include: A response unit, used to respond to a user's Internet access request and detect whether the URL address in the Internet access request is a safe URL through a URL list library detection method; A sending unit, configured to send the Internet access request to a website server when detecting that the URL address is a secure website; A receiving unit, configured to receive a response result sent by the website server to the Internet access request, and detect the response result to obtain a detection result, wherein the response result includes at least one of the following: a web page title and web page content; An updating unit is used to update the sample library based on the detection result, wherein the detection result includes at least one of the following: a web page title detection result and a web page content detection result.

9. A computer-readable storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the network detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, a display device, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the network detection method described in any one of claims 1 to 7.

Images