Network resource request interception method and device, electronic equipment and medium
By dynamically modifying the block list during browser operation through a full-duplex communication channel, the problem of the inability to dynamically update block rules in existing technologies is solved, achieving efficient and secure network resource request blocking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING JINKANG NEW ENERGY VEHICLE CO LTD
- Filing Date
- 2026-02-13
- Publication Date
- 2026-05-12
AI Technical Summary
In existing technologies, network resource request interception rules cannot be dynamically modified and require a browser restart to take effect, which increases complexity and security risks, and also requires the additional deployment of proxy servers or the installation of root certificates.
By dynamically modifying the block list during browser operation through a full-duplex communication channel, the block list update command is directly sent to the browser, avoiding the need to restart the browser and deploy additional proxy servers.
It enables dynamic modification of interception rules, improving change efficiency, reducing complexity and security risks, and avoiding additional operation and maintenance costs.
Smart Images
Figure CN122027293A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of network request interception, specifically relating to a method, apparatus, electronic device, and medium for intercepting network resource requests. Background Technology
[0002] In application scenarios such as web (network) automated testing, web crawler protection, ad filtering, and privacy compliance, it is often necessary to prevent the browser from loading specific network resources, that is, to intercept specific network resource requests in order to achieve filtering or protection of specific content.
[0003] In existing technologies, rules for blocking network resource requests are typically loaded when the browser starts up and cannot be dynamically modified during browser operation. Furthermore, changes to these rules require a browser restart to take effect, reducing the efficiency of rule modification. Moreover, existing technologies require the additional deployment of proxy servers, installation of root certificates, or approval from app stores to effectively block network resource requests, increasing the complexity of blocking and introducing additional security risks and operational costs. Summary of the Invention
[0004] The purpose of this application is to provide a method, apparatus, electronic device, and medium for intercepting network resource requests.
[0005] In a first aspect, embodiments of this application provide a method for intercepting network resource requests, applied to a network resource interception system, wherein the network resource interception system communicates with a browser via a full-duplex communication channel, and the method includes: Receive the identifier of the resource to be intercepted from the user input; The identifier of the resource to be intercepted is adaptively encapsulated to obtain the target interception list; Construct an intercept list update instruction containing the target intercept list based on the target intercept list; Obtain the network control function status of the browser; If the network control function status indicates that the browser's network control function is enabled, the intercept list update instruction is sent to the browser through the full-duplex communication channel, and the browser is used to intercept network resource requests according to the target intercept list.
[0006] Optionally, the step of adaptively encapsulating the identifier of the resource to be intercepted to obtain the target interception list includes: Obtain the identifier type of the resource identifier to be intercepted; the identifier type includes one of null, string, list, tuple and other types; the other types are identifier types other than null, string, list and tuple; If the identifier type is null, then the identifier of the resource to be intercepted is encapsulated into an empty list, and the empty list is used as the target interception list; If the identifier type is the string, then the identifier of the resource to be intercepted is encapsulated into a single-element list, and the single-element list is used as the target interception list; If the identifier type is the list or the tuple, then the identifier of the resource to be intercepted is used as the target interception list; If the identifier type is one of the other types, then a type error message is generated based on the identifier of the resource to be intercepted, and the type error message is sent to the user.
[0007] Optionally, the step of constructing an update instruction for a target interception list containing the target interception list includes: The target interception list is processed by deduplication, sorting and security verification of several resource identifiers to be intercepted in the target interception list to obtain the processed target interception list. If the number of resource identifiers to be intercepted in the processed target interception list is greater than a preset threshold, then the target interception list is split into several target interception sub-lists, and an interception list update instruction is constructed according to the several target interception sub-lists respectively; If the number of resource identifiers to be intercepted in the target interception list is less than or equal to the number threshold, then an interception list update instruction containing the target interception list is constructed based on the target interception list.
[0008] Optionally, the method further includes: If the initial state of the network control function indicates that the browser's network control function is disabled, then a network control domain enable instruction is generated; The network control domain enable command is sent to the browser, and the browser is used to enable network control functions according to the network control domain enable command.
[0009] Optionally, after sending the network control domain enable command, the method further includes: If the intercept list update command fails to be sent, a network control domain shutdown command is generated. The network control domain shutdown command is sent to the browser, and the browser is used to disable the network control function according to the network control domain shutdown command.
[0010] Optionally, after sending the block list update instruction to the browser, the method further includes: Generate an intercept list update event, and associate the target intercept list with the intercept list update event and save it.
[0011] Optionally, the method further includes: Receive a request blocking event sent by the browser; wherein the request blocking event is generated by the browser when it successfully blocks a network resource request that matches the target blocking list; Based on the received request blocking event, generate and record a network resource blocking log.
[0012] Secondly, embodiments of this application provide a network resource request interception device, applied to a network resource interception system, wherein the network resource interception system communicates with a browser via a full-duplex communication channel, and the device includes: The identifier acquisition module is used to receive the identifier of the resource to be intercepted input by the user; The interception list acquisition module is used to adaptively encapsulate the identifier of the resource to be intercepted to obtain the target interception list; The update instruction acquisition module is used to construct an interception list update instruction containing the target interception list based on the target interception list; A control function status acquisition module is used to acquire the network control function status of the browser; The request interception module is used to send the interception list update instruction to the browser through the full-duplex communication channel if the network control function status indicates that the browser's network control function is enabled. The browser is used to intercept network resource requests according to the target interception list.
[0013] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and capable of running on the processor, wherein the program or instructions, when executed by the processor, implement the method described above.
[0014] Fourthly, a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described above.
[0015] The embodiments of this application have the following advantages: In this embodiment, the system can receive user-inputted identifiers of resources to be intercepted, adaptively encapsulate these identifiers to obtain a target interception list, construct an interception list update instruction containing the target interception list, and obtain the browser's network control function status. If the network control function status indicates that the browser's network control function is enabled, the interception list update instruction is sent to the browser via a full-duplex communication channel. The browser then uses this information to intercept network resource requests based on the target interception list. This application sends the interception list update instruction to the browser via a full-duplex communication channel, allowing for dynamic modification of the target interception list during browser operation, thus improving the efficiency of rule changes. Furthermore, this application can directly send the interception list update instruction to the browser without requiring additional deployment of proxy servers, installation of root certificates, or app store approval, thereby reducing the complexity of network resource request interception and avoiding additional security risks and operational costs. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0017] Figure 1 This is a flowchart illustrating the steps of a network resource request interception method according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the steps of a network resource request interception method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a network resource request interception device provided in an embodiment of this application. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of this application to enable readers to better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and updates based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.
[0019] In application scenarios such as web automation testing, web crawler protection, ad filtering, and privacy compliance, it is often necessary to prevent the browser from loading specific network resources, that is, to intercept specific network resource requests in order to filter or protect specific content.
[0020] The existing technologies mainly include the following solutions: The initial blocking rule configuration is achieved by setting parameters such as --host-rules (host rules) or --host-resolver-rules (host resolution rules) when the browser starts. This solution only loads rules during the browser startup phase, and the rules cannot be dynamically modified or updated during browser operation. Any adjustment to the blocking strategy must be made by restarting the browser process to take effect.
[0021] This approach involves deploying independent proxy servers, such as mitmproxy (Man-In-The-Middle Proxy) or Fiddler (HTTP debugging proxy), to intercept requests. However, this solution requires introducing additional proxy nodes into the existing network architecture, increasing system topology complexity. Furthermore, when handling HTTPS (Hypertext Transfer Protocol Secure) requests, the client must import the proxy service's self-signed root certificate, adding not only to the configuration burden but also significant risks related to certificate trust and security compliance.
[0022] Browser extension platforms, such as Chrome Extensions, utilize the webRequest API (web request blocking interface) to implement request blocking. This approach requires users to pre-install the extension and grant the appropriate permissions; it is also subject to strict limitations imposed by browser extension specifications, such as Manifest V3, resulting in significant limitations in terms of the maximum number of usable rules and the real-time updates of rules.
[0023] Some testing frameworks, such as Selenium (a web automation testing framework), provide a preliminary encapsulation of the CDP protocol (Chrome DevTools Protocol), supporting functions such as calling Network.setBlockedURLs (a method for intercepting URLs in network settings). However, there are obvious defects in the implementation: the interface only accepts standard list type parameters and lacks the ability to adaptively convert to other data types such as strings and tuples; it lacks runtime parameter validation and robust exception handling mechanisms, and when users pass in illegal types such as dictionaries and sets, it is easy to cause command execution failure and throw runtime exceptions that are difficult to troubleshoot; at the same time, it does not provide a unified high-level abstraction encapsulation, and different business parties still need to repeatedly implement the basic code for underlying communication and state management.
[0024] Therefore, this application provides a method, apparatus, electronic device, and medium for intercepting network resource requests. By sending an update instruction to the intercept list to a browser through a full-duplex communication channel, the target intercept list can be dynamically modified during browser operation, achieving dynamic modification of interception rules and improving the efficiency of rule changes. Furthermore, this application can directly send the update instruction to the browser without requiring additional deployment of a proxy server, installation of root certificates, or approval from app stores, thus reducing the complexity of network resource request interception and avoiding additional security risks and operational costs.
[0025] Reference Figure 1 The diagram illustrates a flowchart of a network resource request interception method according to an embodiment of this application.
[0026] In this embodiment, the network resource request interception method can be applied to a network resource interception system, and the network resource interception system communicates with the browser through a full-duplex communication channel.
[0027] Among them, the network resource interception system can be a software entity independent of the browser, serving as a logical control center to implement a complete control process of user command reception, data standardization, protocol encapsulation, status management, and command distribution.
[0028] A network resource blocking system can include a script layer, a CDP client layer, and a URL (Uniform Resource Locator) blocking layer.
[0029] The script layer can be used to provide multi-language support, including SDKs (Software Development Kits) for Python (Python programming language), Java (Java programming language), and C# (C Sharp programming language). The script layer provides users with a flexible interface, enabling them to write and control interception rules using familiar programming languages, thereby lowering the development threshold and improving development efficiency.
[0030] The CDP client layer is responsible for establishing a WebSocket connection with the browser's DevTools port. Through this layer, the network resource interception system can communicate with the browser in real time, using the Chrome DevTools Protocol (CDP) to achieve fine-grained control over browser behavior, such as intercepting specific URLs or dynamically updated rules.
[0031] The URL blocking layer can be the core functional layer of a network resource interception system. Processes such as parameter adaptation, hot updates, and transaction rollback rely on the URL blocking layer for implementation. The URL blocking layer can ensure that the network resource interception system can dynamically adjust the interception rules according to different scenarios and needs.
[0032] In practical implementations, the URL blocking layer can be implemented using specific software class instances. For example, in a Python implementation, the core blocking logic can be encapsulated by defining a NetworkBlocker class, as shown below: class NetworkBlocker: def __init__(self, owner): self._owner = owner # owner is the CDP client instance The `owner` parameter is a reference to the CDP client layer instance. This dependency injection design decouples the URL blocking layer from the underlying communication layer. The `NetworkBlocker` class further includes methods such as `blocked_urls` (a list of blocked URLs) to implement the aforementioned parameter adaptation, hot update, and transaction rollback processes.
[0033] The browser can be a local or remote instance of Chrome (Google Chrome), Edge (Microsoft Edge browser), or Chromium (Chromium open-source browser project). These browser instances can serve as the execution environment for interception operations, enabling the interception of specific URLs according to system instructions and achieving fine-grained management of network resources.
[0034] A full-duplex communication channel can be a persistent data transmission link established based on standard network protocols, used to provide bidirectional parallel data exchange capabilities, ensuring real-time synchronous transmission of commands and events between the system and the browser. In specific implementations, a full-duplex communication channel can be a persistent bidirectional communication link established based on the WebSocket protocol.
[0035] The method may specifically include the following steps: Step 101: Receive the identifier of the resource to be intercepted input by the user.
[0036] In this embodiment, a user-inputted identifier for a resource to be intercepted can be received. This identifier can specify the location information of the network resource to be intercepted. The identifier type can be a string, a list, or a tuple, and it can represent a single resource address or a set of resource addresses. In a specific implementation, the identifier can be represented as URLs.
[0037] Step 102: Perform adaptive encapsulation processing on the resource identifier to be intercepted to obtain the target interception list.
[0038] In this embodiment, the identifiers of the resources to be intercepted can be adaptively encapsulated to obtain a target interception list. The target interception list can refer to a set of structured interception rules after system standardization processing, and the target interception list can contain zero or one or more normalized resource identifiers.
[0039] In practical implementation, input data can be uniformly encapsulated into a predefined list structure. The script layer of the network resource interception system can execute corresponding conversion logic based on the identifier type of the input resource identifier to be intercepted, ultimately obtaining the target interception list. This ensures that input data of different formats can be converted into a standardized data structure that conforms to protocol requirements, providing a consistent data foundation for subsequent instruction construction and protocol transmission.
[0040] Step 103: Construct an interception list update instruction containing the target interception list based on the target interception list.
[0041] In this embodiment of the application, an interception list update instruction containing the target interception list can be constructed based on the target interception list.
[0042] The block list update command can be a structured command message conforming to the CDP protocol specification, used to transmit a standardized target block list to the browser. This command carries complete rule update information and is sent to the browser via a full-duplex communication channel, directly triggering the real-time update and activation of the browser's internal block rules.
[0043] Step 104: Obtain the network control function status of the browser.
[0044] In this embodiment, the browser's network control function status can be obtained. The network control function status can be a Boolean state indicating whether the browser's Network domain (network control function domain) is active, used to indicate whether the browser is ready to receive network control commands. This status is obtained by querying the browser's internal operating state, and its value directly determines whether a network control domain activation command needs to be sent in advance, which is a prerequisite for ensuring the correct execution of subsequent interception operations.
[0045] Step 105: If the network control function status indicates that the browser's network control function is enabled, the interception list update instruction is sent to the browser through the full-duplex communication channel. The browser is used to intercept network resource requests according to the target interception list.
[0046] In this embodiment of the application, if the network control function status indicates that the browser's network control function is enabled, the intercept list update instruction is sent to the browser through the full-duplex communication channel. This instruction carries the standardized target intercept list. After receiving the instruction, the browser immediately parses the target intercept list into internal intercept rules and performs real-time matching and interception processing on subsequent network resource requests based on the internal intercept rules.
[0047] In the actual implementation, the system calls the Network.enable command before sending the setBlockedURLs command.
[0048] In this context, "Network" refers to the network control domain in the Chrome DevTools Protocol.
[0049] "Enable" indicates an operation command that enables the functionality of this network control domain.
[0050] `setBlockedURLs` is the specific command to set the list of blocked URLs.
[0051] In this context, "Domain" specifically refers to the functional domain module in the CDP protocol.
[0052] If the domain is detected to be in an enabled state, the enabling step is skipped and subsequent operations are executed directly. This conditional judgment mechanism effectively reduces the number of network round trips and improves the overall communication efficiency of the system by avoiding redundant command interactions.
[0053] In practice, the following code can be used for hot updates: self._owner._run_cdp('Network.setBlockedURLs', urls=urls) Hot update is a dynamic rule update mechanism used to modify the list of blocking rules in real time while the browser is running. This operation is achieved by calling the Network.setBlockedURLs protocol directive, passing the new target blocking list as a parameter to the browser. The updated rules take effect immediately without restarting the browser process, enabling real-time synchronization and dynamic adjustment of blocking rules.
[0054] At any point during browser runtime, the user can call the blocked_urls interface again to pass in a new list of rules.
[0055] Browser runtime refers to the complete lifecycle of a browser process from startup to termination.
[0056] blocked_urls is a system-provided configuration interface for blocking resources.
[0057] Network.setBlockedURLs is a method in the CDP protocol for updating the block list.
[0058] Overriding old rules means that the newly submitted rule will completely replace all previously set rules.
[0059] This application achieves dynamic refreshing of blocking rules by sending updated protocol instructions in real time, so that new rules can take effect immediately without restarting the browser process.
[0060] In this embodiment, the system can receive user-inputted identifiers of resources to be intercepted, adaptively encapsulate these identifiers to obtain a target interception list, construct an interception list update instruction containing the target interception list, and obtain the browser's network control function status. If the network control function status indicates that the browser's network control function is enabled, the interception list update instruction is sent to the browser via a full-duplex communication channel. The browser then uses this information to intercept network resource requests based on the target interception list. This application sends the interception list update instruction to the browser via a full-duplex communication channel, allowing for dynamic modification of the target interception list during browser operation, thus improving the efficiency of rule changes. Furthermore, this application can directly send the interception list update instruction to the browser without requiring additional deployment of proxy servers, installation of root certificates, or app store approval, thereby reducing the complexity of network resource request interception and avoiding additional security risks and operational costs.
[0061] In one optional embodiment of this application, the step of adaptively encapsulating the identifier of the resource to be intercepted to obtain the target interception list includes: S11, Obtain the identifier type of the resource identifier to be intercepted; the identifier type includes one of null, string, list, tuple and other types; the other types are identifier types other than null, string, list and tuple; S12, if the identifier type is the null value, then the identifier of the resource to be intercepted is encapsulated into an empty list, and the empty list is used as the target interception list; S13, if the identifier type is the string, then the identifier of the resource to be intercepted is encapsulated into a single-element list, and the single-element list is used as the target interception list; S14, if the identifier type is the list or the tuple, then the identifier of the resource to be intercepted is used as the target interception list; S15, if the identifier type is one of the other types, then generate a type error message based on the identifier of the resource to be intercepted, and send the type error message to the user.
[0062] In this embodiment, the identifier type of the resource to be intercepted can be obtained; the identifier type includes one of null, string, list, tuple, and other types; other types are identifier types other than null, string, list, and tuple. In specific implementations, other types may include, but are not limited to: dictionaries, sets, integers, floating-point numbers, booleans, functions, class instances, generators, byte strings, and other non-serialized or non-string data structures.
[0063] In this embodiment, if the identifier type is null, the identifier of the resource to be intercepted is encapsulated into an empty list, and the empty list is used as the target interception list. A null value can be a special value indicating that data does not exist, and can be used to indicate that a variable is uninitialized or empty. An empty list can be a data set that does not contain any elements. As a standardized representation of an empty set, an empty list can be used to clear or reset the target interception list.
[0064] In the specific implementation, during parameter processing, when the value of the URLs to be intercepted is None (null pointer) or an empty container, that is, the identifier type is null, the system automatically corrects it to an empty list [].
[0065] In this embodiment, if the identifier type is a string, the identifier of the resource to be intercepted is encapsulated into a single-element list, and this single-element list is used as the target interception list. The single-element list can also be in the form of a tuple. A string can be a serialized data structure composed of characters, used to represent the textual location information of a single resource to be intercepted. A single-element list can be a standard sequence structure containing only a single element. In specific implementations, this structure can use either a list or a tuple data structure; both can functionally achieve the purpose of encapsulating a single resource identifier into a standardized sequence. Choosing either a list or a tuple as the implementation method does not affect its core functionality; both ensure that the single resource identifier is correctly encapsulated and subsequently processed uniformly.
[0066] During parameter processing, when the system detects that the variable urls is of type str (string), it automatically converts the string into a single-element tuple and uses the single element (tuple) as the target interception list.
[0067] In this embodiment, if the identifier type is a list or a tuple, the identifiers of the resources to be intercepted are used as the target interception list. A list can be a collection of elements arranged in a specific order; as a mutable sequence data structure, it can be used to store a set of identifiers of resources to be intercepted that need to be dynamically modified. A tuple can be an immutable ordered sequence of elements; as a lightweight data structure, it can be used to store a set of identifiers of resources to be intercepted that do not need to be modified.
[0068] In this embodiment of the application, if the identifier type is other types, a type error message is generated based on the identifier of the resource to be intercepted, and the type error message is sent to the user.
[0069] In the actual implementation, if it is detected that the type of the variable urls is neither a string, a list, nor a tuple, the system will immediately throw a ValueError (value error exception) wrapped by i18n (internationalization).
[0070] Here, urls represents the input variable that stores the addresses of the resources to be intercepted.
[0071] A list is a data type in Python that represents a mutable sequence.
[0072] A tuple is a data type in Python that represents an immutable sequence.
[0073] i18n is an abbreviation for internationalization.
[0074] ValueError is an exception type in Python that represents a value error.
[0075] This type of error message can include the type check result "Current type is xxx" and the allowed input type range "str, list, tuple", while also carrying the actual value of the parameter for subsequent log auditing and analysis.
[0076] In a practical implementation, the following code can be used to achieve adaptive parameter encapsulation: def blocked_urls(self, urls): #Parameter Adaptation if not URLs: urls = [] elif isinstance(urls, str): urls = (urls,) if not isinstance(urls, (list, tuple)): raise ValueError( _S._lang.join( _S._lang.INCORRECT_TYPE_, 'urls', ALLOW_TYPE='str,list,tuple',CURR_VAL=type(urls).__name__ ) ) This application achieves unified processing and standardized output for different input types by adaptively encapsulating the identifier of the resource to be intercepted, thereby improving flexibility and compatibility. Specific advantages include: support for multiple input types such as null values, strings, lists, and tuples, automatically converting them into a standardized target interception list to ensure adaptability to different inputs; enhanced robustness and security of the network resource interception system through type checking and error message mechanisms, avoiding anomalies caused by illegal input; and simplification of subsequent processing logic by encapsulating strings in the form of single-element lists or tuples, improving code maintainability and scalability.
[0077] In an optional embodiment of this application, the step of constructing an update instruction for an interception list containing the target interception list includes: S21, perform deduplication, sorting and security verification on a number of the resource identifiers to be intercepted in the target interception list to obtain the processed target interception list; S22, if the number of resource identifiers to be intercepted in the processed target interception list is greater than a preset number threshold, then the target interception list is split into several target interception sub-lists, and an interception list update instruction is constructed according to the several target interception sub-lists respectively; S23, if the number of resource identifiers to be intercepted in the target interception list is less than or equal to the number threshold, then construct an interception list update instruction containing the target interception list based on the target interception list.
[0078] In this embodiment of the application, several resource identifiers to be intercepted in the target interception list can be deduplicated, sorted, and security verified to obtain the processed target interception list.
[0079] In practice, the blocking list can be deduplicated and sorted to reduce the size of CDP messages.
[0080] During the data preprocessing stage, deduplication and sorting are achieved by executing the statement urls = list({u.strip() for u in urls ifu.strip()}).
[0081] Here, urls represents a variable that stores the addresses of the resources to be intercepted.
[0082] The `list()` function in Python is a constructor that converts an iterable object into a list.
[0083] {} is the syntax symbol for representing a set in Python.
[0084] u.strip() means performing the operation of removing leading and trailing whitespace characters from the string u.
[0085] The `for u in urls` loop syntax iterates through each element in the URLs.
[0086] `if u.strip()` is a conditional statement used to filter non-empty strings.
[0087] This statement automatically removes duplicates and empty strings using set comprehensions and converts the result into an ordered list, thus achieving data standardization.
[0088] In the specific implementation, before executing the setBlockedURLs operation, the system performs a regular expression pre-compilation verification process for each URL (Uniform Resource Locator).
[0089] The `setBlockedURLs` command in the Chrome DevTools Protocol sets a list of URLs to be blocked.
[0090] The URL refers to a Uniform Resource Locator, which is a list of resource identifiers to be intercepted in the target interception list of this application.
[0091] Pre-compiled regular expressions mean that pre-compiled regular expression patterns are used for matching and validation.
[0092] scheme refers to the protocol identifier in the URL (such as http / https / file, etc.).
[0093] file: / / dev / zero is a special device file path in Unix-like systems.
[0094] By rejecting illegal protocol schemes (such as file: / / and other local file protocols), the security risks of bypassing the blocking rules through unconventional protocols are effectively prevented.
[0095] In this embodiment, if the number of resource identifiers to be intercepted in the processed target interception list exceeds a preset threshold, the target interception list is split into several target interception sub-lists, and an interception list update instruction is constructed based on each of the several target interception sub-lists. The threshold can be set according to actual conditions.
[0096] In practical implementation, large list fragmented transmission can be an adaptive data transmission mechanism based on a length threshold. When the number of URLs in the target interception list exceeds a preset threshold (e.g., 500), the list can be automatically divided into multiple appropriately sized sub-list batches and transmitted in stages via WebSocket connections. By controlling the data size of each transmission, transmission anomalies caused by excessively large WebSocket data frames are effectively avoided, ensuring that large-scale interception rules can be delivered to the browser stably and reliably.
[0097] If the number of resource identifiers to be intercepted in the target interception list is less than or equal to the number threshold, then an interception list update instruction containing the target interception list is constructed based on the target interception list.
[0098] In the implementation, when the number of resource identifiers in the target interception list does not exceed a preset threshold, a single update command can be directly constructed based on the complete interception list. This command contains a target interception list of all currently intercepted resource identifiers, and the transmission and update of the entire target interception list can be completed in a single interaction, achieving optimal transmission efficiency while ensuring functional integrity.
[0099] This application's embodiments ensure the accuracy and security of the target interception list through deduplication, sorting, and security verification. Simultaneously, a large list fragmentation transmission mechanism addresses the efficiency and stability issues of transmitting large-scale interception rules. Deduplication and sorting reduce redundant data and optimize transmission efficiency; security verification prevents illegal protocols from bypassing interception rules, enhancing security; the large list fragmentation transmission mechanism dynamically splits the target interception list based on a quantity threshold, avoiding WebSocket transmission anomalies and ensuring stable transmission of large-scale interception rules; for small-scale interception lists, a single update instruction is directly constructed, simplifying the transmission process and improving efficiency.
[0100] In one optional embodiment of this application, the method further includes the following steps: S31, if the initial state of the network control function indicates that the browser's network control function is disabled, then a network control domain enable instruction is generated; S32, the network control domain enable command is sent to the browser, and the browser is used to enable network control function according to the network control domain enable command.
[0101] In this embodiment, if the initial state of the network control function indicates that the browser's network control function is disabled, a network control domain enable command is generated and sent to the browser. The browser then enables the network control function according to the network control domain enable command. The network control domain enable command can be a function domain activation command conforming to the CDP protocol specification, used to send a protocol command to the browser to enable the Network control domain.
[0102] The Network Control Domain Disable command can be a functional domain deactivation command conforming to the CDP protocol specification, used to send a protocol instruction to the browser to disable the Network control domain. This command is used to proactively disable the browser's network traffic monitoring and interception functions when needed, or as part of a state rollback mechanism to restore the browser's Network domain to its inactive initial state.
[0103] In practical implementation, transaction rollback can be achieved in the following ways: #Transactional enable if not self._owner._domain_enabled.get('Network'): self._owner._run_cdp('Network.enable') self._owner._domain_enabled['Network'] = True As shown in the code, the system maintains the enabled status of Network domains through the internal status flag `_domain_enabled`. If the status check indicates that the domain is not yet enabled, the `Network.enable` directive is invoked via the CDP client, and the internal status record is updated upon success, thus avoiding unnecessary duplicate calls and effectively reducing communication round-trip overhead. This process ensures that subsequent directives such as `setBlockedURLs` can be executed in the correct browser environment.
[0104] This application embodiment detects the initial state of the browser's network control function and dynamically generates and sends a network control domain activation command, ensuring that the browser can promptly enable network control functions when needed. It can automatically detect the initial state of the network control function, avoiding tedious manual configuration. Through a network control domain activation command conforming to the CDP protocol specification, it achieves precise activation of the function domain, ensuring that the browser can correctly respond to interception requests and dynamically enable network control functions. This improves the flexibility and response speed of the network resource interception system, while reducing the risk of human error and enhancing the stability and security of the network resource interception system.
[0105] In an optional embodiment of this application, after sending the network control domain enable command, the method further includes the following steps: S41, if the intercept list update instruction fails to be sent, a network control domain shutdown instruction is generated; S42, the network control domain shutdown command is sent to the browser, and the browser is used to shut down the network control function according to the network control domain shutdown command.
[0106] In this embodiment of the application, if the intercept list update instruction fails to be sent, a network control domain shutdown instruction is generated and sent to the browser. The browser is used to disable the network control function according to the network control domain shutdown instruction.
[0107] If subsequent setBlockedURLs operations fail, a rollback operation can be automatically performed to restore the Network domain state, ensuring that no side effects occur on the browser side.
[0108] Among them, `setBlockedURLs` is a command in the Chrome DevTools Protocol used to set a list of blocked URLs. Network refers to the network control function domain in the CDP protocol.
[0109] Rollback means restoring the network control function state to its original state before the operation was executed.
[0110] Side effects refer to unexpected state changes or resource residues on the browser side.
[0111] This application ensures that the Network domain state on the browser side remains consistent with that before the operation by automatically triggering a reverse state recovery process when the operation fails, thus maintaining the stability and consistency of the browser environment.
[0112] This application automatically generates and sends a network control domain shutdown command when the intercept list update command fails to be sent, ensuring that the browser can promptly disable network control functions and avoid side effects caused by operation failure. Through an automatic rollback mechanism, it ensures that the Network domain state on the browser side remains consistent with that before the operation, maintaining the stability and consistency of the browser environment. It quickly restores the network control function state in the event of operation failure, reducing the risk of unexpected state changes or resource residue. Dynamically managing the enabling and disabling of network control functions improves fault tolerance and security, ensuring that the browser continues to operate stably even under abnormal conditions.
[0113] In an optional embodiment of this application, after sending the block list update instruction to the browser, the method further includes the following steps: S51, Generate an intercept list update event, and associate and save the target intercept list with the intercept list update event.
[0114] In this embodiment, after sending the block list update instruction to the browser, a block list update event can be generated, and the target block list can be associated with and saved with the block list update event. The block list update event can refer to a signal or message generated within the network resource interception system after the target block list has been successfully modified. The block list update event can include an event type (e.g., 'blocked_urls.updated') and the new target block list. The block list update event can be used to notify other layers of the network resource interception system that the block list has just changed. This enables real-time auditing, logging, or integration with other systems.
[0115] In practical implementation, callback auditing can be achieved in the following ways: # Callback Audit if self._owner._audit_callback: self._owner._audit_callback('blocked_urls.updated', urls) Here, self._owner._audit_callback is an optional callback function reference.
[0116] The conditional check verifies whether a callback function has been configured. If the callback function exists, two parameters are passed: 'blocked_urls.updated' is the rule update event identifier; urls is the latest target interception list; This application uses an event-driven approach to notify external systems of rule changes, thereby enabling the tracking and recording of operational traces.
[0117] This application embodiment supports the tracking and recording of network resource interception system's interception list update operations through callback auditing function, which improves the auditability and security of the network resource interception system. It also links and saves the target interception list with the update event, which facilitates subsequent log analysis and operation trace tracking, enhances the maintainability and traceability of the network resource interception system, and ensures the controllability and compliance of interception rule changes.
[0118] In one optional embodiment of this application, the method further includes the following steps: S61, receive a request blocking event sent by the browser; wherein the request blocking event is generated by the browser when it successfully blocks a network resource request that matches the target blocking list; S62, Based on the received request blocking event, generate and record a network resource blocking log.
[0119] In this embodiment of the application, a request blocking event sent by a browser can be received. The request blocking event is generated by the browser when it successfully blocks a network resource request that matches the target blocking list. Based on the received request blocking event, a network resource blocking log is generated and recorded.
[0120] A request blocking event can be a protocol notification message actively pushed by the browser. It is automatically generated when the browser successfully blocks a network resource request that matches the target blocking list and is used to report the specific blocking execution status to the network resource blocking system.
[0121] Network resource blocking logs can be a structured collection of event records. The network resource blocking system generates corresponding log entries based on the received request blocking events, recording detailed information about the blocked resources (such as URL, blocking timestamp, etc.), providing data support for security auditing and operation monitoring.
[0122] In its implementation, the network resource interception system asynchronously receives request blocking events sent by the browser through a full-duplex communication channel. After parsing the event content, it extracts key parameters (such as the blocked URL and trigger time), and finally generates standardized blocking log records, which are persistently stored in a designated medium. This mechanism enables end-to-end tracing of the interception operation, providing data for subsequent security analysis and rule optimization.
[0123] This application uses browser-initiated request blocking events to obtain real-time interception execution status, ensuring the transparency and traceability of interception operations. It generates structured blocking logs based on events, recording detailed information about the blocked resources, providing data support for security auditing and operational monitoring. Events are received asynchronously through a full-duplex communication channel, and key parameters are parsed and extracted to ensure the accuracy and completeness of log records. The persistent storage of blocking logs provides data for subsequent security analysis and rule optimization, improving the security and maintainability of the network resource interception system.
[0124] In the specific implementation, refer to Figure 2 The diagram illustrates the steps of a network resource request interception method according to an embodiment of this application. The specific process is as follows: First, it can receive the identifier of the resource to be intercepted from the user's input.
[0125] Secondly, the identifiers of the resources to be intercepted can be adaptively encapsulated to obtain the target interception list. Specifically, if the identifier type is null, the identifiers of the resources to be intercepted are encapsulated into an empty list, and the empty list is used as the target interception list. If the identifier type is string, the identifier of the resource to be intercepted is encapsulated into a single-element list, and the single-element list is used as the target interception list; If the identifier type is a list or tuple, then the identifier of the resource to be intercepted will be used as the target interception list. If the identifier type is other types, a type error message is generated based on the identifier of the resource to be intercepted, and the type error message is sent to the user. In this case, the process ends.
[0126] Then, the CDP message volume can be reduced by deduplicating and sorting the identifiers of several resources to be intercepted in the target interception list.
[0127] If the network control function is initially disabled, a network control domain enable command is generated and sent to the browser. The browser then uses the network control domain enable command to enable the network control function.
[0128] If network control is enabled initially, security checks are performed on several resource identifiers in the target block list. If the security check fails, rule updates are terminated, meaning no block list update command is sent, and the process ends.
[0129] If the number of resource identifiers to be intercepted in the processed target interception list exceeds a preset threshold, the target interception list is split into several target interception sublists, and interception list update instructions are constructed according to the several target interception sublists to achieve fragmented transmission.
[0130] If the number of resource identifiers to be intercepted in the target interception list is less than or equal to the number threshold, then an interception list update instruction containing the target interception list is constructed based on the target interception list, and a single transmission is achieved.
[0131] If the block list update fails, meaning the block list update command fails to send, a network control domain shutdown command is generated and sent to the browser. The browser then disables the network control function based on this command. In other words, in the event of a block list update failure, the network control function's state is rolled back to its initial state, and the process ends.
[0132] If the intercept list is updated successfully and the callback function is enabled, an intercept list update event is generated, and the target intercept list is associated with and saved with the intercept list update event.
[0133] If the block list is updated successfully and the callback function is not enabled, then determine whether blocking logs are enabled.
[0134] If blocking logs are enabled, request blocking events can be received from the browser. These events are generated by the browser when it successfully blocks a network resource request that matches the target blocking list. Based on the received request blocking events, network resource blocking logs are generated and recorded.
[0135] If the blocking log is turned off, the process ends in this case.
[0136] Reference Figure 3 This illustration shows a schematic diagram of a network resource request interception device according to an embodiment of this application, applied to a network resource interception system. The network resource interception system communicates with a browser via a full-duplex communication channel. The device includes: The identifier acquisition module 301 is used to receive the identifier of the resource to be intercepted input by the user; The interception list acquisition module 302 is used to perform adaptive encapsulation processing on the identifier of the resource to be intercepted to obtain the target interception list; The update instruction acquisition module 303 is used to construct an interception list update instruction containing the target interception list based on the target interception list; Control function status acquisition module 304 is used to acquire the network control function status of the browser; The request interception module 305 is used to send the interception list update instruction to the browser through the full-duplex communication channel if the network control function status indicates that the browser's network control function is enabled. The browser is used to intercept network resource requests according to the target interception list.
[0137] In one optional embodiment of this application, the interception list acquisition module 302 includes: The identifier type acquisition submodule is used to acquire the identifier type of the identifier of the resource to be intercepted; the identifier type includes one of null, string, list, tuple and other types; the other types are identifier types other than null, string, list and tuple; The first list acquisition module is used to encapsulate the identifier of the resource to be intercepted into an empty list if the identifier type is the null value, and use the empty list as the target interception list; The second list acquisition module is used to encapsulate the identifier of the resource to be intercepted into a single-element list if the identifier type is the string, and use the single-element list as the target interception list; The third list acquisition module is used to take the identifier of the resource to be intercepted as the target interception list if the identifier type is the list or the tuple. The error message acquisition module is used to generate type error message information based on the identifier of the resource to be intercepted if the identifier type is one of the other types, and then send the type error message information to the user.
[0138] In one optional embodiment of this application, the update instruction acquisition module 303 includes: The standardization processing submodule is used to perform deduplication, sorting and security verification on several of the target interception resource identifiers in the target interception list to obtain the processed target interception list. The sharding processing submodule is used to split the target interception list into several target interception sublists if the number of the resource identifiers to be intercepted in the processed target interception list is greater than a preset number threshold, and to construct an interception list update instruction according to the several target interception sublists respectively. The update instruction acquisition submodule is used to construct an interception list update instruction containing the target interception list if the number of the resource identifiers to be intercepted in the target interception list is less than or equal to the number threshold.
[0139] In one optional embodiment of this application, the apparatus further includes: The enable command acquisition module is used to generate a network control domain enable command if the initial state of the network control function indicates that the browser's network control function is disabled. The network control function activation module is used to send the network control domain activation command to the browser, and the browser is used to activate the network control function according to the network control domain activation command.
[0140] In an optional embodiment of this application, after sending the network control domain enable command, the apparatus further includes: The shutdown instruction acquisition module is used to generate a network control domain shutdown instruction if the interception list update instruction fails to be sent. The network control function shutdown module is used to send the network control domain shutdown command to the browser, and the browser is used to shut down the network control function according to the network control domain shutdown command.
[0141] In an optional embodiment of this application, after sending the block list update instruction to the browser, the device further includes: The update event acquisition module is used to generate an interception list update event and associate and save the target interception list with the interception list update event.
[0142] In one optional embodiment of this application, the apparatus further includes: The blocking event acquisition module is used to receive request blocking events sent by the browser; wherein the request blocking event is generated by the browser when it successfully blocks a network resource request that matches the target blocking list; The blocking log acquisition module is used to generate and record network resource blocking logs based on the received request blocking events.
[0143] As the apparatus embodiment is basically similar to the method embodiment, it is described in a relatively simple manner. For relevant details, please refer to the description of the method embodiment.
[0144] An embodiment of this application also provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the method described above.
[0145] An embodiment of this application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the method described above.
[0146] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0147] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0148] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0149] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0150] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0152] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other modifications and updates to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all modifications and updates falling within the scope of the embodiments of the present application.
[0153] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.
[0154] The above provides a detailed description of the network resource request interception method, apparatus, electronic device, and medium. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for intercepting network resource requests, characterized in that, An application to a network resource interception system, wherein the network resource interception system communicates with a browser via a full-duplex communication channel, the method comprising: Receive the identifier of the resource to be intercepted from the user input; The identifier of the resource to be intercepted is adaptively encapsulated to obtain the target interception list; Construct an intercept list update instruction containing the target intercept list based on the target intercept list; Obtain the network control function status of the browser; If the network control function status indicates that the browser's network control function is enabled, the intercept list update instruction is sent to the browser through the full-duplex communication channel, and the browser is used to intercept network resource requests according to the target intercept list.
2. The method according to claim 1, characterized in that, The step of adaptively encapsulating the identifier of the resource to be intercepted to obtain the target interception list includes: Obtain the identifier type of the resource identifier to be intercepted; the identifier type includes one of null, string, list, tuple and other types; the other types are identifier types other than null, string, list and tuple; If the identifier type is null, then the identifier of the resource to be intercepted is encapsulated into an empty list, and the empty list is used as the target interception list; If the identifier type is the string, then the identifier of the resource to be intercepted is encapsulated into a single-element list, and the single-element list is used as the target interception list; If the identifier type is the list or the tuple, then the identifier of the resource to be intercepted is used as the target interception list; If the identifier type is one of the other types, then a type error message is generated based on the identifier of the resource to be intercepted, and the type error message is sent to the user.
3. The method according to claim 1, characterized in that, The step of constructing an update instruction containing the target interception list based on the target interception list includes: The target interception list is processed by deduplication, sorting and security verification of several resource identifiers to be intercepted in the target interception list to obtain the processed target interception list. If the number of resource identifiers to be intercepted in the processed target interception list is greater than a preset threshold, then the target interception list is split into several target interception sub-lists, and an interception list update instruction is constructed according to the several target interception sub-lists respectively; If the number of resource identifiers to be intercepted in the target interception list is less than or equal to the number threshold, then an interception list update instruction containing the target interception list is constructed based on the target interception list.
4. The method according to claim 1, characterized in that, The method further includes: If the initial state of the network control function indicates that the browser's network control function is disabled, then a network control domain enable instruction is generated; The network control domain enable command is sent to the browser, and the browser is used to enable network control functions according to the network control domain enable command.
5. The method according to claim 4, characterized in that, After sending the network control domain enable command, the method further includes: If the intercept list update command fails to be sent, a network control domain shutdown command is generated. The network control domain shutdown command is sent to the browser, and the browser is used to disable the network control function according to the network control domain shutdown command.
6. The method according to claim 1, characterized in that, After sending the block list update instruction to the browser, the method further includes: Generate an intercept list update event, and associate the target intercept list with the intercept list update event and save it.
7. The method according to claim 1, characterized in that, The method further includes: Receive a request blocking event sent by the browser; wherein the request blocking event is generated by the browser when it successfully blocks a network resource request that matches the target blocking list; Based on the received request blocking event, generate and record a network resource blocking log.
8. A network resource request interception device, characterized in that, An apparatus for use in a network resource interception system, wherein the network resource interception system communicates with a browser via a full-duplex communication channel, the apparatus comprising: The identifier acquisition module is used to receive the identifier of the resource to be intercepted input by the user; The interception list acquisition module is used to adaptively encapsulate the identifier of the resource to be intercepted to obtain the target interception list; The update instruction acquisition module is used to construct an interception list update instruction containing the target interception list based on the target interception list; A control function status acquisition module is used to acquire the network control function status of the browser; The request interception module is used to send the interception list update instruction to the browser through the full-duplex communication channel if the network control function status indicates that the browser's network control function is enabled. The browser is used to intercept network resource requests according to the target interception list.
9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored on the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the method as described in any one of claims 1-7.
10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the method as described in any one of claims 1-7.