A data processing method, apparatus, electronic device, and storage medium
By adding a preset decryption method to the webpage script file to generate a second script file, the problem of the server being unable to obtain non-user interface layer data is solved, thus achieving both effectiveness and efficiency in data acquisition.
Patent Information
- Application Number
- CN202211006606.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-22
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-08-22
Smart Images

Figure CN115374331B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of Internet technology, and in particular to a data processing method, apparatus, electronic device and storage medium. Background Technology
[0002] Currently, servers can monitor and retrieve data from web pages based on a browser plugin.
[0003] However, in the above method, the server may only be able to obtain user interface (UI) layer data (such as the account name of an account) on the webpage, and may not be able to obtain non-UI layer data (such as the account identifier of the account), which affects the effectiveness of data acquisition. Summary of the Invention
[0004] This disclosure provides a data processing method, apparatus, electronic device, and storage medium, which solves the problem in related technologies that servers can only obtain UI layer data from web pages, but cannot obtain non-UI layer data.
[0005] The technical solution of this disclosure is as follows:
[0006] According to a first aspect of the present disclosure, a data processing method is provided. The method may include: when a first script file of a target webpage includes a preset decryption method, performing preset processing on the first script file to generate a second script file, the preset decryption method being used to decrypt encrypted data to obtain decrypted data; and executing the second script file to add the decrypted data to a target queue.
[0007] Optionally, before performing the preset processing on the first script file, the data processing method further includes: in response to the preset operation, sending a script file retrieval request to the server, the script file retrieval request including the Uniform Resource Locator (URL) of the first script file, the script file retrieval request being used to request retrieval of the first script file; and receiving a script file retrieval response sent by the server, the script file retrieval response including the first script file.
[0008] Optionally, the above-mentioned pre-processing of the first script file specifically includes: adding a listening method to the first script file, which is used to listen for the decrypted data.
[0009] Optionally, the above data processing method further includes: injecting the listening method into a preset object of the target webpage so that the electronic device can call the listening method.
[0010] Optionally, the above data processing method further includes: when the form engine of the preset application has a display function, replacing the form engine with an embedded engine, the embedded engine being used to prevent the window of the target webpage from being displayed, and the preset application being an application that executes the second script file.
[0011] According to a second aspect of the present disclosure, a data processing apparatus is provided. The apparatus may include: a processing module; the processing module is configured to, when a first script file of a target webpage includes a preset decryption method, perform preset processing on the first script file to generate a second script file, the preset decryption method being used to decrypt encrypted data to obtain decrypted data; the processing module is further configured to execute the second script file to add the decrypted data to a target queue.
[0012] Optionally, the data processing apparatus further includes a sending module and a receiving module; the sending module is configured to send a script file retrieval request to the server in response to a preset operation, the script file retrieval request including the Uniform Resource Locator of the first script file, the script file retrieval request being used to request retrieval of the first script file; the receiving module is configured to receive a script file retrieval response sent by the server, the script file retrieval response including the first script file.
[0013] Optionally, the processing module is specifically configured to add a listening method to the first script file, which is used to listen for the decrypted data.
[0014] Optionally, the processing module is also configured to inject the listening method into a preset object of the target webpage, so that the electronic device can call the listening method.
[0015] Optionally, the processing module is further configured to replace the form engine of the preset application with an embedded engine when the form engine of the preset application has a display function. The embedded engine is used to prevent the window of the target webpage from being displayed. The preset application is an application that executes the second script file.
[0016] According to a third aspect of the present disclosure, an electronic device is provided, which may include: a processor and a memory configured to store processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the optional data processing methods of the first aspect described above.
[0017] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform any of the optional data processing methods of the first aspect described above.
[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions that, when executed on a processor of an electronic device, cause the electronic device to perform any of the optional data processing methods of the first aspect.
[0019] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:
[0020] Based on any of the above aspects, in this disclosure, when the first script file of the target webpage includes a preset decryption method, the electronic device can perform preset processing on the first script file to generate a second script file. Then, when the electronic device executes the second script file, it adds the decrypted data to the target queue. In this embodiment of the disclosure, the electronic device can add the decrypted data to the target queue by generating and executing the second script file. This allows downstream devices (i.e., devices that need to obtain the decrypted data) to conveniently and quickly obtain the decrypted data, thereby improving the effectiveness of data acquisition.
[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.
[0023] Figure 1 A schematic diagram of a data processing system provided in an embodiment of this disclosure is shown;
[0024] Figure 2 A flowchart illustrating a data processing method provided in an embodiment of this disclosure is shown.
[0025] Figure 3 A flowchart illustrating another data processing method provided in an embodiment of this disclosure is shown;
[0026] Figure 4 A flowchart illustrating another data processing method provided in an embodiment of this disclosure is shown;
[0027] Figure 5 A flowchart illustrating another data processing method provided in an embodiment of this disclosure is shown;
[0028] Figure 6 A flowchart illustrating another data processing method provided in an embodiment of this disclosure is shown;
[0029] Figure 7A flowchart illustrating another data processing method provided in an embodiment of this disclosure is shown;
[0030] Figure 8 A schematic diagram of the structure of a data processing apparatus provided in an embodiment of this disclosure is shown;
[0031] Figure 9 A schematic diagram of the structure of another data processing apparatus provided in an embodiment of the present disclosure is shown. Detailed Implementation
[0032] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0033] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0034] It should also be understood that the term "comprising" indicates the presence of the described feature, whole, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements and / or components.
[0035] It should be noted that the user information (including but not limited to user device information, user personal information, user behavior information, etc.) and data (including but not limited to scripts) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.
[0036] In related technologies, servers can monitor and retrieve data from web pages based on a browser plugin. However, the server may only be able to retrieve UI-level data (such as an account's username) from the web page, and cannot retrieve non-UI-level data (such as the account's identifier), affecting the effectiveness of data retrieval.
[0037] Based on this, this disclosure provides a data processing method. When a first script file of a target webpage includes a preset decryption method, an electronic device can perform preset processing on the first script file to generate a second script file. Since the first script file includes the preset decryption method, the server can obtain decrypted data when executing the second script file. In this disclosure, the electronic device can add the decrypted data to the target queue by generating and executing the second script file. This allows downstream devices (i.e., devices that need to obtain the decrypted data) to obtain the decrypted data conveniently and quickly, improving the effectiveness of data acquisition.
[0038] The data processing method, apparatus, electronic device, and storage medium provided in this disclosure are applied in data acquisition scenarios. When the first script file of the target webpage includes a preset decryption method, the electronic device can perform preset processing on the first script file according to the method provided in this disclosure to generate a second script file, and execute the second script file to add the decrypted data to the target queue. Subsequently, the downstream device (i.e., the device that needs to obtain the decrypted data) can obtain the decrypted data based on the target queue.
[0039] The data processing method provided in the embodiments of this disclosure is illustrated below with reference to the accompanying drawings:
[0040] Figure 1 This is a schematic diagram of a data processing system provided in an embodiment of the present disclosure, such as... Figure 1 As shown, the data processing system may include an electronic device 101 and a server 102. The electronic device 101 can establish a connection with the server 102 through a wired network or a wireless network.
[0041] The electronic device 101 can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device. This disclosure does not impose any special restrictions on the specific form of the electronic device 101. It can interact with the user through one or more methods such as a keyboard, touchpad, touchscreen, remote control, voice interaction, or handwriting device.
[0042] Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN) services, as well as big data and artificial intelligence platforms.
[0043] In this embodiment of the disclosure, electronic device 101 can send a script file retrieval request to server 102. After receiving the script file retrieval request, server 102 can determine the first script file based on the Uniform Resource Locator included in the script file retrieval request, and send a script file retrieval response to electronic device 101, which includes the first script file.
[0044] like Figure 2 As shown, when the data processing method provided in this embodiment is applied to the above-mentioned electronic device 101, the data processing method may include S101-S102.
[0045] S101. When the first script file of the target webpage includes a preset decryption method, the electronic device performs preset processing on the first script file to generate a second script file.
[0046] The preset decryption method is used to decrypt encrypted data to obtain decrypted data.
[0047] It should be understood that the electronic device can obtain the first script file and parse it to obtain at least one method (specifically, the name of the at least one method). Then, the electronic device can determine whether the name of the at least one method includes the name of the aforementioned preset decryption method. When the name of the at least one method includes the name of the preset decryption method, it indicates that the at least one method includes the preset decryption method, and the electronic device can determine that the first script file includes the preset decryption method.
[0048] It is understandable that a webpage (such as the target webpage) may contain multiple script files, which can be used to implement different functions. For example, the first script file mentioned above can be used to decrypt encrypted data. Specifically, when an electronic device executes the first script file, it decrypts the encrypted data based on the preset decryption method to obtain the decrypted data.
[0049] In this embodiment of the disclosure, a script file (e.g., a first script file) may be a js file, which is a text file with the extension ".js".
[0050] Optionally, a method in this embodiment of the present disclosure (such as the preset decryption method described above) may be a text message, which may consist of one or more lines of code.
[0051] In this embodiment of the disclosure, the decrypted data is data from the target webpage. The data in the target webpage may include images, numbers, text, etc.
[0052] Optionally, the decrypted data can exist in the form of a text file, such as a JSON text file.
[0053] It should be understood that the aforementioned preset processing can specifically be modification processing or replacement processing. Specifically, modifying the first script file by the electronic device can be understood as modifying the content included in the first script file to generate the aforementioned second script file; replacing the first script file by the electronic device can be understood as replacing the content included in the first script file with other content to generate the second script file, or directly replacing the first script file with other script files (i.e., the second script file).
[0054] S102, The electronic device executes the second script file to add the decrypted data to the target queue.
[0055] It should be understood that after the electronic device adds the decrypted data to the target queue, the device that needs to obtain the decrypted data (hereinafter referred to as the downstream device) can monitor and obtain the decrypted data based on the target queue.
[0056] In one alternative implementation, the downstream device can convert the decrypted data into a short message and notify the target webpage's followers in the form of a short message; or the downstream device can convert the decrypted data into a data analysis report and display the data analysis report to the analyst; or the downstream device can generate a dynamic data monitoring chart based on the decrypted data and display the dynamic data monitoring chart in the corresponding monitoring panel, etc.
[0057] Optionally, the target queue mentioned above can be a Kafka message queue or a RocketMQ queue.
[0058] The technical solution provided by the above embodiments can bring at least the following beneficial effects: As shown in S101-S102, when the first script file of the target webpage includes a preset decryption method, the electronic device can perform preset processing on the first script file to generate a second script file. Then, when the electronic device can execute the second script file, it can add the decrypted data to the target queue. In this embodiment of the present disclosure, the electronic device can add the decrypted data to the target queue by generating and executing the second script file. This allows downstream devices (i.e., devices that need to obtain the decrypted data) to obtain the decrypted data conveniently and quickly, thereby improving the effectiveness of data acquisition.
[0059] Combination Figure 2 ,like Figure 3 As shown, before the electronic device performs preset processing on the first script file, the data processing method provided in this embodiment further includes S103-S104.
[0060] S103. The electronic device responds to a preset operation by sending a script file retrieval request to the server.
[0061] The script file retrieval request includes the Uniform Resource Locator (URL) of the first script file, and the script file retrieval request is used to request the retrieval of the first script file.
[0062] It should be understood that the aforementioned preset operation can be a click operation on an object included in the target webpage, or a page refresh operation on the target webpage. Specifically, in response to a user account's click operation on the object (or a page refresh operation on the target webpage), the electronic device can send a request to the server to retrieve the script file.
[0063] It is understood that, in response to the aforementioned preset operations, the electronic device can send various types of retrieval requests to the server, including the script retrieval request mentioned above. These various types of retrieval requests may also include retrieval requests for images (or videos).
[0064] In this embodiment of the disclosure, after receiving the script file retrieval request, the server can determine and retrieve the first script file based on the uniform resource locator (URL) of the first script file included in the script file retrieval request, and then the server can send a script file retrieval response including the first script file to the electronic device.
[0065] S104. The electronic device receives the script file sent by the server and obtains a response.
[0066] The response obtained from this script file includes the aforementioned first script file.
[0067] In this embodiment of the disclosure, the electronic device receives a script file sent by the server and obtains a response, which means that it has obtained the first script file.
[0068] The technical solution provided by the above embodiments can bring at least the following beneficial effects: As shown in S103-S104, in response to a preset operation, the electronic device can send a script file retrieval request to the server. Since the script file retrieval request includes the Uniform Resource Locator (URL) of the first script file, the server can determine the first script file based on the URL after receiving the script file retrieval request. Then, the server can send a script file retrieval response to the electronic device, which includes the first script file. In this embodiment, the electronic device can obtain the first script file from the server by sending a script file retrieval request, which can improve the efficiency of obtaining the first script file and thus improve the efficiency of generating the second script file.
[0069] Combination Figure 2 ,like Figure 4 As shown, in one implementation of this embodiment, the electronic device performs preset processing on the script file, which may specifically include S1011.
[0070] S1011. Add a listening method to the electronic device in the first script file.
[0071] The monitoring method is used to monitor the decrypted data mentioned above.
[0072] In this embodiment of the disclosure, the above-described monitoring method can also be understood as a text message (or a line of code). The electronic device can add the text message (or code) to the first script file to monitor the decrypted data, so that when the electronic device executes the second script file, it can add the decrypted data to the target queue.
[0073] Optionally, the electronic device may add the listening method before the preset decryption method in the first script file; the electronic device may also add the listening method after the preset decryption method.
[0074] In an optional implementation, the first script file may further include the encrypted data and decrypted data, where the encrypted data can be understood as input parameters and the decrypted data as output parameters. In this embodiment, the electronic device adds the decrypted data to the target queue, which can also be understood as adding the output parameter to the target queue. Subsequently, downstream devices can monitor and obtain the output parameter.
[0075] The technical solution provided by the above embodiments can bring at least the following beneficial effects: As shown in S1011, the electronic device can add a listening method to the first script file. Since this listening method is used to listen to decrypted data, the electronic device can add a second script file generated by this listening method to the first script file to listen to the decrypted data, and can accurately and effectively process the first script file, improving the effectiveness of data processing. Furthermore, when the electronic device executes the second script file, it can add the decrypted data to the target queue, so that downstream devices can listen to and obtain the decrypted data, thereby improving the efficiency of data acquisition.
[0076] Combination Figure 4 ,like Figure 5 As shown, the data processing method provided in this embodiment of the disclosure further includes S105.
[0077] S105. The electronic device injects a listening method into a preset object on the target webpage so that the electronic device can call the listening method.
[0078] It should be noted that the target webpage may include multiple objects, which can be divided into public objects and hidden objects. The public objects can be understood as objects that electronic devices can directly use.
[0079] In this embodiment of the disclosure, the aforementioned preset object can be the window object of the target webpage.
[0080] In this embodiment of the disclosure, the electronic device injects the above-described listening method into the preset object, so that the electronic device can add the listening method to the first script file.
[0081] It is understandable that simply executing the first script file described above may not enable downstream devices to obtain the decrypted data. In this embodiment, the electronic device can inject the listening method into the preset object using the jsbridge extension capability built into the JavaFX framework. Specifically, this can be understood as bridging the JavaScript code to the Java layer. In this way, the listening method can transmit the decrypted data to the Java layer. After receiving the decrypted data, the Java layer can add the decrypted data to the target queue and transmit the decrypted data to the downstream device based on the target queue.
[0082] The technical solutions provided by the above embodiments can bring at least the following beneficial effects: As shown in S105, the electronic device can inject a listening method into a preset object of the target webpage, so that the electronic device can call the listening method. In this embodiment of the present disclosure, after the electronic device calls the listening method, it can add the listening method to the first script file, which can accurately process the first script file and improve the efficiency of data processing.
[0083] Combination Figure 2 ,like Figure 6 As shown, the data processing method provided in this embodiment further includes S106.
[0084] S106. When the form engine of the default application has a display function, the electronic device replaces the form engine with an embedded engine.
[0085] The embedded engine is used to prevent the window displaying the target webpage from being shown, and the default application is the application that executes the second script file mentioned above.
[0086] In this embodiment of the disclosure, the preset application may include a graphics engine, a window engine, a media engine, and a web engine. Specifically, the graphics engine is used to display the graphics (or controls) of the target webpage; the window engine is used to display the window of the target webpage; the media engine is used to play images and videos; and the web engine is used to parse the code in the target webpage and execute script files.
[0087] It should be understood that when the form engine has display functionality, it means that the data of the target webpage (such as the decrypted data mentioned above) needs to be displayed on the electronic device's monitor (or display interface). The electronic device replaces the form engine with the embedded engine, that is, replaces the form engine with a form engine that does not have display functionality. When the form engine does not have display functionality (e.g., after the electronic device replaces the form engine with an embedded engine), the electronic device can hide the window of the target webpage so that the default application can run in a pure background environment. That is, the data of the target webpage does not need to be displayed on the electronic device's monitor, or the data of the target webpage can be stored on an electronic device without a monitor.
[0088] Optionally, the aforementioned window engine with display functionality can be a Glass engine, and the embedded engine can be a Monocel engine. Replacing the Glass engine with the Monocel engine in an electronic device allows the device to run in an Internet data center (IDC) room. Consequently, when a pre-installed application in the electronic device executes the second script file, it can listen for and retrieve data (or data changes) of the target webpage in the IDC room.
[0089] The technical solution provided by the above embodiments can bring at least the following beneficial effects: As shown in S106, when the window engine of the preset application has a display function, the electronic device can replace the window engine with an embedded engine. Since the embedded engine prohibits the display of the target webpage window, the electronic device does not need to configure the target webpage window in the electronic device. In this embodiment of the present disclosure, the electronic device can replace the window engine with a display function with an embedded engine (i.e., a window engine that prohibits the display of the target webpage window). In this way, the electronic device does not need to rely on the display to display the data of the target webpage, and can listen to and obtain the data of the target webpage in the IDC data center, which can reduce the usage cost of the electronic device.
[0090] The following example illustrates the data processing method provided in the embodiments of this disclosure.
[0091] For example, such as Figure 7 As shown, the URLLoader.java network layer core class is a browser control in Java that can proxy requests and responses sent by the browser (i.e., the default application) (e.g., retrieving requests and responses from the script file mentioned above). JavaFX is a graphical user interface (GUI) programming framework built into Java for developing Windows Forms applications. It includes a built-in WebView control based on the WebKit browser engine, allowing it to open any webpage like a regular browser. This JavaFX WebView control contains the WebKit engine and the JFX library, a utility library for JavaFX that can open a window and pass data.
[0092] Continue as Figure 7 As shown, in S201, the process begins; specifically, the electronic device can start executing the data processing method provided in this embodiment. In S202, Javassist is used to inject extension methods into the network layer class; specifically, Javassist is a bytecode editor built into the Java language, which can dynamically modify the bytecode of a program, enabling extended modifications to the basic libraries in a Java program. In this embodiment, the electronic device uses the Java programming language and modifies the network sending and response parts of the core network layer class URLLoader.java using the Javassist bytecode modification library. This can be understood as the process of generating a second script file based on the first script file.
[0093] Specifically, S202 includes S2021-S2027: S2021, Initiating a network request; that is, the electronic device can send a network request to the server. S2022, Whether it is a JS file; the JS file can be understood as the first script file mentioned above; specifically, the electronic device can determine whether the network request is used to request and obtain the JS file. S2023, Whether to replace; specifically, when the network request is used to request and obtain the JS file, the electronic device can determine whether to replace the JS file (specifically the first script file) with (i.e., generate) the second script file. S2024, Partial code of the replaced JS file: / / Target method to be monitored var r = target_fun(p1,p2); / / Call jsbridge to listen to the data / / Send it out through Kafka send_kafka(r,p1,p2). Specifically, the electronic device can perform partial replacement of the JS file, specifically by adding a method to send to Kafka after the target method (i.e., the aforementioned preset decryption method), and this method to send to Kafka is the aforementioned monitoring method. Wherein, var r represents the decrypted data, target_fun represents the preset decryption method, and p1 and p2 represent the encrypted data. S2025, Kafka message queue; this Kafka message queue can be understood as the target queue. Specifically, the electronic device can add the decrypted data to this Kafka message queue. S2026, Original network request; specifically, when the network request is not a request to obtain a JS file or does not need to replace the JS file (i.e., the preset decryption method is not included in the first script file), no processing is performed on the network request, and the original network request is the original network request. S2027, Return response body (i.e., network response, such as the script file retrieval response) to the WebKit kernel. Specifically, the electronic device can return the first script file or the second script file to the WebKit kernel.
[0094] S203. Using jsbridge, add the send_kafka method to WebKit. Specifically, jsbridge can provide browser extension capabilities, allowing the browser to switch from one language to another. In this embodiment, jsbridge is used to switch the JavaScript language to Java. S204. OpenJFX-Monocel window virtualization technology enables the program to run in headless mode. Specifically, headless refers to a browser without a UI (also understood as a headless browser), running in the background and usable without a monitor. That is, the electronic device can replace the aforementioned window engine with an embedded engine. S205. Start the JavaFX program, load the target webpage, and monitor its operation. Specifically, starting the JavaFX program on the electronic device means beginning to listen to the data on the target webpage. In an optional implementation, when the data on the target webpage changes, the electronic device (specifically the JavaFX program) calls jsbridge to add the decrypted data to the Kafka message queue. S206. Various downstream consumers perform sentiment analysis or generate new message notifications; specifically, these downstream consumers are the aforementioned downstream devices. This public opinion analysis can be understood as the electronic device analyzing the decrypted data and generating a certain analysis strategy (or analysis report); the new message notification can be used to alert the account to changes in the decrypted data.
[0095] It is understood that, in actual implementation, the electronic device / server described in the embodiments of this disclosure may include one or more hardware structures and / or software modules for implementing the aforementioned corresponding data processing methods. These hardware structures and / or software modules can constitute an electronic device / server. Those skilled in the art should readily recognize that, based on the algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.
[0096] Based on this understanding, the present disclosure also provides a data processing apparatus. Figure 8 A schematic diagram of the structure of a data processing apparatus provided in an embodiment of this disclosure is shown. Figure 8 As shown, the data processing device 20 may include a processing module 201.
[0097] The processing module 201 is configured to perform preset processing on the first script file when the first script file of the target webpage includes a preset decryption method, and generate a second script file. The preset decryption method is used to decrypt encrypted data to obtain decrypted data. The processing module is also configured to execute the second script file to add the decrypted data to the target queue.
[0098] Optionally, the data processing device 20 further includes a sending module 202 and a receiving module 203; the sending module is configured to send a script file retrieval request to the server in response to a preset operation, the script file retrieval request including the Uniform Resource Locator of the first script file, the script file retrieval request being used to request retrieval of the first script file; the receiving module is configured to receive a script file retrieval response sent by the server, the script file retrieval response including the first script file.
[0099] Optionally, the processing module 201 is specifically configured to add a listening method to the first script file, the listening method being used to listen for the decrypted data.
[0100] Optionally, the processing module 201 is further configured to inject the listening method into a preset object of the target webpage, so that the electronic device calls the listening method.
[0101] Optionally, the processing module 201 is further configured to replace the form engine of the preset application with an embedded engine when the form engine of the preset application has a display function. The embedded engine is used to prevent the window of the target webpage from being displayed. The preset application is an application that executes the second script file.
[0102] As described above, the embodiments of this disclosure can divide the data processing device into functional modules according to the above method examples. The integrated modules can be implemented in hardware or as software functional modules. Furthermore, it should be noted that the module division in these embodiments is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module.
[0103] The specific methods by which each module performs its operations and the beneficial effects of the data processing apparatus in the above embodiments have been described in detail in the foregoing method embodiments, and will not be repeated here.
[0104] Figure 9 This is a schematic diagram of another data processing apparatus provided in this disclosure. Figure 9The data processing apparatus 30 may include at least one processor 301 and a memory 303 for storing processor-executable instructions. The processor 301 is configured to execute the instructions in the memory 303 to implement the data processing method described in the above embodiments.
[0105] In addition, the data processing device 30 may also include a communication bus 302 and at least one communication interface 304.
[0106] Processor 301 may be a processor (central processing unit, CPU), microprocessor unit, ASIC, or one or more integrated circuits for controlling the execution of programs according to the present disclosure.
[0107] The communication bus 302 may include a path for transmitting information between the aforementioned components.
[0108] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.
[0109] Memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.
[0110] The memory 303 stores instructions for executing the present invention, and the processor 301 controls the execution of these instructions. The processor 301 executes the instructions stored in the memory 303 to implement the functions of the method disclosed herein.
[0111] In a specific implementation, as one example, processor 301 may include one or more CPUs, for example... Figure 9 CPU0 and CPU1 in the CPU.
[0112] In a specific implementation, as one example, the data processing device 30 may include multiple processors, such as... Figure 9 Processors 301 and 307 are described herein. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).
[0113] In a specific implementation, as one embodiment, the data processing device 30 may further include an output device 305 and an input device 306. The output device 305 communicates with the processor 301 and can display information in various ways. For example, the output device 305 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device 306 communicates with the processor 301 and can accept user input in various ways. For example, the input device 306 may be a mouse, keyboard, touchscreen device, or sensing device, etc.
[0114] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the data processing device 30, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.
[0115] In addition, this disclosure also provides a computer-readable storage medium including instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the data processing method provided in the above embodiments.
[0116] In addition, this disclosure also provides a computer program product including instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the data processing method provided in the above embodiments.
[0117] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.
Claims
1. A data processing method applied to electronic devices, characterized in that, The method includes: When the first script file of the target webpage includes a preset decryption method, the first script file is processed to generate a second script file. The preset processing includes adding a listening method to the first script file. The listening method is used to listen for decrypted data. The preset decryption method is used to decrypt encrypted data to obtain decrypted data. Execute the second script file to add the decrypted data to the target queue; When the form engine of the preset application has a display function, the form engine is replaced with an embedded engine. The embedded engine is used to prevent the window of the target webpage from being displayed. The preset application is an application that executes the second script file.
2. The data processing method according to claim 1, characterized in that, Before performing the preset processing on the first script file, the method further includes: In response to a preset operation, a script file retrieval request is sent to the server. The script file retrieval request includes the Uniform Resource Locator (URL) of the first script file. The script file retrieval request is used to request the retrieval of the first script file. The server receives a script file retrieval response, which includes the first script file.
3. The data processing method according to claim 1, characterized in that, The method further includes: The listening method is injected into a preset object on the target webpage so that the electronic device can call the listening method.
4. A data processing apparatus, characterized in that, include: Processing module; The processing module is configured to perform preset processing on the first script file to generate a second script file when the first script file of the target webpage includes a preset decryption method. The preset processing includes adding a listening method to the first script file. The listening method is used to listen for decrypted data. The preset decryption method is used to decrypt the encrypted data to obtain decrypted data. The processing module is further configured to replace the window engine with an embedded engine when the window engine of the preset application has a display function. The embedded engine is used to prevent the window of the target webpage from being displayed. The preset application is an application that executes the second script file. The processing module is also configured to execute the second script file to add the decrypted data to the target queue.
5. The data processing apparatus according to claim 4, characterized in that, The data processing device further includes an acquisition module and a receiving module; The acquisition module is configured to send a script file acquisition request to the server in response to a preset operation. The script file acquisition request includes the Uniform Resource Locator (URL) of the first script file and is used to request the acquisition of the first script file. The receiving module is configured to receive a script file retrieval response sent by the server, wherein the script file retrieval response includes the first script file.
6. An electronic device, characterized in that, The electronic device includes: processor; A memory configured to store processor-executable instructions; The processor is configured to execute the instructions to implement the data processing method as described in any one of claims 1-3.
7. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the data processing method as described in any one of claims 1-3.
8. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on a processor of an electronic device, cause the electronic device to perform the data processing method as described in any one of claims 1-3.
Citation Information
Patent Citations
Webpage data crawling method, device, equipment and storage medium
CN111639248A
