Data acquisition method, device, storage medium and electronic device

Abstract

The present application discloses a data acquisition method, device, storage medium and electronic device, relating to the field of Internet technology. The method includes: when receiving a data acquisition instruction, sending a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application; receiving the crawler code in the form of a string returned by the cloud; converting the crawler code into a script code block matching the target web page; and injecting the script code block into the target web page for data acquisition. The present application effectively avoids the problem of data acquisition failure caused by changes in web page rules and improves the reliability of data acquisition.

Description

Technical Field

[0001] This application relates to the field of Internet technologies, and in particular, to a data collection method, apparatus, storage medium, and electronic device. Background Art

[0002] In a device side taking an Android mobile terminal as an example, there is usually a need to crawl and collect data on a web page through legal crawler code. Usually, crawler code for formulating crawling rules for the corresponding web page is injected to collect data. If the corresponding web page is updated, etc., the corresponding crawler code needs to be updated.

[0003] In a conventional processing solution, the crawler code is built into an application (app) on the device side. If the web page rules of the web page corresponding to the application change, the crawler code needs to be updated by publishing an updated application to collect data normally. Otherwise, data collection fails, resulting in low reliability of data collection. Summary of the Invention

[0004] An embodiment of this application provides a solution that can effectively avoid the problem of data collection failure caused by changes in web page rules and improve the reliability of data collection.

[0005] To solve the above technical problems, the embodiments of this application provide the following technical solutions:

[0006] According to an embodiment of this application, a data collection method is applied to a target application. The method includes: when receiving a data collection instruction, sending a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application; receiving the crawler code in the form of a string returned by the cloud; converting the crawler code into a script code block matching the target web page; and injecting the script code block into the target web page for data collection.

[0007] In some embodiments of this application, before sending the code acquisition request to the cloud, the method further includes: obtaining the web page rule change information in the target web page; determining whether an updated crawler code needs to be obtained according to the web page rule change information; and generating the code acquisition request when it is determined that an updated crawler code needs to be obtained.

[0008] In some embodiments of this application, the script code block is marked with a specific mark; injecting the script code block into the target web page for data collection includes: detecting whether a preset script code block corresponding to the specific mark exists in the target web page; and injecting the script code block into the target web page for data collection when the preset script code block corresponding to the specific mark does not exist in the target web page.

[0009] In some embodiments of the present application, generating the code acquisition request includes: acquiring application information corresponding to the target application; generating the code acquisition request carrying the application information, where the application information is used for the cloud to determine the crawler code matching the target application.

[0010] In some embodiments of the present application, acquiring the application information corresponding to the target application includes: sending the web page rule change information in the target web page to a specific authorized terminal, so that the specific authorized terminal parses the crawling rules based on the web page rule change information and generates code authorization information matching the web page rule change information; receiving the code authorization information as the application information corresponding to the target application.

[0011] In some embodiments of the present application, acquiring the application information matching the target application includes: acquiring the application description information of the target application as the application information matching the target application.

[0012] In some embodiments of the present application, sending a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application includes: sending the acquisition request carrying the application information to the cloud to trigger the cloud to determine the crawler code corresponding to the application information as the crawler code matching the target application.

[0013] According to an embodiment of the present application, a data acquisition device is applied to a target application. The device includes: a sending module, configured to send a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application when receiving a data acquisition instruction; a receiving module, configured to receive the crawler code in the form of a string returned by the cloud; a conversion module, configured to convert the crawler code into a script code block matching the target web page; and an injection module, configured to inject the script code block into the target web page for data acquisition.

[0014] According to another embodiment of the present application, a storage medium stores a computer program thereon. When the computer program is executed by a processor of a computer, the computer is caused to execute the method described in the embodiments of the present application.

[0015] According to another embodiment of the present application, an electronic device may include: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the method described in the embodiments of the present application.

[0016] In the embodiments of the present application, the target application can: when receiving a data collection instruction, send a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application; receive the crawler code in the form of a string returned by the cloud; convert the crawler code into a script code block matching the target web page; and inject the script code block into the target web page for data collection.

[0017] In this way, when performing data collection, the terminal can obtain the matching crawler code from the cloud based on the target application, and then convert the crawler code into a script code block matching the target web page and inject it into the target web page for data collection, effectively avoiding the problem of data collection failure caused by changes in web page rules and improving the reliability of data collection. Description of the Drawings

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 The figure shows a schematic diagram of a system to which the embodiments of the present application can be applied.

[0020] Figure 2 The figure shows a flowchart of a data collection method according to an embodiment of the present application.

[0021] Figure 3 The figure shows a block diagram of a data collection device according to an embodiment of the present application.

[0022] Figure 4 The figure shows a block diagram of an electronic device according to an embodiment of the present application. Detailed Embodiments

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0024] In the following description, specific embodiments of the present application will be described with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to several times as being performed by a computer, and the computer execution referred to herein includes the operation of a computer processing unit that represents data in a structured form by an electronic signal. This operation transforms the data or maintains it in a position in the computer's memory system, which can be reconfigured or otherwise changed in a manner well known to those skilled in the art. The data structure in which the data is maintained is a physical location in the memory, which has specific characteristics defined by the data format. However, the principles of the present application are described in the above text, which does not represent a limitation, and those skilled in the art will understand that the various steps and operations described below can also be implemented in hardware.

[0025] Figure 1 FIG. shows a schematic diagram of a system 100 to which embodiments of the present application can be applied. As Figure 1 shown, the system 100 may include a cloud server 101 and a terminal 102. The terminal 102 can be any computer device, such as a computer, a mobile phone, a smart watch, and a home appliance device, etc. The cloud server 101 and the terminal 102 can be network-connected through a network device.

[0026] In one implementation of this example, the target application on the terminal 102 can: when receiving a data collection instruction, send a code acquisition request to the cloud (i.e., the cloud server 101) to trigger the cloud to determine the crawler code matching the target application; receive the crawler code in the form of a string returned by the cloud; convert the crawler code into a script code block matching the target web page; and inject the script code block into the target web page for data collection.

[0027] Figure 2 FIG. schematically shows a flowchart of a data collection method according to an embodiment of the present application. The execution subject of this data collection method can be the target application in any device, and the device is, for example, Figure 1 the terminal 102 shown.

[0028] As Figure 2 shown, this data collection method may include step S210 to step S240.

[0029] Step S210, when receiving a data collection instruction, send a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application;

[0030] Step S220, receive the crawler code in the form of a string returned by the cloud;

[0031] Step S230: Convert the crawler code into a script code block that matches the target web page.

[0032] Step S240: Inject the script code block into the target web page for data collection.

[0033] The data collection instruction can be an instruction triggered when the user triggers a specific control in the target application or an instruction received by the target application from the server or other applications. When the target application receives this data collection instruction, it automatically generates a code acquisition request and sends it to the cloud.

[0034] After the cloud receives this code acquisition request, it can respond to the request and determine the crawler code that matches the target application from the crawler codes pre - saved in the cloud database. For example, the application information of the target application can be carried in the code acquisition request, and the cloud can determine the crawler code that matches this application information from the crawler codes pre - saved in the cloud database. This crawler code is adapted to the web page rules of the target web page.

[0035] The crawler code in the cloud can be saved as a string. After the cloud determines the matching crawler code, it can send it to the target application. Then, the target application receives the crawler code in the form of a string returned by the cloud.

[0036] The target application can convert the crawler code into a script code block that matches the target web page (i.e., is compatible with the target web page) according to the characteristics of the target web page and the data crawling requirements. For example, web page feature information and objects corresponding to data information required for data crawling in the target web page can be added to the crawler code to form a script code block.

[0037] Furthermore, the target application can inject the script code block that adapts to the web page rules of the target web page and is compatible with the target web page into the target web page, and crawl data such as audio and video from the target web page through the script code block to achieve data collection.

[0038] In this way, based on steps S210 to S240, when performing data collection, the terminal can obtain the matching crawler code from the cloud based on the target application. Then, convert the crawler code into a script code block that matches the target web page and inject it into the target web page for data collection, effectively avoiding the problem of data collection failure caused by changes in web page rules and improving the reliability of data collection.

[0039] The following describes the specific process of each step when performing data collection.

[0040] In step S210, when receiving the data collection instruction, send a code acquisition request to the cloud to trigger the cloud to determine the crawler code that matches the target application.

[0041] In the implementation of this example, the data collection instruction can be an instruction triggered when the user triggers a specific control in the target application or an instruction received by the target application from the server or other applications. When the target application receives the data collection instruction, it automatically generates a code acquisition request and sends it to the cloud. After receiving the code acquisition request, the cloud can respond to the request and determine the crawler code matching the target application from the crawler codes pre-stored in the cloud database. For example, the application information of the target application can be carried in the code acquisition request, and the cloud can determine the crawler code matching the application information from the crawler codes pre-stored in the cloud database. The crawler code is adapted to the web rules of the target web page.

[0042] In one embodiment, before sending the code acquisition request to the cloud in step S210, the method further includes: obtaining the web rule change information in the target web page; determining whether an updated crawler code needs to be obtained according to the web rule change information; when it is determined that an updated crawler code needs to be obtained, generating the code acquisition request.

[0043] The web rule change information is information describing whether the rules in the web page have changed. The target application can monitor the update of the target web page. When the target web page is updated, the update log of the target web page is obtained, and the web rule change information is obtained based on the analysis of the update log.

[0044] When the web rule change information describes that the rules in the web page have changed, it can be determined that an updated crawler code needs to be obtained, and then the code acquisition request is generated. On the contrary, when the web rule change information describes that the rules in the web page have not changed, the target application can selectively generate the code acquisition request (for example, generate the code acquisition request when there is no crawler code locally, and do not generate the code acquisition request when there is a pre-obtained crawler code locally). Therefore, the crawler code is obtained only when the web rules change, further improving the reliability of data collection.

[0045] The crawler code adapted to the web rules of the target web page can be saved in the cloud. Each time the rules of the target web page are updated, the matching crawler code can be uploaded to the cloud.

[0046] In one embodiment, in step S210, generating the code acquisition request includes: obtaining the application information corresponding to the target application; generating the code acquisition request carrying the application information, where the application information is used for the cloud to determine the crawler code matching the target application.

[0047] The application information can include information corresponding to the target application such as code authorization information and application description information. Based on the target application information, the cloud can accurately determine the crawler code matching the target application.

[0048] In one embodiment, obtaining the application information corresponding to the target application includes: sending the web page rule change information in the target web page to a specific authorized terminal, so that the specific authorized terminal parses the crawling rule based on the web page rule change information and generates code authorization information matching the web page rule change information; receiving the code authorization information as the application information corresponding to the target application.

[0049] The specific authorized terminal can be a terminal that can be authorized to collect data from the target web page, such as a computer device or a server. After the target application sends the obtained web page rule change information to the specific authorized terminal, the specific authorized terminal parses the crawling rule through the web page rule change information, determines or generates a crawler code that can crawl data under the web page rule of the target web page, and generates code authorization information corresponding to the crawler code.

[0050] After the target application receives the code authorization information, as the application information corresponding to the target application, it can request the crawler code corresponding to the code authorization information from the cloud, and then can collect data safely, reliably and legally.

[0051] In one embodiment, obtaining the application information matching the target application includes: obtaining the application description information of the target application as the application information matching the target application.

[0052] The application description information can be the application version information of the target application and the information of the web page that the target application needs to start, etc. Sending the application description information as the application information matching the target application to the cloud can determine the crawler code matching the keyword by parsing the keyword of the application description information.

[0053] In one embodiment, in step S210, sending a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application includes: sending the acquisition request carrying the application information to the cloud to trigger the cloud to determine the crawler code corresponding to the application information as the crawler code matching the target application.

[0054] After the target application receives the code authorization information, as the application information corresponding to the target application, it can request the crawler code corresponding to the code authorization information from the cloud, and then can collect data safely, reliably and legally. Sending the application description information as the application information matching the target application to the cloud can determine the crawler code matching the keyword by parsing the keyword of the application description information.

[0055] In step S220, receive the crawler code in the form of a string returned by the cloud.

[0056] In the implementation of this example, the crawler code in the cloud can be saved as a string. After the cloud determines the matching crawler code, it can be sent to the target application, and then the target application receives the crawler code in the form of a string returned by the cloud. Among them, the cloud can send the crawler code to the target application through a network device (such as a router, etc.).

[0057] In step S230, convert the crawler code into a script code block that matches the target web page.

[0058] In the implementation of this example, the target application can convert the crawler code into a script code block that matches the target web page (i.e., is compatible with the target web page) according to the characteristics of the target web page and the data crawling requirements. For example, in the crawler code, add the web page feature information required for data crawling in the target web page and form a script code block with the object corresponding to the data information.

[0059] In step S240, inject the script code block into the target web page for data collection.

[0060] In the implementation of this example, the script code is adapted to the web page rules of the target web page and matches the target web page (i.e., is compatible with the target web page). The target application injects the script code block that is adapted to the web page rules of the target web page and matches the target web page (i.e., is compatible with the target web page) into the target web page, and can effectively crawl data such as audio and video from the target web page through the script code block to achieve data collection.

[0061] In one embodiment, the script code block is marked with a specific mark; in step S240, the injecting the script code block into the target web page for data collection includes:

[0062] Detect whether there is a preset script code block corresponding to the specific mark in the target web page; when there is no preset script code block corresponding to the specific mark in the target web page, inject the script code block into the target web page for data collection.

[0063] In the cloud, a specific mark can be marked for the crawler code, and each crawler code is marked with a corresponding specific mark. Then the script code block will carry this specific mark accordingly. When there is no preset script code block corresponding to this specific mark in the target web page, it means that the target web page needs the script code block corresponding to this specific mark to perform data collection. Then, injecting the script code block into the target web page at this time can perform data collection. On the contrary, if there is a preset script code block corresponding to the specific mark in the target web page, information injection can be selected first (such as choosing not to inject or replacing the preset script code block with a new script code block).

[0064] To facilitate better implementation of the data collection method provided in the embodiments of the present application, the embodiments of the present application also provide a data collection device based on the above data collection method. The meanings of the terms are the same as those in the above data collection method, and the specific implementation details can be referred to the description in the method embodiments. Figure 3 The block diagram of a data collection device according to an embodiment of the present application is shown.

[0065] As Figure 3 shown, the data collection device 300 may include a sending module 310, a receiving module 320, a conversion module 330, and an injection module 340.

[0066] The sending module 310 may be used to send a code acquisition request to the cloud when receiving a data collection instruction, so as to trigger the cloud to determine the crawler code matching the target application; the receiving module 320 may be used to receive the crawler code in the form of a string returned by the cloud; the conversion module 330 may be used to convert the crawler code into a script code block matching the target web page; the injection module 340 may be used to inject the script code block into the target web page for data collection.

[0067] In some embodiments of the present application, the device further includes: a change information acquisition module, configured to acquire web page rule change information in the target web page; a change judgment module, configured to determine whether to acquire an updated crawler code according to the web page rule change information; a request generation module, configured to generate the code acquisition request when it is determined that an updated crawler code needs to be acquired.

[0068] In some embodiments of the present application, the script code block is marked with a specific mark; the injection module 340 includes: a detection unit, configured to detect whether a preset script code block corresponding to the specific mark exists in the target web page; an injection unit, configured to inject the script code block into the target web page for data collection when the preset script code block corresponding to the specific mark does not exist in the target web page.

[0069] In some embodiments of the present application, the request generation module includes: an application information acquisition unit, configured to acquire application information corresponding to the target application; a code generation unit, configured to generate the code acquisition request carrying the application information, where the application information is used for the cloud to determine the crawler code matching the target application.

[0070] In some embodiments of the present application, the application information acquisition unit is configured to: send the web page rule change information in the target web page to a specific authorized terminal, so that the specific authorized terminal parses the crawling rule based on the web page rule change information and generates code authorization information matching the web page rule change information; receive the code authorization information as the application information corresponding to the target application.

[0071] In some embodiments of the present application, the application information acquisition unit is configured to: acquire the application description information of the target application as the application information matching the target application.

[0072] In some embodiments of the present application, the sending module 310 includes: a request sending unit, configured to send the acquisition request carrying the application information to the cloud, so as to trigger the cloud to determine the crawler code corresponding to the application information as the crawler code matching the target application.

[0073] In this way, based on the data acquisition device 300, when data is collected, the terminal can obtain the matching crawler code from the cloud based on the target application, and then convert the crawler code into a script code block matching the target web page and inject it into the target web page for data collection, effectively avoiding the problem of data collection failure caused by web page rule changes and improving the reliability of data collection.

[0074] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0075] In addition, an embodiment of the present application further provides an electronic device, which can be a terminal or a server. As Figure 4 shown, it shows a schematic structural diagram of the electronic device involved in the embodiment of the present application. Specifically:

[0076] The electronic device may include a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input unit 404 and other components. Those skilled in the art can understand that Figure 4 the structural diagram of the electronic device shown in does not constitute a limitation on the electronic device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements. Among them:

[0077] The processor 401 is the control center of the electronic device, connecting various parts of the entire computer device through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 402, and by invoking the data stored in the memory 402, it executes various functions of the computer device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interfaces, and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 401 either.

[0078] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.); the data storage area can store data created according to the use of the computer device. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. Correspondingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0079] The electronic device further includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0080] The electronic device may further include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0081] Although not shown, the electronic device may further include a display unit, etc., which will not be elaborated here. Specifically, in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 will run the computer programs stored in the memory 402 to implement various functions.

[0082] In one embodiment, the target application on the electronic device can be executed by the processor 401 as follows: when receiving a data collection instruction, send a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application; receive the crawler code in the form of a string returned by the cloud; convert the crawler code into a script code block matching the target web page; and inject the script code block into the target web page for data collection.

[0083] In some embodiments of the present application, before sending the code acquisition request to the cloud, the processor 401 can execute: obtain the web page rule change information in the target web page; determine whether an updated crawler code needs to be obtained according to the web page rule change information; and when it is determined that an updated crawler code needs to be obtained, generate the code acquisition request.

[0084] In some embodiments of the present application, the script code block is marked with a specific mark; when injecting the script code block into the target web page for data collection, the processor 401 can execute: detect whether a preset script code block corresponding to the specific mark exists in the target web page; and when the preset script code block corresponding to the specific mark does not exist in the target web page, inject the script code block into the target web page for data collection.

[0085] In some embodiments of the present application, when generating the code acquisition request, the processor 401 can execute: obtain the application information corresponding to the target application; and generate the code acquisition request carrying the application information, where the application information is used for the cloud to determine the crawler code matching the target application.

[0086] In some embodiments of the present application, when obtaining the application information corresponding to the target application, the processor 401 can execute: send the web page rule change information in the target web page to a specific authorized terminal, so that the specific authorized terminal parses the crawling rule based on the web page rule change information and generates code authorization information matching the web page rule change information; and receive the code authorization information as the application information corresponding to the target application.

[0087] In some embodiments of the present application, when obtaining the application information matching the target application, the processor 401 can execute: obtain the application description information of the target application as the application information matching the target application.

[0088] In some embodiments of the present application, when sending a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application, the processor 401 may execute: sending the acquisition request carrying the application information to the cloud to trigger the cloud to determine the crawler code corresponding to the application information as the crawler code matching the target application.

[0089] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a computer program or by controlling relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0090] Therefore, the embodiments of the present application further provide a storage medium in which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any of the methods provided by the embodiments of the present application.

[0091] Among them, the storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0092] Since the computer program stored in the storage medium can execute the steps in any of the methods provided by the embodiments of the present application, the beneficial effects that can be achieved by the methods provided by the embodiments of the present application can be realized. For details, please refer to the previous embodiments and will not be elaborated here.

[0093] After considering the specification and practicing the disclosed embodiments herein, those skilled in the art will readily conceive of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application.

[0094] It should be understood that the present application is not limited to the embodiments described above and shown in the drawings, but various modifications and changes can be made without departing from its scope.

Claims

1. A data acquisition method, characterized in that, Applied to a target application, including: When receiving a data collection instruction, send a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application; Receive the crawler code in the form of a string returned by the cloud; According to the characteristics of the target web page and the data crawling requirements, convert the crawler code into a script code block matching the target web page; Inject the script code block into the target web page for data collection; Before sending the code acquisition request to the cloud, the method further includes: Obtain the web page rule change information in the target web page, where the web page rule change information is information describing whether the rules in the web page have changed; Determine whether an updated crawler code needs to be obtained according to the web page rule change information; When it is determined that an updated crawler code needs to be obtained, generate the code acquisition request, and the application information is carried in the code acquisition request, and the application information is used by the cloud to determine the crawler code matching the target application; The acquisition method of the application information includes: sending the web page rule change information in the target web page to a specific authorized terminal, so that the specific authorized terminal parses the crawling rules based on the web page rule change information to generate code authorization information matching the web page rule change information; receive the code authorization information and use the code authorization information as the application information.

2. The method according to claim 1, characterized in that, The script code block is marked with a specific mark; injecting the script code block into the target web page for data collection includes: Detect whether there is a preset script code block corresponding to the specific mark in the target web page; When there is no preset script code block corresponding to the specific mark in the target web page, inject the script code block into the target web page for data collection.

3. The method according to claim 1, wherein Generating the code acquisition request includes: Obtain the application information corresponding to the target application; Generate the code acquisition request carrying the application information.

4. The method according to claim 3, wherein Obtaining the application information matching the target application includes: Obtain the application description information of the target application as the application information matching the target application.

5. The method according to any one of claims 3 to 4, characterized in that Sending a code acquisition request to the cloud to trigger the cloud to determine the crawler code matching the target application includes: Send the acquisition request carrying the application information to the cloud to trigger the cloud to determine the crawler code corresponding to the application information as the crawler code matching the target application.

6. A data acquisition device, characterized in that, Applied to a target application, the device includes: A sending module, configured to send a code acquisition request to the cloud when receiving a data collection instruction to trigger the cloud to determine the crawler code matching the target application; A receiving module, configured to receive the crawler code in the form of a string returned by the cloud; A conversion module, configured to convert the crawler code into a script code block matching the target web page according to the characteristics of the target web page and the data crawling requirements; An injection module, configured to inject the script code block into the target web page for data collection; A change information acquisition module, which is used to acquire the web page rule change information in the target web page, and the web page rule change information is information describing whether the rules in the web page have changed; A change judgment module, which is used to determine whether it is necessary to acquire an updated crawler code according to the web page rule change information; A request generation module, which is used to generate the code acquisition request when it is determined that an updated crawler code needs to be acquired, and the application information is carried in the code acquisition request, and the application information is used for the cloud to determine the crawler code matching the target application; An application information acquisition unit, which is used to send the web page rule change information in the target web page to a specific authorized terminal, so that the specific authorized terminal parses the crawling rules based on the web page rule change information and generates code authorization information matching the web page rule change information; receive the code authorization information and use the code authorization information as the application information.

7. A storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, Comprising: A memory storing a computer program; A processor that reads the computer program stored in the memory to execute the method according to any one of claims 1 to 5.

Images