Data collection method, system, electronic device and storage medium

Abstract

The present invention provides a data collection method, including: obtaining a data sending request sent by a target client, the data sending request including a collection manufacturer, AppID, token and timestamp; obtaining the corresponding authentication parameters based on the collection manufacturer, and authenticating the target client based on the authentication parameters, AppID, token and timestamp; when the authentication result is valid, obtaining the collection task corresponding to the collection manufacturer; collecting the data of the target client based on the collection task, and storing the data based on a preset storage mapping rule. The present invention authenticates the data sending request sent by the target client, collects the data based on a standard collection task, and configures the data for storage based on the preset storage mapping rule in the data storage link, thereby realizing the standardized establishment of Http data collection, greatly improving the adaptability of http data collection, and reducing the labor cost of data collection.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and more specifically, to a data acquisition method, system, electronic equipment and storage medium. Background Art

[0002] HTTP protocol is the abbreviation of Hyper Text Transfer Protocol, which is a transmission protocol used to transfer hypertext from the server to the local client browser. HTTP protocol works on the client-server architecture and adopts the request / response model. The browser, as the HTTP client, sends all requests to the WEB server through the URL. The web server sends a response message to the client based on the received request. The client sends a request message to the server, which contains the request method, URL, protocol version, request header and request data. The server responds with a status line, and the response content includes the protocol version, success or error code, server information, response header and response data.

[0003] Every time the client sends a request, the message contains the request method, URL, protocol version, request header and request data, and the server's response includes the protocol version, success or error code, server information, response header and response data. With so many inconsistent request parameters and response parameters, and each parameter and response parameter is inconsistent, we need to adapt through different interfaces to collect data. If there are many types of collected data, a large number of interface programs are needed to support it. Therefore, how to standardize Http data collection and improve the adaptability of http data collection is an urgent problem to be solved. Summary of the invention

[0004] The present invention aims at the technical problems existing in the prior art and provides a data collection method, system, electronic device and storage medium to solve the problem of how to standardize Http data collection and improve the adaptability of http data collection.

[0005] According to a first aspect of the present invention, there is provided a data collection method, comprising:

[0006] Obtain a data sending request sent by the target client, wherein the data sending request includes the collection manufacturer, AppID, token and timestamp;

[0007] Acquire the corresponding authentication parameters based on the acquisition manufacturer, and authenticate the target client based on the authentication parameters, the AppID, the token and the timestamp;

[0008] When the authentication result is valid, obtaining the collection task corresponding to the collection manufacturer;

[0009] The data of the target client is collected based on the collection task, and the data is stored in the database based on a preset storage mapping rule.

[0010] Based on the above technical solution, the present invention can also make the following improvements.

[0011] Preferably, the authentication parameters are authentication address, request method, parameter transfer, request header, response format, authentication time limit and complete authentication value.

[0012] Preferably, the step of authenticating the target client based on the authentication parameter, the AppID, the token and the timestamp includes:

[0013] When the time difference between the timestamp and the current system time is within the authentication time limit, the authentication address, the AppID, the parameter transfer and the timestamp are encrypted using a preset encryption algorithm, and the encrypted result is matched with the token. When the matching results are consistent, the authentication result is determined to be valid.

[0014] Preferably, the step of authenticating the target client based on the authentication parameter, the AppID, the token and the timestamp further includes:

[0015] When the time difference between the timestamp and the current system time is not within the authentication time limit, the authentication result is determined to be invalid.

[0016] Preferably, when the time difference between the timestamp and the current time of the system is not within the authentication time limit, after the step of determining that the authentication result is invalid, the method further comprises:

[0017] The authentication result is returned to the target client, and the data sending request is rejected.

[0018] Preferably, the collection task includes: task identification, task name, collection manufacturer, collection method, collection interval, collection address, request method, parameter transfer method, request method, request header, response parsing format, whether to paginate and storage method.

[0019] Preferably, the preset storage mapping rules include: interface field, storage field, data type, field length, whether it is a primary key and field description.

[0020] According to a second aspect of the present invention, there is provided a data acquisition system, comprising:

[0021] A request acquisition module is used to acquire a data sending request sent by a target client, wherein the data sending request includes a collection manufacturer, AppID, token and timestamp;

[0022] An authority identification module, used to obtain the corresponding authentication parameters based on the acquisition manufacturer, and authenticate the target client based on the authentication parameters, the AppID, the token and the timestamp;

[0023] A task acquisition module, used to obtain the collection task corresponding to the collection manufacturer when the authentication result is valid;

[0024] The data storage module is used to collect the data of the target client based on the collection task, and store the data in the storage based on a preset storage mapping rule.

[0025] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the processor is configured to implement the steps of any one of the data collection methods in the first aspect when executing a computer management program stored in the memory.

[0026] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium on which a computer management program is stored, and when the computer management program is executed by a processor, the steps of any data collection method in the first aspect are implemented.

[0027] The present invention provides a data collection method, system, electronic device and storage medium, the method comprising: obtaining a data sending request sent by a target client, the data sending request including a collection manufacturer, AppID, token and timestamp; obtaining the corresponding authentication parameters based on the collection manufacturer, authenticating the target client based on the authentication parameters, AppID, token and timestamp; when the authentication result is valid, obtaining the collection task corresponding to the collection manufacturer; collecting the data of the target client based on the collection task, and storing the data based on a preset storage mapping rule. The present invention authenticates the data sending request sent by the target client, thereby reducing the access pressure of the illegal data sending request on the server, and obtains the corresponding collection task through different collections, collects the data based on the standard collection task, thereby standardizing the traditional data collection work, reducing the development of additional interface codes, and configuring the data for storage based on the preset storage mapping rule in the data storage link, thereby standardizing the non-standard data for storage, improving the adaptability of the collection method, and then realizing the standardized establishment of Http data collection, greatly improving the adaptability of http data collection, and reducing the labor cost of data collection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flow chart of a data collection method provided by the present invention;

[0029] Figure 2 A schematic diagram of the data acquisition parameter configuration provided by the present invention;

[0030] Figure 3 A schematic diagram of a data acquisition manufacturer configuration page provided by the present invention;

[0031] Figure 4 A schematic diagram of a data collection rule configuration page provided by the present invention;

[0032] Figure 5 A structural schematic diagram of a data acquisition system provided by the present invention;

[0033] Figure 6 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0034] Figure 7 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0035] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0036] Figure 1 A data collection method flow chart provided by the present invention is as follows: Figure 1 As shown, the method includes:

[0037] Step S100: Obtain a data sending request sent by a target client, wherein the data sending request includes a collection manufacturer, AppID, token, and timestamp;

[0038] It should be noted that the execution subject of the method of this embodiment can be a computer terminal device with data processing, network communication and program running functions, such as a computer, a tablet computer, etc.; it can also be a server device with the same or similar functions, or a cloud server with similar functions, and this embodiment does not limit this. For ease of understanding, this embodiment and the following embodiments will be described using a server device as an example.

[0039] It can be understood that the above-mentioned target client acts as a data sending end, and only one target client can send a data sending request to the server at the same time, or there can be multiple target clients sending data sending requests to the server. The administrator can set the number of concurrent target clients on the server side according to the performance of the server. The number of concurrent targets is set by the administrator, and this embodiment does not impose any restrictions on this.

[0040] It should be understood that the target client may also store in advance the authentication address of the server and its corresponding user name and password. The acquisition manufacturer may be used to identify the manufacturer to which the data on the target client belongs. The AppID may be used to identify a specific application of the acquisition manufacturer. Since the same manufacturer may have multiple applications on the same client, the AppID is required to distinguish between the applications. The timestamp may be the current system time of the data sending request obtained by the target client. The token may be generated by concatenating the authentication address, password, AppID and timestamp on the target client and then using an encryption algorithm to generate a token. The encryption algorithm may be an existing symmetric / asymmetric encryption algorithm, which is not limited in this embodiment.

[0041] In a specific implementation, the server receives the data sending request sent by the target client, and obtains the collection manufacturer, AppID, token and timestamp by parsing the parameters therein.

[0042] Step S200: obtaining the corresponding authentication parameters based on the acquisition manufacturer, and authenticating the target client based on the authentication parameters, the AppID, the token and the timestamp;

[0043] It is understandable that the above authentication parameters can be stored in the server side, mapped with each target client through the above acquisition manufacturer, and when there are multiple applications on the same target client that need to collect data, the above AppID can also be used for mapping. The above authentication parameters at least include: authentication address, request method, parameter transfer, request header, response format, authentication time limit and complete authentication value, which is not limited in this embodiment.

[0044] The meanings of the various parameters in the authentication parameters are as follows:

[0045] Authentication address: URL address used to obtain collection authorization, generally the login interface address, etc.

[0046] Request method: the request method of the request interface, including get or post;

[0047] Parameter passing: whether the request parameter is in the URL or in the body;

[0048] Request parameters: json format, used to send parameters to the authentication interface, such as username, password, etc. For example: {"clientFlag":0,"password":"123456","username":"nanqi"};

[0049] Request header: json format, used to send request header information to the authentication interface, similar to request parameters. If there are parameters, add them, otherwise leave them blank;

[0050] Response parsing format: authentication response format, generally the values ​​corresponding to keywords such as token and result, and also supports the corresponding values ​​of other fields, such as "xxxxxxxx". The return value example in the figure above is: {"result":"xxxxxxxx"};

[0051] Authentication validity period: the validity period of authentication. If the validity period exceeds, you need to request again to obtain the authentication value again;

[0052] Complete authentication value: The variable $result indicates the authentication value returned by the interface. For example, if the complete token of the backend interface is "basic xxxxxxxx", it can be configured as "basic$result".

[0053] Further, the step of authenticating the target client based on the authentication parameter, the AppID, the token and the timestamp includes:

[0054] Step S201: When the time difference between the timestamp and the current system time is within the authentication time limit, the authentication address, the AppID, the parameter transfer and the timestamp are encrypted using a preset encryption algorithm, and the encrypted result is matched with the token. When the matching results are consistent, the authentication result is determined to be valid.

[0055] It can be understood that the above-mentioned preset encryption algorithm is the same encryption algorithm used by the above-mentioned target client, which is used to encrypt using the same parameters and the same encryption algorithm, thereby detecting the token of the target client.

[0056] Furthermore, the step of authenticating the target client based on the authentication parameter, the AppID, the token and the timestamp further includes:

[0057] Step S202: When the time difference between the timestamp and the current system time is not within the authentication time limit, the authentication result is determined to be invalid.

[0058] Furthermore, when the time difference between the timestamp and the current time of the system is not within the authentication time limit, after the step of determining that the authentication result is invalid, the method further includes:

[0059] Step S203: Return the authentication result to the target client and reject the data sending request.

[0060] In the specific implementation, the target client concatenates the token, AppID, and timestamp into the URL and sends it to the server. After receiving the interface request from the caller, the server extracts the token, AppID, and timestamp from the request. The server first checks whether the passed timestamp and the current time are within the token expiration time window. If the expiration time has exceeded, the interface call authentication fails and the interface call request is rejected. If the token verification has not expired, the server will take out the password corresponding to the AppID from its own storage, and generate another token through the same token generation algorithm to match the token passed by the caller; if they are consistent, the authentication is successful and the interface call is allowed, otherwise the interface call is rejected.

[0061] In this embodiment, due to the development of information technology in actual situations, in the actual data set collection process, since projects at different information stages may be customized and developed by different manufacturers, when collecting data from different manufacturers, it is necessary to connect to different manufacturer interfaces, and different request parameters, request headers, response formats, authentication time limits, and authentication return result formats may all be different. By performing optional configuration on request parameters and request methods, adapting parameter transfer and request headers to the standard json format, and performing keyword matching on the response parsing format and complete authentication value, the authentication process is standardized.

[0062] Step S300: when the authentication result is valid, obtaining the collection task corresponding to the collection manufacturer;

[0063] It is understandable that the collection task needs to be configured before performing the collection task, and the collection task parameters include: task identifier, task name, collection manufacturer, collection method, collection interval, collection address, request method, parameter passing method, request method, request header, response parsing format, whether to paginate and storage method.

[0064] The above collection task parameters are explained as follows:

[0065] Collection interval (minutes): task collection cycle, automatically scheduled every few minutes;

[0066] Collection address: collection interface address;

[0067] Request method: the request method of the collection interface, including get or post;

[0068] Parameter transfer method: corresponding to the request parameter transfer method, optional URL splicing or body transmission;

[0069] Request parameters: JSON format, used to send parameters to the collection interface, such as paging identifiers or filter conditions, with a built-in $authentication variable to represent the complete authentication value;

[0070] Request header: json format, used for request header information sent to the collection interface, with built-in $authentication variable, indicating the complete authentication value;

[0071] Response parsing format: Parsed data format. If the interface returns directly in the [{},{}] array format, it will be empty.

[0072] Whether to paginate: Whether the interface opens pagination query, which can improve the collection efficiency and multi-threaded pagination collection; PageNoCode previous page field identifier, pageNo represents the current page field of the interface, PageSizeCode per page number field identifier, such as pageSize represents the interface field, PageSize per page number when paginating, number; total page number parsing format interface returns the total page number field format;

[0073] Storage method: Full storage means that each collection clears the previously collected data; incremental collection means that each collected data is incremented, no new data is added, and if it exists (with the same primary key), it is modified. There is no data deletion function.

[0074] In the specific implementation, this embodiment standardizes the collection tasks, mainly including the adaptation of the collection interface, through the selection and adaptation of the request method and parameter passing method, the standard json format adaptation of the request method and request header, and the adaptation of keywords and return formats for whether the collection interface is paginated, including the paging code, the paging size, the total number of rows of data, etc. At the same time, the selection and adaptation of the storage method, including incremental or full storage.

[0075] Step S400: collecting data of the target client based on the collection task, and storing the data in a database based on a preset database mapping rule.

[0076] It is understandable that the above-mentioned preset warehousing rules can be configured in advance on the server side for each client to achieve standardization of information entry, including: standardization of fields. The fields returned by each interface are different. We can configure the fields to be entered into the warehouse to achieve configurable entry fields. According to needs, valuable fields can be selectively stored; standardization of field types. All collected fields support text and number formats to avoid errors due to incorrect formats; standardization of relationships. The parameters passed through the configuration interface are used to establish a relationship with the data we actually want, which can be 1 to many or 1 to 1; standardization of primary key information. The primary key can be configured according to the characteristics of the data, thereby realizing the judgment and processing of duplicate data and invalid data.

[0077] In a specific implementation, this embodiment completes data collection by collecting target client data, matching the data collection fields with the fields in the preset storage mapping rules based on the data collection fields, and then storing the data after the match is successful.

[0078] It is understandable that, based on the defects in the background technology, an embodiment of the present invention proposes a data collection method. The method includes: obtaining a data sending request sent by a target client, the data sending request includes a collection manufacturer, AppID, token and timestamp; obtaining the corresponding authentication parameters based on the collection manufacturer, authenticating the target client based on the authentication parameters, AppID, token and timestamp; when the authentication result is valid, obtaining the collection task corresponding to the collection manufacturer; collecting the data of the target client based on the collection task, and storing the data in the database based on the preset database mapping rules. The present invention authenticates the data sending request sent by the target client, thereby reducing the access pressure of illegal data sending requests on the server, and obtains corresponding collection tasks through different collections, collects data based on standard collection tasks, thereby standardizing traditional data collection work and reducing the development of additional interface codes. The data warehousing link configures data entry based on preset warehousing mapping rules, thereby standardizing non-standard data entry, improving the adaptability of the collection method, and further realizing the standardized establishment of Http data collection, greatly improving the adaptability of http data collection, and reducing the labor cost of data collection.

[0079] In a possible implementation manner, this embodiment also provides full process parameter configuration during data collection, see Figure 2 , Figure 2 A schematic diagram of the data acquisition parameter configuration provided by the present invention; Figure 2In the configuration, you need to complete the five parts of collection manufacturer, authentication parameters, collection tasks, standard collection and collection storage. Among them, the collection manufacturer includes manufacturer ID, manufacturer name and protocol type; authentication parameters include authentication address, request method, parameter transmission, request header, response format, authentication time limit and complete authentication value; collection tasks include task ID, task name and equipment manufacturer; standard collection includes collection interval, collection address, request method, parameter transmission, request header, response format, paging and storage method; collection storage includes primary key configuration, field mapping and field length. The above data collection parameters are configured in the system configuration page, see Figure 3 and Figure 4 .

[0080] This embodiment, by realizing the standardization of client authentication, collection tasks, and data storage during the data collection process, can achieve data collection on different platforms, different manufacturers, and different interfaces, and play a good role in standardized collection. At the same time, it can also provide support for new additions and modifications to data in the collection and storage link by combining the data characteristics of the business system, thereby realizing the addition, deletion, and modification of data through configuration. This embodiment reduces the development work of interface code by standardizing the various processes of data collection, and configures parameters for various situations of a data collection, including the authentication configuration of the collection, the parameter transmission configuration of the collection, and the storage configuration of the collection, so as to achieve a set of logic code that can cover data collection in various situations, improve code reuse, enhance the robustness of product functions, and reduce labor costs.

[0081] See also Figure 5 , Figure 5 A schematic diagram of a data acquisition system structure provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, a data collection system includes a request acquisition module 100, an authority identification module 200, a task acquisition module 300 and a data storage module 400, wherein:

[0082] The request acquisition module 100 is used to obtain the data sending request sent by the target client, and the above data sending request includes the collection manufacturer, AppID, token and timestamp; the authority identification module 200 is used to obtain the corresponding authentication parameters based on the above collection manufacturer, and authenticate the above target client based on the above authentication parameters, the above AppID, the above token and the above timestamp; the task acquisition module 300 is used to obtain the collection task corresponding to the above collection manufacturer when the authentication result is valid; the data storage module 400 is used to collect the data of the above target client based on the above collection task, and store the above data into the storage based on the preset storage mapping rules.

[0083] It can be understood that the data acquisition system provided by the present invention corresponds to the data acquisition method provided by the aforementioned embodiments, and the relevant technical features of the data acquisition system can refer to the relevant technical features of the data acquisition method, which will not be repeated here.

[0084] See also Figure 6 , Figure 6 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 6 As shown, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, the following steps are implemented:

[0085] Obtain a data sending request sent by a target client, the data sending request including a collection manufacturer, AppID, token and timestamp; obtain corresponding authentication parameters based on the collection manufacturer, and authenticate the target client based on the authentication parameters, the AppID, the token and the timestamp; when the authentication result is valid, obtain a collection task corresponding to the collection manufacturer; collect data of the target client based on the collection task, and store the data into a database based on a preset database mapping rule.

[0086] See also Figure 7 , Figure 7 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 7 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, the following steps are implemented:

[0087] Obtain a data sending request sent by a target client, the data sending request including a collection manufacturer, AppID, token and timestamp; obtain corresponding authentication parameters based on the collection manufacturer, and authenticate the target client based on the authentication parameters, the AppID, the token and the timestamp; when the authentication result is valid, obtain a collection task corresponding to the collection manufacturer; collect data of the target client based on the collection task, and store the data into a database based on a preset database mapping rule.

[0088] A data collection method, system and storage medium provided by an embodiment of the present invention include: obtaining a data sending request sent by a target client, the data sending request includes a collection manufacturer, AppID, token and timestamp; obtaining the corresponding authentication parameters based on the collection manufacturer, authenticating the target client based on the authentication parameters, the AppID, the token and the timestamp; when the authentication result is valid, obtaining the collection task corresponding to the collection manufacturer; collecting the data of the target client based on the collection task, and storing the data based on a preset storage mapping rule. The present invention authenticates the data sending request sent by the target client, thereby reducing the access pressure of the illegal data sending request on the server, and obtains the corresponding collection task through different collections, collects data based on the standard collection task, thereby standardizing the traditional data collection work, reducing the development of additional interface codes, and configuring the data for storage based on the preset storage mapping rule in the data storage link, thereby standardizing the non-standard data for storage, improving the adaptability of the collection method, and then realizing the standardized establishment of Http data collection, greatly improving the adaptability of http data collection, and reducing the labor cost of data collection.

[0089] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0090] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0094] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0095] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A data collection method, characterized in that: The method comprises: Obtain a data sending request sent by the target client, wherein the data sending request includes the collection manufacturer, AppID, token and timestamp; Based on the acquisition manufacturer, the corresponding authentication parameters are obtained, and the authentication parameters include the authentication address and the authentication time limit; Obtain the password corresponding to the AppID from the storage unit; When the time difference between the timestamp and the current system time is within the authentication time limit, the authentication address, the password, the AppID and the timestamp are concatenated and encrypted using a preset encryption algorithm to obtain an encrypted result; Matching the encrypted result with the token, and when the matching results are consistent, determining that the authentication result is valid; When the authentication result is valid, obtaining the collection task corresponding to the collection manufacturer; The data of the target client is collected based on the collection task, and the data is configured for storage based on a preset storage mapping rule. After the configuration is successful, the data is stored in the storage.

2. The data collection method according to claim 1, characterized in that: The authentication parameters also include request method, parameter transfer, request header, response format and complete authentication value.

3. The data collection method according to claim 1, characterized in that: The method further comprises: When the time difference between the timestamp and the current system time is not within the authentication time limit, the authentication result is determined to be invalid.

4. The data collection method according to claim 3, characterized in that: After the step of determining that the authentication result is invalid when the time difference between the timestamp and the current system time is not within the authentication time limit, the method further comprises: The authentication result is returned to the target client, and the data sending request is rejected.

5. The data collection method according to claim 1, characterized in that: The collection task includes: task identification, task name, collection manufacturer, collection method, collection interval, collection address, parameter transfer method, request method, request header, response parsing format, whether to paginate and storage method.

6. The data collection method according to claim 1, characterized in that: The preset storage mapping rules include: interface field, storage field, data type, field length, whether it is a primary key and field description.

7. A data acquisition system, characterized in that: include: A request acquisition module is used to acquire a data sending request sent by a target client, wherein the data sending request includes a collection manufacturer, AppID, token and timestamp; The authority identification module is used to obtain the corresponding authentication parameters based on the acquisition manufacturer, and the authentication parameters include the authentication address and the authentication time limit; obtain the password corresponding to the AppID from the storage unit; when the time difference between the timestamp and the current time of the system is within the authentication time limit, the authentication address, the password, the AppID and the timestamp are spliced, and encrypted by a preset encryption algorithm to obtain an encrypted result; the encrypted result is matched with the token, and when the matching results are consistent, the authentication result is determined to be valid; A task acquisition module, used to acquire the collection task corresponding to the collection manufacturer when the authentication result is valid; The data storage module is used to collect the data of the target client based on the collection task, and configure the data for storage based on the preset storage mapping rules. After the configuration is successful, the data is stored in the storage.

8. An electronic device, characterized in that: It comprises a memory and a processor, wherein the processor is used to implement the steps of the data collection method according to any one of claims 1 to 6 when executing a computer management program stored in the memory.

9. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by a processor, the steps of the data collection method according to any one of claims 1 to 6 are implemented.

Images