Data acquisition method and system for cross-business theme data platform

Abstract

The invention discloses a data acquisition method and system of a cross-business theme data platform, a business end sends a data request to middleware through a request interface, the middleware establishes a data task according to the type of the request interface corresponding to the data request, and updates a data task table according to the data task, and triggering the data platform to download the to-be-collected data based on the data task table, storing the downloaded to-be-collected data to a platform database by the data platform, and returning a downloading result to the middleware, and updating the data task table by the middleware according to the downloading result, and based on the updated data task table, triggering a collector to collect the to-be-collected data from the platform database and return the to-be-collected data to the service end, thereby simplifying a data interaction process between the service end and different service subject data platforms by using middleware, storing the downloaded data in the data platform, and then triggering the collector by the middleware to collect the data to the service end. The burden of the database is relieved, and the data downloading efficiency is improved, so that the data interaction efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of data collection technology, and in particular to a data collection method and system for a cross-business subject data platform. Background Art

[0002] In the prior art, when business department personnel need to analyze data across business topics, they need to make corresponding data requests on the business side. The requests will be forwarded to the data middle platform, and the data middle platform will be used to obtain the matching data synchronization table from the data middle platform database. The data snapshot information of the current business end user's associated business is stored in this synchronization table, and then based on the data synchronization strategy of the current business end user's business and associated businesses, the data snapshot information is returned to the current business end user.

[0003] In the above method, when users submit requests for the same data range at different times, the data center will also generate two identical synchronization records. A large amount of work in the process is mechanized and repetitive, resulting in low data access efficiency. In addition, for some data platforms with large amounts of market data, the data return efficiency is low. Moreover, user requested data are all stored in the data center database, which increases the database burden and affects query efficiency. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a data collection method and system across business subject data platforms, which can improve data interaction efficiency.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: A data collection method for a cross-business subject data platform, comprising the steps of: The business end sends a data request to the middleware through a request interface, wherein the data request includes the data to be collected and a data platform corresponding to the data to be collected; The middleware establishes a data task according to the type of the request interface corresponding to the data request, wherein the data task includes a task creation time, a task update time, a task status and request parameters, and updates a data task table according to the data task; The middleware triggers the data platform to download the data to be collected based on the data task table; The data platform stores the downloaded data to be collected in the platform database, and returns the download result of the data to be collected to the middleware; The middleware updates the task update time and task status of the data task in the data task table according to the download result, and triggers the collector to collect the data to be collected from the platform database and return it to the business end based on the updated data task table.

[0006] In order to solve the above technical problems, another technical solution adopted by the present invention is: A data collection system across business subject data platforms, comprising a business end, middleware and a data platform, wherein the business end comprises a first memory, a first processor and a first computer program stored in the first memory and executable on the first processor, the middleware comprises a second memory, a second processor and a second computer program stored in the second memory and executable on the second processor, the data platform comprises a third memory, a third processor and a third computer program stored in the third memory and executable on the third processor, and the first processor implements the following steps when executing the first computer program; Sending a data request to the middleware through a request interface, wherein the data request includes the data to be collected and a data platform corresponding to the data to be collected; When the second processor executes the second computer program, the following steps are implemented: Establishing a data task according to the type of the request interface corresponding to the data request, wherein the data task includes a task creation time, a task update time, a task status, and request parameters, and updating a data task table according to the data task; Triggering the data platform to download the data to be collected based on the data task table; Update the task update time and task status of the data task in the data task table according to the download result, and trigger the collector to collect the data to be collected from the platform database and return it to the business end based on the updated data task table; When the third processor executes the third computer program, the following steps are implemented: The downloaded data to be collected are stored in the platform database, and the download result of the data to be collected is returned to the middleware.

[0007] The beneficial effects of the present invention are as follows: the business end sends a data request to the middleware through a request interface, the middleware establishes a data task according to the type of request interface corresponding to the data request, and updates the data task table according to the data task, triggers the data platform to download the data to be collected based on the data task table, the data platform stores the downloaded data to be collected in the platform database, and returns the download result to the middleware, the middleware updates the task update time and task status of the data task in the data task table according to the download result, and triggers the collector to collect the data to be collected from the platform database and returns it to the business end based on the updated data task table, thereby simplifying the data interaction process between the business end and different business theme data platforms by using the middleware, because the market data volume is large, the downloaded data is first stored in the data platform, rather than directly stored in the business end, and then the middleware triggers the collector to collect it to the business end, which alleviates the database burden and improves the data download efficiency, thereby improving the data interaction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A flowchart of the steps of a data collection method for a cross-business subject data platform according to an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a data collection system for a cross-business subject data platform according to an embodiment of the present invention; Figure 3 This is a schematic diagram of data collection in a data collection method for a cross-business subject data platform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0009] Please refer to Figure 1 , a data collection method across business subject data platforms, comprising the steps of: The business end sends a data request to the middleware through a request interface, wherein the data request includes the data to be collected and a data platform corresponding to the data to be collected; The middleware establishes a data task according to the type of the request interface corresponding to the data request, wherein the data task includes a task creation time, a task update time, a task status and request parameters, and updates a data task table according to the data task; The middleware triggers the data platform to download the data to be collected based on the data task table; The data platform stores the downloaded data to be collected in the platform database, and returns the download result of the data to be collected to the middleware; The middleware updates the task update time and task status of the data task in the data task table according to the download result, and triggers the collector to collect the data to be collected from the platform database and return it to the business end based on the updated data task table.

[0010] From the above description, it can be seen that the beneficial effects of the present invention are: the business end sends a data request to the middleware through the request interface, the middleware establishes a data task according to the type of request interface corresponding to the data request, and updates the data task table according to the data task, triggers the data platform to download the data to be collected based on the data task table, the data platform stores the downloaded data to be collected in the platform database, and returns the download result to the middleware, the middleware updates the task update time and task status of the data task in the data task table according to the download result, and triggers the collector to collect the data to be collected from the platform database and returns it to the business end based on the updated data task table, thereby using the middleware to simplify the data interaction process between the business end and different business theme data platforms. Due to the large amount of market data, the downloaded data is first stored in the data platform instead of being directly stored in the business end, and then the middleware triggers the collector to collect it to the business end, which alleviates the database burden and improves the data download efficiency, thereby improving the data interaction efficiency.

[0011] Furthermore, the middleware establishes a data task according to the type of the request interface corresponding to the data request, including: The middleware determines whether the request parameters corresponding to the unfinished data tasks in the data task table contain the request parameters corresponding to the data request. If so, the data request is not processed. If not, a data task is established according to the type of the request interface corresponding to the data request. The request parameters include start time, end time, site, data type and report type.

[0012] From the above description, it can be seen that when the request parameters corresponding to the unfinished data tasks in the data task table contain request parameters corresponding to the data request, it means that the business-end user has submitted requests for the same data range at different times, and the middleware will not process the data request, thereby effectively avoiding repeated data downloads and affecting data interaction efficiency.

[0013] Furthermore, the middleware triggers the data platform to download the data to be collected based on the data task table, including: The middleware triggers the data platform to download the data to be collected according to the task order in the data task table.

[0014] From the above description, it can be seen that the middleware triggers the data platform to download the data to be collected according to the task order in the data task table, ensuring the orderliness of data download.

[0015] Furthermore, the middleware updates the task update time and task status of the data task in the data task table according to the download result, including: If the download result is a successful download, the middleware updates the task status of the data task corresponding to the data request in the data task table to completed, and updates the task update time of the data task; If the download result is a download failure, the middleware updates the task status of the data task corresponding to the data request in the data task table to incomplete, updates the task update time of the data task, and returns to execute the step of triggering the data platform to download the data to be collected based on the data task table.

[0016] From the above description, it can be seen that the task status and task update time of the data task in the data task table are updated according to the download result. When a data task fails to be processed, it waits for the next round of processing to ensure the effectiveness of data interaction.

[0017] Further, the triggering of the collector to collect the to-be-collected data from the platform database and return it to the service end based on the updated data task table includes: The middleware determines all completed data tasks from the updated data task table, and triggers the collector to collect the data to be collected from the platform database according to the task sorting time; The middleware extracts, cleans and converts the data to be collected to obtain processed data; The middleware determines incremental data from the processed data according to the data in the business end database, and stores the incremental data in the business end database of the business end.

[0018] From the above description, it can be seen that after the middleware collects the data to be collected from the platform database, it extracts, cleans and converts the data to obtain the processed data, and also determines the incremental data, and stores the incremental data in the business-end database of the business end, avoiding repeated data collection. Through data extraction, cleaning and conversion, the data that finally reaches the business end not only meets business needs, but also improves data query efficiency, thereby improving data interaction efficiency.

[0019] Please refer to Figure 2Another embodiment of the present invention provides a data collection system across business subject data platforms, including a business end, a middleware and a data platform, wherein the business end includes a first memory, a first processor and a first computer program stored in the first memory and executable on the first processor, the middleware includes a second memory, a second processor and a second computer program stored in the second memory and executable on the second processor, the data platform includes a third memory, a third processor and a third computer program stored in the third memory and executable on the third processor, and the first processor implements the following steps when executing the first computer program; Sending a data request to the middleware through a request interface, wherein the data request includes the data to be collected and a data platform corresponding to the data to be collected; When the second processor executes the second computer program, the following steps are implemented: Establishing a data task according to the type of the request interface corresponding to the data request, wherein the data task includes a task creation time, a task update time, a task status, and request parameters, and updating a data task table according to the data task; Triggering the data platform to download the data to be collected based on the data task table; Update the task update time and task status of the data task in the data task table according to the download result, and trigger the collector to collect the data to be collected from the platform database and return it to the business end based on the updated data task table; When the third processor executes the third computer program, the following steps are implemented: The downloaded data to be collected are stored in the platform database, and the download result of the data to be collected is returned to the middleware.

[0020] From the above description, it can be seen that the beneficial effects of the present invention are: the business end sends a data request to the middleware through the request interface, the middleware establishes a data task according to the type of request interface corresponding to the data request, and updates the data task table according to the data task, triggers the data platform to download the data to be collected based on the data task table, the data platform stores the downloaded data to be collected in the platform database, and returns the download result to the middleware, the middleware updates the task update time and task status of the data task in the data task table according to the download result, and triggers the collector to collect the data to be collected from the platform database and returns it to the business end based on the updated data task table, thereby using the middleware to simplify the data interaction process between the business end and different business theme data platforms. Due to the large amount of market data, the downloaded data is first stored in the data platform instead of being directly stored in the business end, and then the middleware triggers the collector to collect it to the business end, which alleviates the database burden and improves the data download efficiency, thereby improving the data interaction efficiency.

[0021] Furthermore, when the second processor executes the second computer program, the following steps are also implemented: Determine whether the request parameters corresponding to the unfinished data tasks in the data task table contain the request parameters corresponding to the data request. If so, do not process the data request. If not, create a data task according to the type of the request interface corresponding to the data request. The request parameters include start time, end time, site, data type and report type.

[0022] From the above description, it can be seen that when the request parameters corresponding to the unfinished data tasks in the data task table contain request parameters corresponding to the data request, it means that the business-end user has submitted requests for the same data range at different times, and the middleware will not process the data request, thereby effectively avoiding repeated data downloads and affecting data interaction efficiency.

[0023] Furthermore, when the second processor executes the second computer program, the following steps are also implemented: The data platform is triggered to download the data to be collected according to the task order in the data task table.

[0024] From the above description, it can be seen that the middleware triggers the data platform to download the data to be collected according to the task order in the data task table, ensuring the orderliness of data download.

[0025] Furthermore, when the second processor executes the second computer program, the following steps are also implemented: If the download result is a successful download, the task status of the data task corresponding to the data request in the data task table is updated to completed, and the task update time of the data task is updated; If the download result is a download failure, the task status of the data task corresponding to the data request in the data task table is updated to incomplete, the task update time of the data task is updated, and the step of triggering the data platform to download the data to be collected based on the data task table is returned to execute.

[0026] From the above description, it can be seen that the task status and task update time of the data task in the data task table are updated according to the download result. When a data task fails to be processed, it waits for the next round of processing to ensure the effectiveness of data interaction.

[0027] Furthermore, when the second processor executes the second computer program, the following steps are also implemented: Determine all completed data tasks from the updated data task table, and trigger the collector to collect the data to be collected from the platform database according to the task sorting; Extracting, cleaning and converting the data to be collected to obtain processed data; Determine incremental data from the processed data according to data in the business end database, and store the incremental data in the business end database of the business end.

[0028] From the above description, it can be seen that after the middleware collects the data to be collected from the platform database, it extracts, cleans and converts the data to obtain the processed data, and also determines the incremental data, and stores the incremental data in the business-end database of the business end, avoiding repeated data collection. Through data extraction, cleaning and conversion, the data that finally reaches the business end not only meets business needs, but also improves data query efficiency, thereby improving data interaction efficiency.

[0029] The data collection method and terminal of the cross-business subject data platform described above in the present invention can be applied to cross-business subject data analysis and integration scenarios, which are described below through specific implementation methods: Please refer to Figure 1 and Figure 3 , Embodiment 1 of the present invention is: A data collection method for a cross-business subject data platform, comprising the steps of: S1. The business end sends a data request to the middleware through a request interface API, where the data request includes the data to be collected and a data platform corresponding to the data to be collected.

[0030] Since there are multiple business subject data platforms, including sales domain data platform, advertising domain data platform and activity domain data platform, when users request data through the business end, they need to determine the data to be collected and the corresponding data platform, so that the middleware can collect data from the corresponding data platform.

[0031] S2. The middleware creates a data task according to the type of the request interface corresponding to the data request, and updates the data task table according to the data task, such as Figure 3 As shown, specifically including S21-S22: S21. The middleware determines whether the request parameters corresponding to the unfinished data tasks in the data task table contain the request parameters corresponding to the data request. If so, the data request is not processed. If not, a data task is established according to the type of the request interface corresponding to the data request.

[0032] Among them, the data task includes task creation time, task update time, task status and request parameters; the request parameters include start_date (start time), end_date (end time), account_station_id (station), record_type (data type) and ad_type (report type), which are determined by the user on the business side; the type of the request interface includes activity type, product type or order type, etc. Only when all request parameters corresponding to the unfinished data task are consistent with all request parameters corresponding to the data request, it is considered to exist. Data of different request interface types are stored in different task tables, and the corresponding task table will be updated according to the report type requested by the user. S22, update the data task table according to the data task.

[0033] S3. The middleware triggers the data platform to download the data to be collected based on the data task table.

[0034] Specifically, the middleware triggers the data platform to download the data to be collected according to the task order in the data task table.

[0035] S4, the data platform stores the downloaded data to be collected in the platform database, and returns the download result of the data to be collected to the middleware, such as Figure 3 shown.

[0036] S5, the middleware updates the task update time and task status of the data task in the data task table according to the download result, and triggers the collector to collect the data to be collected from the platform database and return it to the business end based on the updated data task table. Figure 3 As shown, specifically including S51-S55: S51. If the download result is a successful download, the middleware updates the task status of the data task corresponding to the data request in the data task table to completed, and updates the task update time of the data task.

[0037] S52. If the download result is a download failure, the middleware updates the task status of the data task corresponding to the data request in the data task table to incomplete, updates the task update time of the data task, and returns to execute S3.

[0038] S53, the middleware determines all completed data tasks from the updated data task table, and triggers the collector to collect the data to be collected from the platform database according to the task sorting time.

[0039] Wherein, the step of timing triggering a collector to collect the data to be collected from the platform database according to the task sorting includes: The collector is triggered periodically according to the task sorting to collect the data to be collected from the platform database according to the general dimensions of the data platform corresponding to the data to be collected.

[0040] For example, for the data to be collected on the advertising domain data platform, the data to be collected is collected from the advertising domain data platform according to its corresponding general dimensions, that is, date + site; for the data to be collected on the sales domain data platform, the data to be collected is collected from the sales domain data platform according to its corresponding general dimensions, that is, date + data type + site. The same business subject data platform can collect data in batches at regular intervals through the corresponding general dimensions. This has the advantage of reducing development costs and ensuring data integrity.

[0041] S54: The middleware extracts, cleans and converts the data to be collected to obtain processed data.

[0042] S55. The middleware determines incremental data from the processed data according to the data in the business-end database, and stores the incremental data in the business-end database of the business end.

[0043] In an optional implementation, it also includes: comparing the data in the business-end database with the platform database to obtain a comparison result; determining missing data and / or misaligned data based on the comparison result; issuing an alarm for the missing data and / or the misaligned data; or repairing the missing data and / or the misaligned data, thereby realizing a data reconciliation function and ensuring data integrity.

[0044] In an optional implementation, it further includes: hierarchical processing of the incremental data, data original layer data, data processing layer data and data application layer data. Specifically, the data original layer data is obtained according to the incremental data; the data original layer data is integrated, screened and / or calculated to obtain the data processing layer data; the data processing layer data is summarized to obtain the data application layer data. The hierarchical processing of data facilitates the business end to analyze and utilize data of different business themes.

[0045] Please refer to Figure 2 , Embodiment 2 of the present invention is: A data collection system across business subject data platforms, comprising a business end, middleware and a data platform, wherein the business end comprises a first memory, a first processor and a first computer program stored in the first memory and executable on the first processor, the middleware comprises a second memory, a second processor and a second computer program stored in the second memory and executable on the second processor, the data platform comprises a third memory, a third processor and a third computer program stored in the third memory and executable on the third processor, and the first processor implements the following steps when executing the first computer program; Sending a data request to the middleware through a request interface, wherein the data request includes the data to be collected and a data platform corresponding to the data to be collected; When the second processor executes the second computer program, the following steps are implemented: Establishing a data task according to the type of the request interface corresponding to the data request, wherein the data task includes a task creation time, a task update time, a task status, and request parameters, and updating a data task table according to the data task; Triggering the data platform to download the data to be collected based on the data task table; Update the task update time and task status of the data task in the data task table according to the download result, and trigger the collector to collect the data to be collected from the platform database and return it to the business end based on the updated data task table; When the third processor executes the third computer program, the following steps are implemented: The downloaded data to be collected are stored in the platform database, and the download result of the data to be collected is returned to the middleware.

[0046] In an optional implementation, the second processor further implements the following steps when executing the second computer program: Determine whether the request parameters corresponding to the unfinished data tasks in the data task table contain the request parameters corresponding to the data request. If so, do not process the data request. If not, create a data task according to the type of the request interface corresponding to the data request. The request parameters include start time, end time, site, data type and report type.

[0047] In an optional implementation, the second processor further implements the following steps when executing the second computer program: The data platform is triggered to download the data to be collected according to the task order in the data task table.

[0048] In an optional implementation, the second processor further implements the following steps when executing the second computer program: If the download result is a successful download, the task status of the data task corresponding to the data request in the data task table is updated to completed, and the task update time of the data task is updated; If the download result is a download failure, the task status of the data task corresponding to the data request in the data task table is updated to incomplete, the task update time of the data task is updated, and the step of triggering the data platform to download the data to be collected based on the data task table is returned to execute.

[0049] In an optional implementation, the second processor further implements the following steps when executing the second computer program: Determine all completed data tasks from the updated data task table, and trigger the collector to collect the data to be collected from the platform database according to the task sorting; Extracting, cleaning and converting the data to be collected to obtain processed data; Determine incremental data from the processed data according to data in the business end database, and store the incremental data in the business end database of the business end.

[0050] In summary, the present invention provides a data collection method and system across business subject data platforms. The business end sends a data request to the middleware through a request interface. The middleware establishes a data task according to the type of request interface corresponding to the data request, and updates the data task table according to the data task. The data platform is triggered to download the data to be collected based on the data task table. The data platform stores the downloaded data to be collected in the platform database and returns the download result to the middleware. The middleware updates the task update time and task status of the data task in the data task table according to the download result, and triggers the collector to collect the data to be collected from the platform database and returns it to the business end based on the updated data task table. In this way, the middleware is used to simplify the data interaction process between the business end and different business subject data platforms. Due to the large amount of market data, the downloaded data is first stored in the data platform instead of being directly stored in On the business side, the middleware will then trigger the collector to collect data to the business side, which alleviates the database burden and improves data download efficiency, thereby improving data interaction efficiency. In addition, when the request parameters corresponding to the unfinished data tasks in the data task table contain request parameters corresponding to the data request, it means that the business-side user has submitted requests for the same data range at different times, and the middleware will not process the data request, thereby effectively avoiding repeated data downloads and affecting data interaction efficiency. Moreover, after the middleware collects the data to be collected from the platform database, it extracts, cleans and converts the data to be collected to obtain the processed data, and also determines the incremental data therefrom, and stores the incremental data in the business-side database of the business side, thereby avoiding repeated data collection. Through data extraction, cleaning and conversion, the data that finally reaches the business side not only meets business needs, but also improves data query efficiency.

[0051] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A data collection method across business subject data platforms, characterized in that: Includes steps: The business end sends a data request to the middleware through a request interface, wherein the data request includes the data to be collected and a data platform corresponding to the data to be collected; The middleware establishes a data task according to the type of the request interface corresponding to the data request, wherein the data task includes a task creation time, a task update time, a task status and request parameters, and updates a data task table according to the data task; The middleware triggers the data platform to download the data to be collected based on the data task table; The data platform stores the downloaded data to be collected in the platform database, and returns the download result of the data to be collected to the middleware; The middleware updates the task update time and task status of the data task in the data task table according to the download result, and triggers the collector to collect the data to be collected from the platform database and return it to the business end based on the updated data task table.

2. According to claim 1, a data collection method for a cross-business subject data platform is characterized in that: The middleware establishes a data task according to the type of the request interface corresponding to the data request, including: The middleware determines whether the request parameters corresponding to the unfinished data tasks in the data task table contain the request parameters corresponding to the data request. If so, the data request is not processed. If not, a data task is established according to the type of the request interface corresponding to the data request. The request parameters include start time, end time, site, data type and report type.

3. The data collection method for a cross-business subject data platform according to claim 1 is characterized in that: The middleware triggering the data platform to download the data to be collected based on the data task table includes: The middleware triggers the data platform to download the data to be collected according to the task order in the data task table.

4. The data collection method for a cross-business subject data platform according to claim 1 is characterized in that: The middleware updates the task update time and task status of the data task in the data task table according to the download result, including: If the download result is a successful download, the middleware updates the task status of the data task corresponding to the data request in the data task table to completed, and updates the task update time of the data task; If the download result is a download failure, the middleware updates the task status of the data task corresponding to the data request in the data task table to incomplete, updates the task update time of the data task, and returns to execute the step of triggering the data platform to download the data to be collected based on the data task table.

5. The data collection method for a cross-business subject data platform according to claim 1 is characterized in that: The triggering of the collector to collect the to-be-collected data from the platform database based on the updated data task table and returning it to the business end comprises: The middleware determines all completed data tasks from the updated data task table, and triggers the collector to collect the data to be collected from the platform database according to the task sorting time; The middleware extracts, cleans and converts the data to be collected to obtain processed data; The middleware determines incremental data from the processed data according to the data in the business end database, and stores the incremental data in the business end database of the business end.

6. A data collection system across business subject data platforms, comprising a business end, a middleware and a data platform, wherein the business end comprises a first memory, a first processor and a first computer program stored in the first memory and executable on the first processor, the middleware comprises a second memory, a second processor and a second computer program stored in the second memory and executable on the second processor, the data platform comprises a third memory, a third processor and a third computer program stored in the third memory and executable on the third processor, characterized in that: When the first processor executes the first computer program, the following steps are implemented: Sending a data request to the middleware through a request interface, wherein the data request includes the data to be collected and a data platform corresponding to the data to be collected; When the second processor executes the second computer program, the following steps are implemented: Establishing a data task according to the type of the request interface corresponding to the data request, wherein the data task includes a task creation time, a task update time, a task status, and request parameters, and updating a data task table according to the data task; Triggering the data platform to download the data to be collected based on the data task table; Update the task update time and task status of the data task in the data task table according to the download result, and trigger the collector to collect the data to be collected from the platform database and return it to the business end based on the updated data task table; When the third processor executes the third computer program, the following steps are implemented: The downloaded data to be collected are stored in the platform database, and the download result of the data to be collected is returned to the middleware.

7. The data collection system for a cross-business subject data platform according to claim 6 is characterized in that: When the second processor executes the second computer program, the following steps are further implemented: Determine whether the request parameters corresponding to the unfinished data tasks in the data task table contain the request parameters corresponding to the data request. If so, do not process the data request. If not, create a data task according to the type of the request interface corresponding to the data request. The request parameters include start time, end time, site, data type and report type.

8. The data collection system for a cross-business subject data platform according to claim 6 is characterized in that: When the second processor executes the second computer program, the following steps are further implemented: The data platform is triggered to download the data to be collected according to the task order in the data task table.

9. The data collection system for a cross-business subject data platform according to claim 6, characterized in that: When the second processor executes the second computer program, the following steps are further implemented: If the download result is a successful download, the task status of the data task corresponding to the data request in the data task table is updated to completed, and the task update time of the data task is updated; If the download result is a download failure, the task status of the data task corresponding to the data request in the data task table is updated to incomplete, the task update time of the data task is updated, and the step of triggering the data platform to download the data to be collected based on the data task table is returned to execute.

10. The data collection system for a cross-business subject data platform according to claim 6, characterized in that: When the second processor executes the second computer program, the following steps are further implemented: Determine all completed data tasks from the updated data task table, and trigger the collector to collect the data to be collected from the platform database according to the task sorting; Extracting, cleaning and converting the data to be collected to obtain processed data; Determine incremental data from the processed data according to data in the business end database, and store the incremental data in the business end database of the business end.

Images