Cross-platform service interaction data processing method and device, computer equipment and storage medium

Abstract

The invention relates to a cross-platform service interaction data processing method and device, computer equipment and a storage medium, and the method comprises the steps: carrying out the polling of a target database according to a preset period, so as to query a target identifier; under the condition that the target identifier is queried, executing a preset data processing operation to obtain a single table identifier; according to the identification type of the single table identification, the single table identification is written into the target database, and different single table identifications of different dates in the target database are unique; under the condition that a plurality of preset single table identifiers exist in a target database and the dates of the preset single table identifiers are the same, generating a combined identifier according to the preset single table identifiers; and realizing data interaction between the off-line data warehouse and the cross-platform business system based on the single table identifier or the combined identifier, so that efficient interaction between the off-line data warehouse and the cross-platform business system is realized through the identifier-driven workflow design, and the reliability and traceability of data circulation are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a cross-platform business interaction data processing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] In the construction of offline data warehouses, multi-platform businesses (such as reconciliation, data synchronization, offline reports, credit reporting, and subsequent accounting operations) all rely on T-1 day business data. Current technologies suffer from insufficient timeliness and accuracy in data transfer between offline data warehouses and cross-platform business systems. Furthermore, when data transfer anomalies occur, it is difficult to quickly locate the abnormal link, resulting in low troubleshooting efficiency. Summary of the Invention

[0003] Therefore, it is necessary to address the aforementioned technical issues by providing a method, device, computer equipment, and readable storage medium that can accurately trigger malicious request interception in data flow of different business systems by maintaining dependency relationships through identifiers, and by locating whether the dependency relationship is completed when anomalies occur in data flow. Furthermore, through the combined application of different basic data identifiers, it is possible to precisely trigger malicious request interception in data flow of different business systems.

[0004] Firstly, this application provides a cross-platform business interaction data processing method, applied to an offline data warehouse, including:

[0005] The target database is polled according to a preset period to query target identifiers; wherein, the target identifier is associated with corresponding date information;

[0006] If the target identifier is found, a preset data processing operation is performed to obtain a single table identifier; wherein, the single table identifier includes table name and date information;

[0007] The single-table identifier is written into the target database according to the identifier type of the single-table identifier; wherein, the different single-table identifiers of different dates in the target database are unique;

[0008] If there are multiple preset single-table identifiers in the target database, and the dates of all preset single-table identifiers are the same, a combined identifier is generated based on the preset single-table identifiers.

[0009] Data interaction with cross-platform business systems is achieved based on the single table identifier or combined identifier; the cross-platform business systems include at least one of the following: accounting core system, reconciliation platform, and BI reporting system.

[0010] In one embodiment, before polling the target database at a preset period to query the target identifier, the method further includes:

[0011] Get real-time time;

[0012] If the real-time time is the same as the preset time corresponding to the preset timed task, the dependency detection association script is invoked to start polling the target database according to the preset period. The dependency detection association script includes target database connection information, target identifier to be detected, and date information corresponding to the target identifier.

[0013] In one embodiment, writing the single-table identifier into the target database according to the identifier type of the single-table identifier includes:

[0014] If the target database is a relational database, query whether the single-table identifier already exists in the relational database; if the single-table identifier exists, do not perform the identifier insertion operation; if the single-table identifier does not exist, perform the identifier insertion operation.

[0015] If the target database is a Hive database, after deleting the historical records of the single table identifier, perform the identifier insertion operation.

[0016] In one embodiment, generating a combined identifier based on the preset single-table identifier includes:

[0017] The system receives input parameters containing multiple identifiers of single tables to be detected, and divides the system into a reference number of identifiers of single tables to be detected based on the input parameters; wherein, the input parameters include the identifiers of single tables to be detected and the corresponding date information;

[0018] Query the actual number of single-table identifiers to be tested that meet the date conditions in the target database. When the actual number is equal to the reference number, generate a combined identifier based on the single-table identifiers to be tested.

[0019] In one embodiment, when interacting with the accounting core system, the method further includes:

[0020] Before the accounting core system executes the current task, it determines whether the completion flag of the previous task of the current task has been detected;

[0021] After detecting the completion status of the preceding task, data interaction is performed with the accounting core system based on the current task.

[0022] In one embodiment, when interacting with the reconciliation platform, the method further includes:

[0023] Detect the target combination identifier associated with the reconciliation task configured on the reconciliation platform;

[0024] Upon detecting the target combination identifier, data interaction is performed with the reconciliation platform to enable the reconciliation platform to initiate a reconciliation task corresponding to the target combination identifier.

[0025] In one embodiment, when interacting with the BI reporting system, the method further includes:

[0026] Detect the target single-table identifier or target combined identifier that the BI reporting system depends on;

[0027] If the target single-table identifier or the target combined identifier is detected, data interaction is performed with the BI reporting system to initiate the report generation process.

[0028] Secondly, this application also provides a cross-platform business interaction data processing device for use in offline data warehouses, including:

[0029] The dependency detection module is used to poll the target database at a preset period to query the target identifier; wherein the target identifier is associated with corresponding date information;

[0030] The data processing module is used to perform preset data processing operations to obtain a single table identifier when the target identifier is found; wherein, the single table identifier includes table name and date information;

[0031] The identifier writing module is used to write the single-table identifier into the target database according to the identifier type of the single-table identifier, wherein the different single-table identifiers of different dates in the target database are unique;

[0032] The identifier generation module is used to generate a combined identifier based on the preset single-table identifiers when there are multiple preset single-table identifiers in the target database and the dates of the preset single-table identifiers are the same.

[0033] The data interaction module is used to realize data interaction with cross-platform business systems based on the single table identifier or the combined identifier. The cross-platform business systems include at least one of the accounting core system, the reconciliation platform, and the BI reporting system.

[0034] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the cross-platform business interaction data processing method described in the first aspect.

[0035] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the cross-platform business interaction data processing method described in the first aspect.

[0036] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the cross-platform business interaction data processing method described in the first aspect.

[0037] In summary, this application proposes a cross-platform business interaction data processing method, apparatus, computer equipment, and storage medium, comprising: polling a target database according to a preset period to query a target identifier; if the target identifier is found, performing a preset data processing operation to obtain a single-table identifier; writing the single-table identifier into the target database according to the identifier type of the single-table identifier, wherein different single-table identifiers on different dates in the target database are unique; if multiple preset single-table identifiers exist in the target database and the dates of each preset single-table identifier are the same, generating a combined identifier based on the preset single-table identifiers; and realizing data interaction between the offline data warehouse and the cross-platform business system based on the single-table identifier or the combined identifier. This application achieves efficient interaction between the offline data warehouse and the cross-platform business system through an identifier-driven workflow design, improving the reliability and traceability of data flow. Attached Figure Description

[0038] Figure 1 This is an application environment diagram of a cross-platform business interaction data processing method in one embodiment;

[0039] Figure 2 This is a schematic diagram illustrating the construction of a single-table identifier in a cross-platform business interaction data processing method according to one embodiment.

[0040] Figure 3 This is a flowchart illustrating a cross-platform business interaction data processing method in one embodiment;

[0041] Figure 4 A schematic diagram illustrating the construction of a combined identifier for a cross-platform business interaction data processing method in one embodiment;

[0042] Figure 5 This is a structural block diagram of a cross-platform business interaction data processing device in one embodiment;

[0043] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] In related technologies, offline data warehouse construction relies on T-1 day business data for multi-platform operations (such as reconciliation, data synchronization, offline reporting, credit reporting, and subsequent core accounting operations). Existing technologies suffer from the following problems in data flow between offline data warehouses and cross-platform business systems: insufficient timeliness and accuracy of data flow, downstream systems cannot promptly detect whether upstream data is ready; difficulty in quickly locating abnormal links when data flow is abnormal, resulting in low troubleshooting efficiency; different businesses rely on different data, lacking precise triggering mechanisms, leading to high coupling between systems. Therefore, there is an urgent need for a cross-platform business interaction data processing solution that maintains dependencies through identifiers, improves the accuracy and timeliness of data interaction, and facilitates anomaly troubleshooting.

[0046] In this embodiment, T-1 day business data refers to business data generated on the previous working day (T-1 day). In practical applications, T-1 day business data can be used for time series analysis, supporting historical data backtracking and decision-making. In the cross-platform business interaction patent solution, T-1 day business data is the core driver of interaction between the offline data warehouse and downstream systems.

[0047] The cross-platform business interaction data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0048] In this embodiment, the cross-platform business interaction data processing method is mainly applied to the data interaction processing between the offline data warehouse and the cross-platform business system. Before introducing the data interaction between the offline data warehouse and the cross-platform business system, the workflow built within the offline data warehouse is first introduced.

[0049] In one embodiment, such as Figure 2As shown, a big data extraction or processing workflow is built within an offline data warehouse, using tables as the dimension. This workflow includes dependency detection nodes, data extraction (incremental or full) nodes, data import nodes, data merging nodes, and identifier writing nodes. The dependency detection node queries the target identifier and executes subsequent operations according to the dependency relationship. The data extraction, data import, and data merging nodes collectively constitute the data processing nodes, used to perform extraction, import, and merging operations on the data. For example, the data import node can import the extracted data into the Hadoop Distributed File System (HDFS), and the data merging node can merge data into the detailed data layer. The identifier writing node is used to write the constructed identifier information into the target database.

[0050] In this embodiment, the aforementioned workflow, which includes a dependency detection node, a data extraction (incremental or full) node, a data import node, a data merging node, and an identifier write node, is used to construct a data extraction workflow or a processing workflow for a single table.

[0051] In one embodiment, such as Figure 3 As shown, a cross-platform business interaction data processing method is provided, which can be applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0052] S301: Poll the target database according to a preset period to query the target identifier. The target identifier is associated with corresponding date information.

[0053] In this embodiment, the offline data warehouse extracts T-1 day business data (such as customer transaction records) from the source system daily, cleans and transforms it, and then stores it in the data warehouse. Downstream systems (accounting core, reconciliation platform, etc.) initiate subsequent operations by detecting T-1 day data identifiers (such as "data extraction completed identifiers"). For example, after the accounting core system detects the T-1 day data ready identifier, it performs initial batch operations at the beginning of the day.

[0054] In this embodiment, the target database refers to the database in the offline data warehouse that stores raw business data (such as customer transaction records). In practical applications, the target database serves as the source of data extraction and the storage location for processed data, supporting data interaction with downstream systems through identifiers (such as single-table / combined identifiers). In this embodiment, the target database includes Hive databases, MySQL databases, and other cross-platform databases. The actual type of the target database can be determined based on the specific content of the target identifier.

[0055] In this embodiment, the identification information is a field in the database used to uniquely identify specific data, typically associated with date information to distinguish data from different periods. The target identifier is the query target for polling the target database. In this embodiment, the date information refers to the timestamp of data generation or processing, used to track data changes, support time-sensitive queries, and ensure data consistency.

[0056] In this embodiment, polling the target database according to a preset period means querying the database once every preset time interval within a waiting thread. For example, a waiting thread is started, and the target database is queried once every 60 seconds (s).

[0057] In this embodiment, after the daily task begins, the offline data warehouse will initiate a dependency detection process to execute step S301 to determine whether to perform subsequent data processing operations.

[0058] S302, if the target identifier is found, perform a preset data processing operation to obtain a single table identifier. The single table identifier includes the table name and date information.

[0059] In this embodiment, the preset data processing operation refers to the process of transforming raw data into structured data, including steps such as cleaning, transformation, and loading. In this embodiment, the preset data processing operation includes at least data extraction, data import, and data merging operations. The data extraction operation is mainly used to extract T-1 day's business data (such as customer transaction records) from the source system at regular intervals each day. The data import and data merging operations are mainly used to clean invalid data and aggregate key indicators (such as the average daily balance of customers) to obtain labeled data in tables.

[0060] In this embodiment, a single-table identifier refers to a unique identifier generated by data processing operations, containing the table name and date information, used to mark the status of a single data table within a specific date. For example, the data structure of a single-table identifier can be composed in the format of "table name:customer_info|date:2025-08-10|status:completed".

[0061] S303. Write the single-table identifier to the target database according to the identifier type of the single-table identifier. The single-table identifiers for different dates in the target database are unique.

[0062] In this embodiment, when writing a single table identifier, it can be written to the target database according to the identifier type (such as "data extraction completed", "reconciliation completed", etc.) to ensure the uniqueness of different identifiers within the same date.

[0063] In this embodiment, the identifier type of a single table identifier refers to the classification used in the database to distinguish data status, such as primary key, unique key, status identifier, etc. In a specific embodiment, the single table identifier type is used to mark the processing stage of the data table, such as "data extraction completed" and "processing completed". The combined identifier type is used to represent the task status after multiple tables are joined, such as "reconciliation task_20250810".

[0064] In this embodiment, the identifier type of a single table also includes data types such as relationship or business data, each corresponding to a different target database. For example, relationship corresponds to a MySQL database, and business data corresponds to a Hive database.

[0065] S304. If there are multiple preset single-table identifiers in the target database, and the dates of each preset single-table identifier are the same, generate a combined identifier based on the preset single-table identifiers.

[0066] In this embodiment, when multiple single-table identifiers have the same date, the system automatically generates a higher-level identifier, namely a combined identifier, for cross-table join operations. If multiple single-table identifiers with the same date exist in the target database (such as customer_info_20250810 and order_20250810), the system merges them into a combined identifier task_20250810. This triggers a cross-platform system (such as a reconciliation platform) to perform batch operations.

[0067] S305 enables data interaction with cross-platform business systems based on single-table identifiers or combined identifiers. Cross-platform business systems include at least one of the following: accounting core system, reconciliation platform, and BI reporting system.

[0068] In this embodiment, the core accounting system refers to the bank's central system, used to process transactions such as deposits, loans, and transfers, ensuring the accuracy and real-time nature of fund flows. The reconciliation platform refers to a system that automatically verifies financial data, ensuring consistency by comparing data such as accounts receivable and accounts payable. The BI reporting system refers to business intelligence tools that display analytical results through visual charts (such as line charts and bar charts) to support decision-making.

[0069] In practical applications, cross-platform systems utilize the single-table identifiers or combined identifiers constructed above to achieve data interaction. For example, the core accounting system receives reconciliation instructions triggered by combined identifiers and executes daily batch operations (such as updating account balances). The processing results are returned to the offline data warehouse, forming a closed loop. The reconciliation platform initiates an automatic reconciliation process based on the combined identifier (e.g., task_20250810), generates reconciliation results, and feeds them back to the BI reporting system for management monitoring. The BI reporting system generates real-time reports (such as daily transaction volume trends) based on single-table / combined identifiers.

[0070] Based on the above steps, this embodiment provides a cross-platform business interaction data processing method. By constructing identifiers to maintain dependencies, it improves the timeliness and accuracy of data flow. Furthermore, when anomalies occur in business interactions, the problem link can be located through identifiers, improving troubleshooting efficiency. Combining identifiers supports precise triggering of different businesses, reducing system coupling. Moreover, the identifier storage is adaptable to multiple database types (relational databases, Hive, etc.), offering strong scalability.

[0071] In one embodiment, before polling the target database at a preset period to query the target identifier, the cross-platform business interaction data processing method further includes:

[0072] Obtain the real-time time; if the real-time time is the same as the preset time corresponding to the preset scheduled task, call the dependency detection association script to start polling the target database according to the preset period. The dependency detection association script includes the target database connection information, the target identifier to be detected, and the date information corresponding to the target identifier.

[0073] In this embodiment, real-time refers to the current time when the offline data warehouse interacts with the cross-platform business system. In practical applications, the dependency detection workflow is configured with scheduled tasks. In this embodiment, the preset scheduled task includes a preset time; for example, the preset time can be configured to 01:00 AM.

[0074] In one specific embodiment, a cron expression can be configured to start a workflow daily at a set time. Once started, the workflow first performs dependency checks by calling a dependency-related Python script. This script can be configured with target database connection information and accepts input parameters, which are passed to Python script A by the dependency check node. These parameters can be pre-configured on the dependency check node and include the identifier `notice_key` and the date `notice_date` to be checked. In practical application, Python script A starts a waiting thread that queries the target database every 60 seconds. When the identifier is found, the thread exits, and the dependency check node passes. If the identifier is not found, it polls and waits. Python script A is maintained as a file in a resource center, which can create and manage multiple scripts to check multiple target identifiers simultaneously.

[0075] In one embodiment, writing the single-table identifier to the target database according to the identifier type of the single-table identifier includes:

[0076] If the target database is a relational database, check if a single table identifier already exists in the relational database; if it exists, do not perform the identifier insertion operation; if it does not exist, perform the identifier insertion operation.

[0077] If the target database is a Hive database, after deleting the historical records of the identifier in a single table, perform the identifier insertion operation.

[0078] In this embodiment, after data processing is complete, the identifier is written. If writing to MySQL or other cross-platform databases, it needs to be associated with resource center script B. Taking writing to a MySQL database as an example, script B's function is to connect to the target database. If the connection is successful, it first checks whether the identifier already exists in the target database. If a single-table identifier exists, the insert operation is not performed; otherwise, the insert operation is performed. If writing to a Hive database, the identifier writing node does not need to be associated with resources. First, the historical records of single-table identifiers are deleted, and then the identifier insertion operation is performed to ensure that there is only one record of different identifiers on different dates in the database.

[0079] Based on the above steps, relational databases (such as MySQL and PostgreSQL) support transactions, ensuring the atomicity of judgment and insertion operations. Hive databases, as data warehouses, need to retain the latest state rather than historical records. For example, if multiple write operations are triggered on the same day, only the result of the last operation is retained. This is suitable for scenarios requiring overwrite updates (such as end-of-day data correction). This embodiment achieves a reliable design for cross-platform data interaction by configuring different target databases, ensuring both the auditing requirements of relational databases and optimizing the storage efficiency of Hive databases, forming a complementary technical solution.

[0080] In one embodiment, generating a combined identifier based on a preset single-table identifier includes:

[0081] The system receives input parameters containing multiple identifiers of the single table to be tested, and then divides the system into a reference number of identifiers based on the input parameters. The input parameters include the identifiers of the single table to be tested and their corresponding date information.

[0082] Query the actual number of the single-table identifiers to be tested that meet the date conditions in the target database. When the actual number is equal to the reference number, generate a combined identifier based on the single-table identifiers to be tested.

[0083] In this embodiment, a workflow specifically designed for generating combined identifiers is constructed. This workflow comprises two nodes: a dependency detection node and an identifier writing node. This workflow can generate multiple combined identifiers. The workflow structure is as follows: Figure 4 As shown. In this embodiment, the writing of each identifier in the workflow is independent of each other, so as to... Figure 4Taking the first workflow as an example, the function of dependency detection nodes A, B, and C is as follows: when the three identifiers dwd_table_name_a, dwd_table_name_b, and dwd_table_name_c exist simultaneously in the target database and the date information notice_date is T-1, the dependency node detection passes.

[0084] At this point, the logic for multiple identifier detection is as follows: The dependent node receives dwd_table_name_a, dwd_table_name_b, and dwd_table_name_c as input parameters, specifying the delimiter. The function of this dependent node is to segment the single-table identifiers to be checked and the corresponding reference quantity of each single-table identifier according to the delimiter in the input parameters, connect to the database, and compare the database query results. When the actual quantity of the query results equals the reference quantity of the segmented identifiers, the detection passes, and the write operation is performed to the combined identifier 1 node.

[0085] For example, after multiple workflows have completed data extraction and processing, the generated combined identifiers can be shown in Tables 1 and 2 below:

[0086] Table 1 (Combined identifiers stored in the Hive database)

[0087]

[0088] Table 2 (Combined identifiers stored in the MySQL database)

[0089]

[0090] In one embodiment, when interacting with the core accounting system, the cross-platform business interaction data processing method further includes:

[0091] Before the accounting core system executes the current task, it determines whether the completion flag of the preceding task has been detected; after the completion flag of the preceding task is detected, data interaction is performed with the accounting core system according to the current task.

[0092] In this embodiment, during the interaction between the offline data warehouse and the accounting core system, the accounting core system sets up a big data extraction and inspection task, specifically for big data extraction notification and receiving big data extraction completion notification. This task is time-controlled through configured task dependencies, as shown in the format in Table 3 below:

[0093] Table 3

[0094]

[0095] In this embodiment, a record is inserted after each task is executed each day, marked as "complete" for success and "fail" for failure. The next task will only start execution when it detects that the status of the task it depends on for the day is "complete". The task execution table is shown in Table 4 below:

[0096] Table 4

[0097]

[0098] In this embodiment, when the completion of the end-of-day task C is detected, the big data extraction and inspection task inserts a record in MySQL. This record contains notice_key and notice_date, which are used by the dependent inspection nodes for big data extraction.

[0099] In actual business operations, the interaction with the accounting core is crucial. In the financial sector, the offline data warehouse needs to periodically extract data from day T-1 to generate customer information, loan balances, various interest and fee balances, and overdue data for the previous day, and push these data to downstream reports and credit reporting systems. The accounting core needs to wait for the big data extraction to be completed before it can proceed with the next step of daily batch processing. Therefore, after the daily batch processing is completed, the accounting side sets a flag in the MySQL database to indicate that the conditions for big data extraction are met. After the big data system detects this flag, it performs data extraction. After the extraction is completed, it sets a corresponding flag indicating that the extraction is complete. After the accounting core detects this flag, it continues with subsequent business operations through big data extraction and batch checks.

[0100] In one embodiment, when interacting with a reconciliation platform, the cross-platform business interaction data processing method further includes:

[0101] Detect the target combination identifier associated with the reconciliation tasks configured on the reconciliation platform;

[0102] After detecting the target combination identifier, data interaction is performed with the reconciliation platform to enable the reconciliation platform to initiate the reconciliation task corresponding to the target combination identifier.

[0103] In this embodiment, during the interaction between the offline data warehouse and the reconciliation platform, the reconciliation platform will configure multiple reconciliation tasks. Each reconciliation task will be configured with a cron expression and a reconciliation identifier. When the reconciliation task is started, the JDBC thread will wait to detect the reconciliation identifier. After the identifier is detected, the subsequent reconciliation tasks will be executed.

[0104] In this embodiment, the interaction between the offline data warehouse and the reconciliation platform allows the reconciliation platform to configure multiple reconciliation tasks. Each reconciliation task relies on different data, and the corresponding dependency identifiers are also different. Depending on business needs, the timeliness of reconciliation tasks is not high and concurrent reconciliation is supported. Therefore, to reduce thread waiting, they can be distinguished according to business lines. For example, after the self-operated loan data extraction is completed, the MySQL database is uniformly assigned to identifier A. After the channel data extraction is completed, the MySQL database is uniformly assigned to identifier B. After the identifier is completed, self-operated loan reconciliation tasks 1, 2, and 3 are uniformly triggered. After identifier B is completed, channel data reconciliation tasks 4, 5, and 6 are uniformly triggered.

[0105] In one embodiment, when interacting with a BI reporting system, the cross-platform business interaction data processing method further includes:

[0106] Detect the target single-table identifier or target combination identifier that the BI reporting system depends on;

[0107] If the presence of a target single-table identifier or a target combination identifier is detected, data interaction is performed with the BI reporting system to initiate the report generation process.

[0108] In this embodiment, during the interaction between the offline data warehouse and BI reports, the generation of BI reports relies on a big data scheduling task. This task continuously detects identifiers, and upon detection, it initiates subsequent data merging and report generation. In practice, the interaction with BI reports involves different data depending on the downstream BI reports and various reporting services, and requires data processed by the big data system. Since this interaction does not involve business systems but is only internal to the big data system, the data identifier library is a Hive database. After each big data table is processed, an identifier, including the table name and date, is created in the Hive database. For example, BI report 1 depends on big data table AB, BI report 2 depends on big data table BCD, and the credit reporting system depends on big data table EF. Therefore, corresponding identifier groups can be created based on different business systems to reduce the coupling between them.

[0109] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0110] Based on the same inventive concept, this application also provides a cross-platform business interaction data processing apparatus for implementing the cross-platform business interaction data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more cross-platform business interaction data processing apparatus embodiments provided below can be found in the limitations of the cross-platform business interaction data processing method described above, and will not be repeated here.

[0111] In one embodiment, such as Figure 5 As shown, a cross-platform business interaction data processing device 500 is provided, including: a dependency detection module 510, a data processing module 520, an identifier writing module 530, an identifier generation module 540, and a data interaction module 550, wherein:

[0112] The dependency detection module 510 is used to poll the target database according to a preset period to query the target identifier; wherein the target identifier is associated with corresponding date information;

[0113] The data processing module 520 is used to perform a preset data processing operation to obtain a single table identifier when the target identifier is found; wherein, the single table identifier includes table name and date information;

[0114] The identifier writing module 530 is used to write the single table identifier into the target database according to the identifier type of the single table identifier, wherein the different single table identifiers of different dates in the target database are unique;

[0115] The identifier generation module 540 is used to generate a combined identifier based on the preset single-table identifiers when there are multiple preset single-table identifiers in the target database and the dates of each preset single-table identifier are the same.

[0116] The data interaction module 550 is used to realize data interaction with a cross-platform business system based on the single table identifier or the combined identifier. The cross-platform business system includes at least one of the accounting core system, the reconciliation platform and the BI reporting system.

[0117] In one embodiment, the dependency detection module 510 is specifically used to obtain the real-time time; if the real-time time is the same as the preset time corresponding to the preset timed task, the dependency detection association script is invoked to start polling the target database according to the preset period, wherein the dependency detection association script includes target database connection information, target identifier to be detected, and date information corresponding to the target identifier.

[0118] In one embodiment, the identifier writing module 530 is specifically used to query whether the single-table identifier already exists in the relational database if the target database is a relational database; if it exists, the identifier insertion operation is not performed; if it does not exist, the identifier insertion operation is performed; if the target database is a Hive database, the identifier insertion operation is performed after deleting the historical records of the single-table identifier.

[0119] In one embodiment, the identifier generation module 540 is specifically used to receive input parameters containing multiple identifiers of single tables to be detected, and to segment a reference number of identifiers of single tables to be detected based on the input parameters; wherein, the input parameters include identifiers of single tables to be detected and corresponding date information; query the actual number of identifiers of single tables to be detected that meet the date conditions in the target database, and when the actual number is equal to the reference number, generate a combined identifier based on the identifiers of single tables to be detected.

[0120] In one embodiment, the data interaction module 550 is specifically used to determine whether a completion identifier of a prerequisite task of the current task is detected before the accounting core system executes the current task; after detecting the completion identifier of the prerequisite task, the module interacts with the accounting core system based on the current task.

[0121] In one embodiment, the data interaction module 550 is specifically used to detect the target combination identifier associated with the reconciliation task configured by the reconciliation platform; after detecting the target combination identifier, it interacts with the reconciliation platform to enable the reconciliation platform to start the reconciliation task corresponding to the target combination identifier.

[0122] In one embodiment, the data interaction module 550 is specifically used to detect the target single table identifier or target combination identifier that the BI reporting system depends on; when the existence of the target single table identifier or the target combination identifier is detected, data interaction is performed with the BI reporting system to start the report generation process.

[0123] In summary, this embodiment provides a cross-platform business interaction data processing device. By maintaining dependencies through identifiers, it improves the timeliness and accuracy of data flow. When data interaction anomalies occur, the problem link can be located through identifiers, improving troubleshooting efficiency. By combining identifiers, it supports precise triggering of different businesses, reducing system coupling. It can adapt to multiple types of databases (relational databases, Hive, etc.) and has strong scalability.

[0124] Each module in the aforementioned cross-platform business interaction data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0125] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores identification data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements a cross-platform business interaction data processing method.

[0126] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0127] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0128] The target database is polled at a preset period to query target identifiers; each target identifier is associated with a corresponding date.

[0129] If the target identifier is found, perform a preset data processing operation to obtain a single table identifier; the single table identifier includes the table name and date information.

[0130] Write the single-table identifier into the target database according to the identifier type of the single-table identifier. In the target database, the different single-table identifiers for different dates are unique.

[0131] If there are multiple preset single-table identifiers in the target database, and the dates of all preset single-table identifiers are the same, a combined identifier is generated based on the preset single-table identifiers.

[0132] Data interaction with cross-platform business systems is achieved based on single-table identifiers or combined identifiers. Cross-platform business systems include at least one of the following: accounting core system, reconciliation platform, and BI reporting system.

[0133] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0134] The target database is polled at a preset period to query target identifiers; each target identifier is associated with a corresponding date.

[0135] If the target identifier is found, perform a preset data processing operation to obtain a single table identifier; the single table identifier includes the table name and date information.

[0136] Write the single-table identifier into the target database according to the identifier type of the single-table identifier. In the target database, the different single-table identifiers for different dates are unique.

[0137] If there are multiple preset single-table identifiers in the target database, and the dates of all preset single-table identifiers are the same, a combined identifier is generated based on the preset single-table identifiers.

[0138] Data interaction with cross-platform business systems is achieved based on single-table identifiers or combined identifiers. Cross-platform business systems include at least one of the following: accounting core system, reconciliation platform, and BI reporting system.

[0139] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0140] The target database is polled at a preset period to query target identifiers; each target identifier is associated with a corresponding date.

[0141] If the target identifier is found, perform a preset data processing operation to obtain a single table identifier; the single table identifier includes the table name and date information.

[0142] Write the single-table identifier into the target database according to the identifier type of the single-table identifier. In the target database, the different single-table identifiers for different dates are unique.

[0143] If there are multiple preset single-table identifiers in the target database, and the dates of all preset single-table identifiers are the same, a combined identifier is generated based on the preset single-table identifiers.

[0144] Data interaction with cross-platform business systems is achieved based on single-table identifiers or combined identifiers. Cross-platform business systems include at least one of the following: accounting core system, reconciliation platform, and BI reporting system.

[0145] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0146] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A cross-platform business interaction data processing method, characterized in that, Applications include offline data warehouses, including: The target database is polled according to a preset period to query target identifiers; wherein, the target identifier is associated with corresponding date information; If the target identifier is found, a preset data processing operation is performed to obtain a single table identifier; wherein, the single table identifier includes table name and date information; The single-table identifier is written into the target database according to the identifier type of the single-table identifier; wherein, the different single-table identifiers of different dates in the target database are unique; If there are multiple preset single-table identifiers in the target database, and the dates of all preset single-table identifiers are the same, a combined identifier is generated based on the preset single-table identifiers. Data interaction with cross-platform business systems is achieved based on the single table identifier or combined identifier; the cross-platform business systems include at least one of the following: accounting core system, reconciliation platform, and BI reporting system.

2. The method according to claim 1, characterized in that, Before polling the target database according to a preset period to query the target identifier, the method further includes: Get real-time time; If the real-time time is the same as the preset time corresponding to the preset timed task, the dependency detection association script is invoked to start polling the target database according to the preset period. The dependency detection association script includes target database connection information, target identifier to be detected, and date information corresponding to the target identifier.

3. The method according to claim 1, characterized in that, The step of writing the single-table identifier into the target database according to the identifier type of the single-table identifier includes: If the target database is a relational database, query whether the single-table identifier already exists in the relational database; if the single-table identifier exists, do not perform the identifier insertion operation; if the single-table identifier does not exist, perform the identifier insertion operation. If the target database is a Hive database, after deleting the historical records of the single table identifier, perform the identifier insertion operation.

4. The method according to claim 1, characterized in that, The step of generating a combined identifier based on the preset single-table identifier includes: The system receives input parameters containing multiple identifiers of single tables to be detected, and divides the system into a reference number of identifiers of single tables to be detected based on the input parameters; wherein, the input parameters include the identifiers of single tables to be detected and the corresponding date information; Query the actual number of single-table identifiers to be tested that meet the date conditions in the target database. When the actual number is equal to the reference number, generate a combined identifier based on the single-table identifiers to be tested.

5. The method according to any one of claims 1 to 4, characterized in that, When interacting with the core accounting system, the method further includes: Before the accounting core system executes the current task, it determines whether the completion flag of the previous task of the current task has been detected; After detecting the completion status of the preceding task, data interaction is performed with the accounting core system based on the current task.

6. The method according to any one of claims 1 to 4, characterized in that, When interacting with the reconciliation platform, the method further includes: Detect the target combination identifier associated with the reconciliation task configured on the reconciliation platform; Upon detecting the target combination identifier, data interaction is performed with the reconciliation platform to enable the reconciliation platform to initiate a reconciliation task corresponding to the target combination identifier.

7. The method according to any one of claims 1 to 4, characterized in that, When interacting with the BI reporting system, the method further includes: Detect the target single-table identifier or target combined identifier that the BI reporting system depends on; If the target single-table identifier or the target combined identifier is detected, data interaction is performed with the BI reporting system to initiate the report generation process.

8. A cross-platform business interaction data processing device, characterized in that, Applications include offline data warehouses, including: The dependency detection module is used to poll the target database at a preset period to query the target identifier; wherein the target identifier is associated with corresponding date information; The data processing module is used to perform preset data processing operations to obtain a single table identifier when the target identifier is found; wherein, the single table identifier includes table name and date information; The identifier writing module is used to write the single-table identifier into the target database according to the identifier type of the single-table identifier, wherein the different single-table identifiers of different dates in the target database are unique; The identifier generation module is used to generate a combined identifier based on the preset single-table identifiers when there are multiple preset single-table identifiers in the target database and the dates of the preset single-table identifiers are the same. The data interaction module is used to realize data interaction with cross-platform business systems based on the single table identifier or the combined identifier. The cross-platform business systems include at least one of the accounting core system, the reconciliation platform, and the BI reporting system.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the cross-platform business interaction data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the cross-platform business interaction data processing method as described in any one of claims 1 to 7.

Images