Data processing method and its device, electronic device, and computer-readable storage medium

Abstract

The present application provides a data processing method for processing data in a data warehouse. The method includes: in response to a selection instruction of a user for a plurality of candidate original tables presented by a data warehouse control platform, retrieving an original scheduling process control instruction corresponding to the original table selected by the user; in response to a modification instruction input by the user on the data warehouse control platform, modifying the original scheduling process control instruction to generate a dynamic scheduling process control instruction; the dynamic scheduling process control instruction is used to generate an updated table corresponding to the original table according to the received tenant control parameter. Since the dynamic scheduling process control instruction is a dynamic control instruction, when the tenant control parameter corresponding to the target tenant is input, it can only update the partition data corresponding to the target tenant, thereby not affecting the reliability of other tenants using the data; it can also make the data warehouse run according to the original scheduling process when no tenant control parameter is received, without affecting the stability of the overall framework of the data warehouse.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data processing method, an apparatus thereof, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the prior art, for a data warehouse that shares data among multiple tenants, the data in the data warehouse is updated as a whole during daily operation. If only the partition data corresponding to a certain tenant is updated, it will not only affect the stability of the overall framework of the data warehouse, but also affect the reliability of other tenants using the data. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a data processing method, a data processing apparatus, an electronic device, and a computer-readable storage medium to solve the technical problem in the prior art that updating the partition data corresponding to a certain tenant affects the stability and reliability of the data warehouse.

[0004] According to a first aspect of the embodiments of the present application, a data processing method is provided for processing data in a data warehouse. The data processing method includes: in response to a selection instruction of a user for a plurality of candidate original tables presented by a data warehouse control platform, retrieving an original scheduling process control instruction corresponding to the to-be-updated original table selected by the user; in response to a modification instruction input by the user on the data warehouse control platform, modifying the original scheduling process control instruction to generate a dynamic scheduling process control instruction; wherein the dynamic scheduling process control instruction is used to generate an updated table corresponding to the to-be-updated original table according to the received tenant control parameter.

[0005] In one embodiment, the data processing method further includes: receiving a tenant control parameter input by the user on the data warehouse control platform for a target tenant; based on the tenant control parameter, updating the original partition data corresponding to the target tenant in the to-be-updated original table to generate an updated table.

[0006] In one embodiment, updating the original partition data corresponding to the target tenant in the to-be-updated original table based on the tenant control parameter to generate an updated table includes: based on the tenant control parameter, determining screening condition configuration information, where the screening condition configuration information is used to screen out data belonging to the target tenant from the to-be-updated original table and multiple source databases corresponding to the data warehouse; based on the screening condition configuration information, generating updated partition data corresponding to the original partition data, so as to generate an updated table based on the updated partition data.

[0007] In one embodiment, based on the screening condition configuration information, updating partition data corresponding to the original partition data includes: based on the screening condition configuration information, indexing the original partition data corresponding to the target tenant from the original table to be updated, and screening, cleaning, and uploading the real-time data corresponding to the target tenant from multiple source databases corresponding to the original table to be updated; generating the updated partition data based on the original partition data and the real-time data.

[0008] In one embodiment, after updating the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameter to generate an updated table, it further includes: generating a visualization view corresponding to the updated table.

[0009] In one embodiment, generating a visualization view corresponding to the updated table includes: using a view editing language to generate an updated view statement, and the updated view statement can direct to the updated partition data corresponding to the target tenant in the updated table according to the input tenant control parameter; generating a visualization view based on the updated view statement.

[0010] In one embodiment, the data warehouse includes a pharmaceutical sales data warehouse, the original table to be updated includes the drug sales information tables of multiple pharmaceutical companies respectively, the target tenant is one of the multiple pharmaceutical companies, and the tenant control parameter is the identity code of the target tenant.

[0011] According to the second aspect of the embodiments of the present application, a data processing device is provided, which is applied to process data in a data warehouse. The data processing device includes: a first determination module configured to, in response to a selection instruction of a user for multiple original tables to be updated presented by a data warehouse control platform, retrieve an original scheduling process control instruction corresponding to the original table to be updated selected by the user; a second determination module configured to, in response to a modification instruction input by the user on the data warehouse control platform, modify the original scheduling process control instruction to generate a dynamic scheduling process control instruction; wherein, the dynamic scheduling process control instruction is used to generate an updated table corresponding to the original table to be updated according to the received tenant control parameter.

[0012] According to the third aspect of the embodiments of the present application, an electronic device is provided, including: a processor; and a memory, in which computer program instructions are stored, and when the computer program instructions are run by the processor, the processor is caused to execute the data processing method as described in the first aspect above.

[0013] According to the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are run by the processor, the processor is caused to execute the data processing method as described in the first aspect above.

[0014] The data processing method provided by the embodiment of the present application is used to process data in a data warehouse. By responding to a selection instruction of a user for multiple candidate original tables to be updated presented on the data warehouse control platform, retrieving the original scheduling process control instruction corresponding to the original table to be updated selected by the user, and responding to a modification instruction input by the user on the data warehouse control platform, modifying the original scheduling process control instruction to generate a dynamic scheduling process control instruction capable of generating an updated table corresponding to the original table to be updated according to the received tenant control parameter. Since the dynamic scheduling process control instruction is a dynamic control instruction, when the tenant control parameter corresponding to the target tenant is input, it can correspondingly control the data warehouse to run dynamically, so that only the partition data corresponding to the target tenant is updated, while the partition data corresponding to other tenants remains unchanged, thus not affecting the reliability of other tenants using the data; it can also make the data warehouse run according to the original scheduling process when no tenant control parameter is received, without affecting the existing scheduling process of the data warehouse, thus not affecting the stability of the overall framework of the data warehouse. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The flowchart of the data processing method provided by an embodiment of the present application is shown.

[0016] Figure 2 The flowchart of the data processing method provided by an embodiment of the present application is shown.

[0017] Figure 3 The flowchart of the data processing method provided by an embodiment of the present application is shown.

[0018] Figure 4 The flowchart of the data processing method provided by an embodiment of the present application is shown.

[0019] Figure 5 The structural diagram of the data processing device provided by an embodiment of the present application is shown.

[0020] Figure 6 The structural diagram of the data processing device provided by an embodiment of the present application is shown.

[0021] Figure 7 The structural diagram of the update module provided by an embodiment of the present application is shown.

[0022] Figure 8 The structural diagram of the view generation module provided by an embodiment of the present application is shown.

[0023] Figure 9 The structural diagram of the electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0025] With the development of information technology, the field of big data has attracted much attention. Data Warehouse (DW or DWH) occupies an important position in the field of big data and has been widely used in the fields of software and hardware, the Internet, and enterprise network deployment. A data warehouse is a strategic collection that provides support for all types of data for the decision-making process at all levels of an enterprise.

[0026] The core technology of a data warehouse is the ETL (Extract Transform Load) process (the ETL process is the scheduling process). The ETL process is used to describe the process of extracting data from the source database, transforming it, and loading it into the data warehouse. Among them, extraction means collecting specified data from the source database, transformation means converting the data into a specified format and performing data cleaning to ensure data quality, and the loading process means loading the data that meets the specified format after transformation into the data warehouse. The data warehouse processes the data in the source database through the ETL process and stores the processed data in the data warehouse (the processed data stored in the data warehouse is a table). Through daily operation, the data warehouse continuously processes the data in the source database periodically and stores the processing results in the data warehouse to achieve data update. Among them, the scheduling process control instruction for controlling the scheduling process (that is, the ETL control instruction for controlling the ETL process).

[0027] For a data warehouse that shares data among multiple tenants, each tenant is independent but also interdependent. During daily operation, all tenants in the data warehouse are updated as a whole at the same time, that is, the tables in the data warehouse are updated as a whole during daily operation. If you want to update only the partition data corresponding to a certain tenant in real time, the partition data corresponding to other tenants will also be updated, thus affecting the reliability of other tenants' use of data. At the same time, if the scheduling process control instruction of the data warehouse changes, it will affect the existing scheduling process of the data warehouse and the stability of the overall framework of the data warehouse.

[0028] To solve the above problems, an embodiment of the present application provides a data processing method for processing data in a data warehouse, so as to achieve that when the partition data corresponding to a certain tenant is updated in real time, the partition data of other tenants is not updated, thereby not affecting the reliability of other tenants using the data, and at the same time not affecting the existing scheduling process of the data warehouse, and thus not affecting the stability of the overall framework of the data warehouse.

[0029] The following Figures 1 to 9 details the data processing method, data processing device, electronic device, and computer-readable storage medium mentioned in the embodiments of the present application.

[0030] Exemplary data processing method

[0031] Figure 1 The following is a schematic flowchart of the data processing method provided by an embodiment of the present application. The data processing method provided by the embodiment of the present application is applied to process data in a data warehouse. The execution subject of the data processing method provided by the embodiment of the present application can be a server or a computer of a data warehouse control platform.

[0032] As Figure 1 shown, the data processing method includes the following steps.

[0033] S101: In response to a selection instruction of a user for a plurality of candidate original tables presented by the data warehouse control platform, retrieve the original scheduling process control instruction corresponding to the to-be-updated original table selected by the user.

[0034] Specifically, the data warehouse control platform is a platform used to control the operation of the data warehouse. A plurality of original scheduling process control instructions (i.e., a plurality of original ETL instructions) are pre-established in the data warehouse control platform. At a unified time, the data warehouse control platform runs a plurality of original scheduling processes according to the plurality of original scheduling process control instructions to obtain a plurality of original tables, and the plurality of original tables are presented as a plurality of candidate original tables on the data warehouse control platform.

[0035] For the plurality of candidate original tables presented by the data warehouse control platform, the user inputs a selection instruction on the data warehouse control platform according to their own needs, determines which one the user specifically selects among the plurality of candidates according to the selection instruction, and the determined candidate original table is the to-be-updated original table selected by the user. Retrieve the original scheduling process control instruction corresponding to the to-be-updated original table selected by the user.

[0036] For example, the data warehouse includes a pharmaceutical sales data warehouse. Suppose there are multiple candidate original tables, namely: (1) tables of the drug sales information of multiple pharmaceutical companies respectively (where each pharmaceutical company among the multiple pharmaceutical companies is a tenant); (2) tables of the sales information of drug M in 34 provinces respectively (where each province among the 34 provinces is a tenant); (3) tables of the sales information of orthopedic surgical consumables in N partitions of Company X respectively (where each partition is a tenant). The original table to be updated selected by the user is the table including the drug sales information of multiple pharmaceutical companies respectively.

[0037] It should be noted that, for the convenience of description, in the following embodiments of the present application, the example of the original table to be updated being the table of the drug sales information of multiple pharmaceutical companies respectively will be used for illustration.

[0038] S102: In response to the modification instruction input by the user on the data warehouse control platform, modify the original scheduling process control instruction to generate a dynamic scheduling process control instruction.

[0039] Exemplarily, the dynamic scheduling process control instruction is used to generate an updated table corresponding to the original table to be updated according to the received tenant control parameter.

[0040] In one embodiment, the tenant control parameter is the tenant identity code (i.e., ID information).

[0041] Specifically, after retrieving the original scheduling process control instruction corresponding to the original table to be updated, the user modifies the original scheduling process control instruction on the data warehouse control platform to generate a dynamic scheduling process control instruction. The dynamic scheduling process control instruction is a dynamic control instruction, and it can dynamically run the scheduling process according to the received tenant control parameter.

[0042] When the tenant control parameter corresponding to the target tenant is input, the dynamic scheduling process control instruction can correspondingly control the operation of the dynamic control data warehouse, so that only the partition data corresponding to the target tenant is updated, thereby generating an updated table corresponding to the original table to be updated. Since only the partition data corresponding to the target tenant is updated and the partition data corresponding to other tenants remains unchanged, the reliability of other tenants' use of data is not affected. The dynamic scheduling process control instruction can also make the data warehouse run according to the original scheduling process when no tenant control parameter is received, without affecting the existing scheduling process of the data warehouse, and thus not affecting the stability of the overall framework of the data warehouse.

[0043] For example, continuing with the above example, the original table to be updated is the table of the drug sales information of multiple pharmaceutical companies respectively. After retrieving the original scheduling process control instruction corresponding to the table of the drug sales information of multiple pharmaceutical companies respectively, the user modifies the original scheduling process control instruction on the data warehouse control platform to generate a dynamic scheduling process control instruction.

[0044] If the tenant control parameters corresponding to Company A are input, the dynamic scheduling process control instruction can dynamically run the scheduling process according to the input tenant control parameters corresponding to Company A, so as to only update the drug sales volume data corresponding to Company A, and obtain the drug sales volume information tables of each pharmaceutical company with the drug sales volume of Company A updated.

[0045] If the tenant control parameters corresponding to Company B are input, the dynamic scheduling process control instruction can dynamically run the scheduling process according to the input tenant control parameters corresponding to Company B, so as to only update the drug sales volume data corresponding to Company B, and obtain the drug sales volume information tables of each pharmaceutical company with the drug sales volume of Company B updated.

[0046] If the tenant control parameters are not input to the data warehouse control platform, the dynamic scheduling process control instruction is equivalent to the original scheduling process control instruction, and the system runs the original scheduling process to update the drug sales volume data of all companies simultaneously at the same time.

[0047] In the embodiment of the present application, by responding to the selection instruction of the user for multiple candidate original tables to be updated presented by the data warehouse control platform, the original scheduling process control instruction corresponding to the original table to be updated selected by the user is retrieved, and in response to the modification instruction input by the user on the data warehouse control platform, the original scheduling process control instruction is modified to generate a dynamic scheduling process control instruction that can generate an updated table corresponding to the original table to be updated according to the received tenant control parameters. Since the dynamic scheduling process control instruction is a dynamic control instruction, when the tenant control parameters corresponding to the target tenant are input, it can correspondingly control the dynamic operation of the data warehouse, so that only the partition data corresponding to the target tenant is updated, while the partition data corresponding to other tenants remains unchanged, thus not affecting the reliability of other tenants' use of data; it can also make the data warehouse run according to the original scheduling process when no tenant control parameters are received, without affecting the existing scheduling process of the data warehouse, thus not affecting the stability of the overall framework of the data warehouse.

[0048] Figure 2 The following is a schematic flowchart of the data processing method provided by an embodiment of the present application. As Figure 2 shown, the data processing method further includes the following steps.

[0049] S201: Receive the tenant control parameters input by the user on the data warehouse control platform for the target tenant.

[0050] Specifically, for the tenant control parameters input by the user on the data warehouse control platform for the target tenant, the dynamic scheduling process control instruction can, after the tenant control parameters corresponding to the target tenant are input, correspondingly control the dynamic operation of the data warehouse scheduling process.

[0051] S202: Update the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameter to generate an updated table.

[0052] Specifically, since multiple tenants share a data warehouse, the original table to be updated includes the original partition data corresponding to each tenant. After receiving the tenant control parameter of the target tenant, when running the scheduling process according to the dynamic scheduling process control instruction, update the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameter to generate an updated table.

[0053] For example, continuing with the above example, assume the target tenant is Company B, and the ID information corresponding to Company B is received in the dynamic scheduling process control instruction. The data warehouse runs the scheduling process according to the dynamic scheduling process control instruction, updates the drug sales data corresponding to Company B in the original table to be updated, and the drug sales quantities of other companies remain the same as the sales data in the original table to be updated, so as to obtain the drug sales information tables of each pharmaceutical company that only update the drug sales of Company B.

[0054] In the embodiment of the present application, after receiving the tenant control parameter of the target tenant input by the user on the data warehouse control platform, run the scheduling process according to the dynamic scheduling process control instruction, and based on the tenant control parameter, only update the original partition data corresponding to the target tenant in the original table to be updated, and the original partition data corresponding to other tenants is not changed, so as not to affect the use of data by other tenants (that is, realize tenant isolation from each other), and then ensure the reliability and accuracy of the data warehouse operation.

[0055] In the embodiment of the present application, Figure 3 The following is a schematic flowchart of the data processing method provided by an embodiment of the present application. As Figure 3 shown, the step of updating the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameter to generate an updated table includes the following steps.

[0056] S301: Determine the screening condition configuration information based on the tenant control parameter.

[0057] Exemplarily, the screening condition configuration information is used to screen out the data belonging to the target tenant in the original table to be updated and the multiple source databases corresponding to the data warehouse.

[0058] Specifically, after receiving the tenant control parameter of the target tenant, when running the scheduling process according to the dynamic scheduling process control instruction, determine the screening condition configuration information based on the tenant control parameter, and screen out the data belonging to the target tenant in the original table to be updated and the multiple source databases corresponding to the data warehouse by configuring the screening conditions.

[0059] S302: Generate updated partition data corresponding to the original partition data based on the filtering condition configuration information, and thus generate an updated table based on the updated partition data.

[0060] In a further embodiment, based on the filtering condition configuration information, index the original partition data corresponding to the target tenant from the original table to be updated, and filter, clean, and upload the real-time data corresponding to the target tenant from multiple source databases corresponding to the original table to be updated; generate updated partition data based on the original partition data and the real-time data.

[0061] Specifically, when the data warehouse runs the scheduling process according to the dynamic scheduling process control instruction, based on the filtering condition configuration information, index the original partition data corresponding to the target tenant from the original table to be updated. Since the data warehouse corresponds to multiple source databases and the multiple source databases provide various real-time updated data, simultaneously filter, clean, and upload the real-time data corresponding to the target tenant from the multiple source databases corresponding to the original table to be updated. The original partition data and the real-time data are calculated during the running process of the scheduling process, thereby generating updated partition data.

[0062] Illustrate with an example. Continuing with the above example, after receiving the ID information corresponding to Company B, the data warehouse runs the scheduling process according to the dynamic scheduling process control instruction. Based on the ID information corresponding to Company B, index the original sales data of Company B from the drug sales information tables of multiple pharmaceutical companies, and filter, clean, and upload the real-time data corresponding to Company B from multiple source databases corresponding to the drug sales information tables of multiple pharmaceutical companies (the source database may be a database including the sales volume of at least one drug of at least one pharmaceutical representative of the company). The original partition data and the real-time data are calculated during the running process of the scheduling process, thereby generating updated partition data, and thus obtaining the drug sales information tables of multiple pharmaceutical companies that only update the drug sales volume of Company B.

[0063] In the embodiment of the present application, determine the filtering condition configuration information based on the tenant control parameter, and generate updated partition data corresponding to the original partition data based on the filtering condition configuration information, and thus generate an updated table based on the updated partition data, so as to achieve the purpose of dynamically running the scheduling process based on the received tenant control parameter and thus dynamically updating the partition data of the target tenant.

[0064] In an embodiment, after the step of updating the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameter to generate an updated table, it further includes: generating a visualization view corresponding to the updated table.

[0065] Specifically, a view is established between tables, which is a new relationship established abstractly and logically for tables. Through the view, the tables it points to can be queried. Considering that only the partition data of the target tenant is updated in the generated updated table, and the partition data corresponding to other tenants is not updated. To provide a more intuitive display for customers and to provide a view that only points to the updated partition data for subsequent other operations, a visualization view corresponding to the updated table is regenerated.

[0066] Figure 4 The figure shows a schematic flowchart of a data processing method provided by an embodiment of the present application. As Figure 4 shown, the step of generating updated partition data corresponding to the original partition data based on the filtering condition configuration information includes the following steps.

[0067] S401: Generate an updated view statement using a view editing language.

[0068] Exemplarily, the updated view statement can be directed to the updated partition data corresponding to the target tenant in the updated table according to the input tenant control parameter.

[0069] Specifically, using a view editing language to generate an updated view statement can direct the input tenant control parameter to the updated partition data corresponding to the target tenant in the updated table. After inputting the same tenant control parameter as that in the dynamic scheduling process control, only the updated partition data can be indexed through this updated view statement.

[0070] For example, if the ID information corresponding to Company A is received in the dynamic scheduling process control instruction, and the ID information corresponding to Company A is also received by the updated view statement, the data warehouse runs the updated view statement and directly indexes to the updated drug sales volume of Company A.

[0071] If the ID information corresponding to Company B is received in the dynamic scheduling process control instruction, and the ID information corresponding to Company B is also received by the updated view statement, the data warehouse runs the updated view statement and directly indexes to the updated drug sales volume of Company B.

[0072] In an optional embodiment, the view editing language can be determined by the view editing language of the original view statement corresponding to the original table to be updated.

[0073] In an optional embodiment, the view editing language can also be preset in advance, such as the Structured Query Language. Structured Query Language (SQL) is a special-purpose programming language, a database query and programming language. Based on the SQL language, an updated view statement is generated.

[0074] S402: Generate a visualization view based on the updated view statement.

[0075] Specifically, in order to more intuitively display updated data to users, the update view statement needs to be converted into a visual view. Since only the updated partition data corresponding to the target tenant can be seen in the generated visual view, users can more intuitively view the latest data of the target tenant.

[0076] In an embodiment of the present application, a view editing language is used to generate an update view statement that can direct to the updated partition data corresponding to the target tenant in the update table according to the input tenant control parameter, and based on the update view statement, a visual view is generated to provide users with updated data that reduces interference information.

[0077] Exemplary data processing apparatus

[0078] Figure 5 The following shows a schematic structural diagram of a data processing device provided by an embodiment of the present application. The data processing device provided by the embodiment of the present application is applied to process data in a data warehouse. As Figure 5 shown, the data processing device 100 includes a first determination module 101 and a second determination module 102.

[0079] The first determination module 101 is configured to, in response to a selection instruction of a user for a plurality of candidate original tables presented on the data warehouse control platform, retrieve the original scheduling process control instruction corresponding to the to-be-updated original table selected by the user. The second determination module 102 is configured to, in response to a modification instruction input by the user on the data warehouse control platform, modify the original scheduling process control instruction to generate a dynamic scheduling process control instruction; wherein, the dynamic scheduling process control instruction is used to generate an update table corresponding to the to-be-updated original table according to the received tenant control parameter.

[0080] In an embodiment of the present application, the first determination module 101 responds to a selection instruction of a user for a plurality of candidate to-be-updated original tables presented on the data warehouse control platform to retrieve the original scheduling process control instruction corresponding to the to-be-updated original table selected by the user, and the second determination module 102 responds to a modification instruction input by the user on the data warehouse control platform to modify the original scheduling process control instruction to generate a dynamic scheduling process control instruction that can generate an update table corresponding to the to-be-updated original table according to the received tenant control parameter.

[0081] Since the dynamic scheduling process control instruction is a dynamic control instruction, when the tenant control parameters corresponding to the input target tenant are input, the corresponding control of the dynamic control data warehouse can be performed, so that only the partition data corresponding to the target tenant is updated, while the partition data corresponding to other tenants remains unchanged, thus not affecting the reliability of other tenants' data usage; it can also make the data warehouse run according to the original scheduling process when no tenant control parameters are received, without affecting the existing scheduling process of the data warehouse, and thus not affecting the stability of the overall framework of the data warehouse.

[0082] Figure 6 The following shows the structural schematic diagram of a data processing device provided by an embodiment of the present application. As Figure 6 shown, the data processing device 100 further includes: a first receiving module 103 and an updating module 104.

[0083] The first receiving module 103 is configured to receive the tenant control parameters for the target tenant input by the user on the data warehouse control platform, and the updating module 104 is configured to update the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameters to generate an updated table.

[0084] Figure 7 The following shows the structural schematic diagram of the updating module provided by an embodiment of the present application. As Figure 7 shown, the updating module 104 includes: a configuration unit 1041 and an updating unit 1042.

[0085] The configuration unit 1041 is configured to determine the screening condition configuration information based on the tenant control parameters, where the screening condition configuration information is used to screen out the data belonging to the target tenant from the original table to be updated and the multiple source databases corresponding to the data warehouse; the updating unit 1042 is configured to generate the updated partition data corresponding to the original partition data based on the screening condition configuration information, so as to generate an updated table based on the updated partition data.

[0086] In one embodiment, the updating unit 1042 is configured to index the original partition data corresponding to the target tenant from the original table to be updated based on the screening condition configuration information, and screen, clean, and upload the real-time data corresponding to the target tenant from the multiple source databases corresponding to the original table to be updated; generate the updated partition data based on the original partition data and the real-time data.

[0087] Figure 8 The following shows the structural schematic diagram of the view generation module provided by an embodiment of the present application. As Figure 8 shown, the data processing device 100 further includes: a view generation module 105. The view generation module 105 is configured to generate a visualization view corresponding to the updated table after updating the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameters to generate an updated table.

[0088] In one embodiment, the view generation module 105 further includes: a first generation unit 1051 and a second generation unit 1052.

[0089] The first generation unit 1051 is configured to generate an updated view statement using a view editing language. The updated view statement can direct to the updated partition data corresponding to the target tenant in the updated table according to the input tenant control parameters. The second generation unit 1052 is configured to generate a visual view based on the updated view statement.

[0090] The specific functions and operations of the other various modules in the above data processing device have been introduced in detail in the Figures 1 to 5 data processing method described above. Therefore, the repeated description thereof will be omitted here.

[0091] Exemplary electronic device

[0092] Figure 9 The following shows a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 9 shown, the electronic device 300 includes one or more processors 310 and a memory 320.

[0093] The processor 310 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 300 to perform desired functions.

[0094] The memory 320 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 310 may run the program instructions to implement the data processing methods of the various embodiments of the present application described above and / or other desired functions.

[0095] In one example, the electronic device 300 may further include: an input device 330 and an output device 340, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0096] Of course, for simplicity, Figure 9Only some of the components related to the present application in the electronic device 300 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 300 may further include any other appropriate components.

[0097] Exemplary computer program product and computer-readable storage medium

[0098] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions that, when run on a processor, cause the processor to execute the steps in the data processing methods provided according to various embodiments of the present application described in the above "Exemplary Data Processing Method" section of this specification.

[0099] The computer program product may be written in any combination of one or more programming languages for executing the program instructions of the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program instructions may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0100] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon that, when run on a processor, cause the processor to execute the steps in the data processing methods provided according to various embodiments of the present application described in the above "Exemplary Data Processing Method" section of this specification.

[0101] The computer-readable storage medium may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0102] It should be noted that the combination methods of the technical features in the embodiments of the present application are not limited to the combination methods recorded in the claims of the present application or the combination methods recorded in the specific embodiments. All the technical features recorded in the present application can be freely combined or combined in any way, unless contradictions occur between them.

[0103] It should also be noted that the above-listed are only specific embodiments of the present application. Obviously, the present application is not limited to the above embodiments, and there are many similar variations. All deformations directly derived or associated by those skilled in the art from the content disclosed in the present application shall fall within the protection scope of the present application.

[0104] It should be understood that the ordinal numbers such as first and second mentioned in the embodiments of the present application are only used to more clearly describe the technical solutions of the embodiments of the present application, and cannot be used to limit the protection scope of the present application.

[0105] The above is only the preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that, For processing data in a data warehouse, the method includes: In response to a selection instruction of a user for multiple candidate original tables presented by a data warehouse control platform, retrieving an original scheduling process control instruction corresponding to the to-be-updated original table selected by the user; In response to a modification instruction input by the user on the data warehouse control platform, modifying the original scheduling process control instruction to generate a dynamic scheduling process control instruction; Wherein, the dynamic scheduling process control instruction is used to generate an updated table corresponding to the to-be-updated original table according to received tenant control parameters; Receiving tenant control parameters of a target tenant input by the user on the data warehouse control platform; Based on the tenant control parameters, updating original partition data corresponding to the target tenant in the to-be-updated original table to generate the updated table; Generating a visualization view corresponding to the updated table; The updating the original partition data corresponding to the target tenant in the to-be-updated original table to generate the updated table based on the tenant control parameters includes: Based on the tenant control parameters, determining screening condition configuration information, where the screening condition configuration information is used to screen data belonging to the target tenant in the to-be-updated original table and multiple source databases corresponding to the data warehouse; Based on the screening condition configuration information, generating updated partition data corresponding to the original partition data, and thus generating the updated table based on the updated partition data.

2. The data processing method according to claim 1, wherein The generating the updated partition data corresponding to the original partition data based on the screening condition configuration information includes: Based on the screening condition configuration information, indexing the original partition data corresponding to the target tenant from the to-be-updated original table, and screening, cleaning, and uploading real-time data corresponding to the target tenant from multiple source databases corresponding to the to-be-updated original table; Based on the original partition data and the real-time data, generating the updated partition data.

3. The data processing method according to claim 1, characterized in that, The generating the visualization view corresponding to the updated table includes: Using a view editing language to generate an updated view statement, where the updated view statement can direct to the updated partition data corresponding to the target tenant in the updated table according to the input tenant control parameters; Based on the updated view statement, generating the visualization view.

4. The data processing method according to claim 1 or 2, characterized in that, The data warehouse includes a pharmaceutical sales data warehouse, the to-be-updated original table includes drug sales information tables of multiple pharmaceutical companies respectively, the target tenant is one of the multiple pharmaceutical companies, and the tenant control parameter is the identity code of the target tenant.

5. A data processing device, characterized in that, Applied to process data in a data warehouse, the device includes: A first determination module configured to, in response to a selection instruction of a user for multiple candidate original tables presented by a data warehouse control platform, retrieve an original scheduling process control instruction corresponding to the to-be-updated original table selected by the user; A second determination module configured to, in response to a modification instruction input by the user on the data warehouse control platform, modify the original scheduling process control instruction to generate a dynamic scheduling process control instruction; Among them, the dynamic scheduling process control instruction is used to generate an updated table corresponding to the original table to be updated according to the received tenant control parameters; The second determination module is configured to receive the tenant control parameters for the target tenant input by the user on the data warehouse control platform; The second determination module is configured to update the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameters to generate the updated table; The second determination module is configured to generate a visualization view corresponding to the updated table; The updating the original partition data corresponding to the target tenant in the original table to be updated based on the tenant control parameters to generate the updated table includes: Determining filtering condition configuration information based on the tenant control parameters, where the filtering condition configuration information is used to filter out data belonging to the target tenant in the original table to be updated and the multiple source databases corresponding to the data warehouse; Generating updated partition data corresponding to the original partition data based on the filtering condition configuration information, and thus generating the updated table based on the updated partition data.

6. An electronic device, comprising: A processor; And A memory, in which computer program instructions are stored, and when the computer program instructions are run by the processor, the processor is caused to execute the data processing method according to any one of claims 1 to 4.

7. A computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are run by a processor, the processor is caused to execute the data processing method according to any one of claims 1 to 4.

Images