A data cleaning and merging method and device based on a multi-source system and a readable medium

Abstract

The application discloses a kind of based on multi-source system's data cleaning and merging method, device and readable medium, by constructing business library, the data change information of multiple source systems is listened to, generates task scheduling single;According to the in-storage configuration of business library, obtain task scheduling single to generate information task scheduling queue, to information task scheduling queue is cleaned, obtain effective information task scheduling queue;According to effective information scheduling task queue generation in-storage table, establish the transfer relationship between source system and business library in in-storage table and information in-storage type;According to task scheduling single, obtain the data of source system, based on the information mapping relationship between source system and business library obtains change information, change information is written into in-storage information in in-storage table;According to in-storage information and information in-storage type, the metadata in the meta model of business library is changed.Therefore, the changed data in multi-source system can be effectively cleaned, and the data of source system is conveniently unified management.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data cleaning, in particular to a data cleaning and merging method and device based on a multi-source system and a readable medium. BACKGROUND

[0002] At present, in order to make the transaction more convenient and fast, many places will establish multiple online business management systems. With the establishment and use of various business systems, the problem of "data island" affects the daily use of users and management units. The following pain points are currently faced:

[0003] 1. Repeatedly entering information: each system maintains data separately, resulting in repeated maintenance workload, and because of different businesses and different personnel, the data is incomplete and the probability of data errors is high.

[0004] 2. Information is not interconnected: information is not uniformly managed, resulting in difficulty in controlling business, such as not knowing the status of enterprise relocation, relocated enterprises still enjoying park enterprise services, building managers not knowing which new registered enterprise information, and not being able to verify the enterprise's entry situation in time.

[0005] 3. The understanding of users is relatively scattered: scattered information obtained from multiple systems cannot clearly know all the situations of users, and cannot provide support for multi-dimensional data analysis. For example, what stage is the enterprise in, what difficulties does it have, and whether it has the intention to move out of the park.

[0006] 4. It is difficult to control illegal behavior: illegal behavior of enterprises such as arrears and illegal parking cannot be further effectively controlled after repeated persuasion.

[0007] 5. The improvement direction is not clear: lacking digital support, the existing improvement of the region relies more on human experience, and there is no way to know where the core advantages and shortcomings of the region are.

[0008] 6. The operation efficiency is not followed up in time: it is not possible to quickly understand the use of the region's assets, resources and service efficiency, and the amount of operation analysis data is insufficient.

[0009] 7. Poor user experience: each business system has an independent database, and data cannot be interconnected, resulting in the need to switch between different systems and repeatedly log in when users handle business, resulting in poor user experience.

[0010] Therefore, in order to meet the needs of information sharing and data analysis of various departments, not only a basic information database under unified data standards should be established, but also a long-term management mechanism for the management, maintenance, updating and use of this basic database should be established, so that the database can be continuously expanded and improved, ensuring the consistency, freshness and accuracy of the data, and laying a good foundation for the planning and construction of the overall information resources of the region.

[0011] Therefore, it is essential to design an innovative data cleaning and merging method based on a multi-source system. SUMMARY

[0012] In view of the above-mentioned problems of non-uniform data of multiple business systems and scattered information, embodiments of the present application propose a data cleaning and merging method based on a multi-source system, a device and a readable medium to solve the technical problems mentioned in the background section.

[0013] In a first aspect, embodiments of the present application provide a data cleaning and merging method based on a multi-source system, comprising the following steps:

[0014] S1, constructing a business library based on a meta-model, listening to data change information of multiple source systems, and generating a task scheduling sheet based on the data change information;

[0015] S2, obtaining the task scheduling sheet according to the warehouse configuration of the business library to generate an information task scheduling queue, cleaning the information task scheduling queue to obtain an effective information task scheduling queue;

[0016] S3, generating a warehouse table according to the effective information scheduling task queue, and establishing a transfer relationship between the source system and the business library and an information warehouse type in the warehouse table;

[0017] S4, obtaining data of the source system according to the task scheduling sheet in the effective information scheduling task queue, obtaining change information based on the information mapping relationship between the source system and the business library, and writing the change information into the warehouse information in the warehouse table; and

[0018] S5, changing the metadata in the meta-model of the business library according to the warehouse information and the information warehouse type.

[0019] In some embodiments, step S1 specifically comprises generating a task scheduling sheet through a trigger after generating data change information in the source system, and establishing communication between the source system and the business library through the task scheduling sheet.

[0020] In some embodiments, the warehouse configuration of the business library includes a warehouse frequency and a warehouse rule.

[0021] In some embodiments, step S2 specifically comprises:

[0022] obtaining the task scheduling sheet according to the warehouse frequency and forming the information task scheduling queue;

[0023] cleaning the invalid task scheduling sheets in the information task scheduling queue, and merging the redundant task scheduling sheets to obtain the effective information task scheduling queue.

[0024] In some embodiments, the business library includes a basic library and a benchmark database, and the basic library stores support data corresponding to information in the benchmark database.

[0025] In some embodiments, the information warehousing type includes new creation, modification or deletion of change information.

[0026] In some embodiments, between steps S4 and S5, the method further includes: cleaning the warehousing table by a warehousing rule, the warehousing rule including a warehousing weight and / or a warehousing approval.

[0027] In some embodiments, the warehousing table cleaning process specifically includes:

[0028] S41: judging whether the warehousing table has passed an approval process according to the warehousing rule, if yes, approving the warehousing table according to the approval process, and judging whether to adjust the warehousing table in the approval process, if yes, adjusting the warehousing table, and then performing step S5, otherwise, directly performing step S5; and / or

[0029] S42: comparing the warehousing weights of the warehousing tables in the warehousing rule, and selecting valid warehousing tables according to the comparison result to perform step S5.

[0030] In a second aspect, embodiments of the present application provide a data cleaning and merging device based on a multi-source system, including:

[0031] The task scheduling module is configured to construct a business library based on a meta-model, listen to data change information of a plurality of source systems, and generate a task scheduling sheet based on the data change information;

[0032] The queue cleaning module is configured to obtain the task scheduling sheet according to the warehousing configuration of the business library to generate an information task scheduling queue, clean the information task scheduling queue, and obtain a valid information task scheduling queue;

[0033] The warehousing table generation module is configured to generate a warehousing table according to the valid information scheduling task queue, and establish a transfer relationship between the source system and the business library and an information warehousing type in the warehousing table;

[0034] The warehousing information generation module is configured to obtain data of the source system according to the task scheduling sheet in the valid information scheduling task queue, obtain change information based on an information mapping relationship between the source system and the business library, and write the change information into the warehousing information in the warehousing table; and

[0035] The warehousing module updates data of the business library according to the warehousing information and the information warehousing type.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, comprising one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0037] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, having stored thereon a computer program, when the computer program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0038] Compared with the prior art, the present application has the following beneficial effects:

[0039] 1. The present application can efficiently clean the data of multiple source systems and transfer them to a business library established based on a meta model. The cleaned reference data can be dynamically maintained through the meta model and the metadata, and the reference data can be completely and dynamically stored.

[0040] 2. The database is managed as an actual warehouse object. All the required warehousing information after cleaning is converted into a warehousing table form, which can clearly mark the information source, update time, change content, etc. The information warehousing table can be audited, and the warehousing table can be manually adjusted during the audit. Through the warehousing table, the dynamic changes of the entire historical life cycle of the data can be traced.

[0041] 3. The data update frequency, update rule, information weight, etc. can be dynamically set according to the update rule in the warehousing configuration of the business library.

[0042] 4. The data of multiple source systems can be integrated into standardized data, and a basic information data platform can be built around the regional construction planning, the overall architecture of the smart region can be improved, the pain points can be solved, the management and control ability of the region can be improved, and the support for scientific management and digital operation can be provided. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0044] Figure 1 is an exemplary device architecture diagram to which an embodiment of the present application can be applied;

[0045] Figure 2 is a flowchart of the data cleaning and merging method based on multiple source systems of an embodiment of the present application;

[0046] Figure 3A schematic diagram of a meta model constructed for a multi-source system based data cleaning and merging method of an embodiment of the present application;

[0047] Figure 4 A schematic diagram of an entry table of a multi-source system based data cleaning and merging method of an embodiment of the present application;

[0048] Figure 5 A schematic diagram of a multi-source system based data cleaning and merging device of an embodiment of the present application;

[0049] Figure 6 A structural schematic diagram of a computer device of an electronic device suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0051] Figure 1 An exemplary device architecture 100 to which a multi-source system based data cleaning and merging method or a multi-source system based data cleaning and merging device of an embodiment of the present application can be applied is shown.

[0052] As shown in Figure 1 , the device architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a communication link medium between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0053] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various applications, such as data processing applications, file processing applications, etc. can be installed on the terminal devices 101, 102, 103.

[0054] The terminal devices 101, 102 and 103 can be hardware or software. When the terminal devices 101, 102 and 103 are hardware, they can be various electronic devices including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc. When the terminal devices 101, 102 and 103 are software, they can be installed in the above-mentioned electronic devices. They can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services) or as a single software or software module. No specific limitation is made herein.

[0055] The server 105 can be a server providing various services, for example, a background data processing server processing files or data uploaded by the terminal devices 101, 102 and 103. The background data processing server can process the obtained files or data to generate a processing result.

[0056] It should be noted that the method for cleaning and merging data based on multiple source systems provided by the embodiments of the present application can be executed by the server 105 or the terminal devices 101, 102 and 103, and accordingly, the data cleaning and transferring device can be arranged in the server 105 or the terminal devices 101, 102 and 103.

[0057] It should be understood that Figure 1 The number of terminal devices, networks and servers in the system architecture is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers. In the case where the data to be processed does not need to be obtained from a remote place, the above-mentioned device architecture can not include a network, but only a server or a terminal device.

[0058] The embodiments of the present application take enterprise data in a park as an example. The source systems include data of existing systems, third-party data, park input data and enterprise maintenance data, etc. The data of existing systems include building system databases, parking system databases, property system databases and housing rental system databases. The third-party systems include business information databases, etc. The business library includes enterprise library, personnel library, resource library and basic library.

[0059] Figure 2 A method for cleaning and merging data based on multiple source systems is shown, which includes the following steps:

[0060] S1, a business library based on a meta-model is constructed, data change information of multiple source systems is listened to, and a task scheduling sheet is generated based on the data change information.

[0061] In specific embodiments, as shown in Figure 3As shown, the business library includes a basic library and a benchmark database, the benchmark database includes an enterprise library, a personnel library and a resource library, and the basic library stores support data corresponding to information in the benchmark database. By establishing a data structure with a meta-model as a framework in the benchmark database, and setting data sources and updating rules, the same enterprise, individual and asset data appearing in different source systems are merged into the benchmark database. The meta-data of the meta-model in the benchmark database includes information name, information code, information type, source (source system), operation, etc. The meta-model and meta-data after data cleaning and merging are as follows Figure 3 As shown, the enterprise library contains enterprise data such as business data, office address, employee information, park access time, etc., the personal library stores personal data including gender, profession, affiliated enterprise, parking space information, contact number, etc., and the asset library stores asset information including park information, resident enterprise, building information, leasing status, advertising site information, etc.

[0062] In specific embodiments, step S1 specifically includes generating a task scheduling sheet through a trigger after data change information is generated in the source system, and establishing communication between the source system and the business library through the task scheduling sheet. The task scheduling sheet of the source system can be transmitted back to the business library through an API interface management system or other data extraction system, and the source system is informed through a callback interface. After receiving the callback information, the source system clears the source system task scheduling queue. The data communication frequency can be set by setting the listening frequency or the source system actively pushes the task scheduling sheet to the business library in real time through the API interface defined by the business library.

[0063] S2, according to the entry configuration of the business library, obtain the task scheduling sheet to generate the information task scheduling queue, clean the information task scheduling queue, and obtain the valid information task scheduling queue.

[0064] In specific embodiments, step S2 specifically includes: removing invalid task scheduling sheets in the information task scheduling queue, and merging redundant task scheduling sheets to obtain the valid information task scheduling queue.

[0065] In specific embodiments, the entry configuration of the business library includes entry frequency and entry rules. According to the entry frequency set in the entry configuration of the business library, the task scheduling sheets in the information task scheduling queue are cleaned, and redundant and invalid tasks are removed from the information task scheduling queue. The entry frequency can be set to every day or every week according to demand, for example, if the same enterprise information is changed N times, N scheduling tasks will appear, but the last changed information needs to be compared with the enterprise information already stored in the business library, and the task scheduling sheet of the last changed information is saved.

[0066] S3, according to the valid information scheduling task queue, generate an entry table, and establish the transfer relationship between the source system and the business library and the information entry type in the entry table.

[0067] In a specific embodiment, after the task scheduling sheet is successfully stored, the task scheduling sheet is removed from the information scheduling task queue and enters the historical task, maintaining the high performance of the task queue. If the task scheduling sheet fails to be stored, the scheduling task number is increased by 1, and if the scheduling task number exceeds 3 times without success, the operation will not be performed again, and a warning is given. Therefore, the task scheduling sheet with a calling number of 3 times or less is obtained in the effective information scheduling task queue, and it is judged whether there is a mapping relationship between the data of the source system and the data of the business library in the task scheduling sheet. If there is no mapping relationship, the mapping relationship needs to be constructed according to the corresponding relationship rule between the data obtained by the task scheduling sheet. After the mapping relationship is established, the storage table can be converted into the storage table according to the business library, the source system and the information storage type through the effective information scheduling task queue, wherein the mapping relationship includes message mapping, data mapping and conversion mapping. The information storage type includes new, modification or deletion, and the storage table records the source system (the data source to be stored) and the business library to be stored, so that the transfer relationship between the business library and the source system is established. At this time, there is no storage information when the storage table is generated, and the storage information can be obtained after the data change information is changed and compared according to the mapping relationship.

[0068] S4, obtaining the data of the source system according to the task scheduling sheet in the effective information scheduling task queue, obtaining the change information based on the information mapping relationship between the source system and the business library, and writing the change information into the storage information in the storage table.

[0069] The storage information includes source information value and host information value. The source information value is the information value (changed information value) of the data change information in the source system, and the host information value is the information value (changed information value) of the data change information in the business library. There is an information mapping relationship between the source information value and the host information value. The source information value is obtained by data extraction according to the task scheduling sheet in the effective information scheduling task queue, the host information value is obtained in the business library through the information mapping relationship, and the storage information of the storage table is generated by change comparison. The configuration of the storage table and the storage information are as shown in Figure 4

[0070] In a specific embodiment, between steps S4 and S5, the storage table is cleaned by the storage rule set by the business library, and the storage rule includes storage weight and / or storage approval.

[0071] In a specific embodiment, the step of cleaning the storage table specifically includes:

[0072] ​S41: judging whether the storage table has passed the approval process according to the storage rule, if yes, approving the storage table according to the approval process, and judging whether the storage table is adjusted in the approval process, if yes, adjusting the storage table, and then performing step S5, otherwise, directly performing step S5; and / or

[0073] S42: comparing the storage weights corresponding to the storage tables in the storage rule, and selecting the effective storage table according to the comparison result to perform step S5.

[0074] In a specific embodiment, it is judged according to the storage rule of the storage table whether the storage table needs to be approved or directly stored, if the storage table needs to be approved, the storage table is approved according to the approval process, and the information in the storage table is adjusted in the approval process, and the adjusted storage table is performed in step S5. Because there are multiple source systems, there are multiple storage tables corresponding to the business library, so the storage weights of the storage tables obtained from different source systems are set, the storage weights are compared, and the appropriate storage table is selected to perform step S5. For example, the storage tables can be sorted according to the high and low of the storage weight corresponding to each storage information, and the storage table with high storage weight is preferentially stored.

[0075] S5, according to the storage information and the information storage type, the metadata in the meta model of the business library is changed.

[0076] In a specific embodiment, the storage information is obtained, it is judged whether to add, modify or delete according to the information storage type, if it is added, the added data is inserted in the business library form, and the foreign key is bound to establish a mapping relationship with the business master table, and the business library is updated according to the storage information; if it is modified, the business library is directly updated according to the storage information; if it is deleted, the mapping relationship between the source system and the business system is removed for the data, and the basic library information is deleted.

[0077] Further referring to Figure 5 , as an implementation of the method shown in the above figures, the application provides an embodiment of a data cleaning and merging device based on multiple source systems, which corresponds to the method embodiment shown in Figure 2 , and the device can be applied to various electronic devices.

[0078] Embodiments of the application provide a data cleaning and merging device based on multiple source systems, which comprises:

[0079] The task scheduling module 1 is configured to construct a business library based on a meta model, listen to data change information of multiple source systems, and generate a task scheduling single based on the data change information;

[0080] The queue cleaning module 2 is configured to obtain task scheduling orders based on the inbound configuration of the business database to generate an information task scheduling queue, clean the information task scheduling queue, and obtain a valid information task scheduling queue.

[0081] The inbound table generation module 3 is configured to generate an inbound table based on an effective information scheduling task queue, and establish the transfer relationship and information inbound type between the source system and the business database in the inbound table;

[0082] The inbound information generation module 4 is configured to obtain data from the source system based on task schedules in the valid information scheduling task queue, obtain change information based on the information mapping relationship between the source system and the business database, and write the change information into the inbound information in the inbound table; and

[0083] The inbound module 5 updates the data in the business database based on the inbound information and the inbound information type.

[0084] The following is for reference. Figure 6 It illustrates an electronic device suitable for implementing embodiments of this application (e.g., Figure 1 The diagram shows the structure of a computer device 600 (a server or terminal device). Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0085] like Figure 6 As shown, the computer device 600 includes a central processing unit (CPU) 601 and a graphics processing unit (GPU) 602, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 603 or programs loaded from storage section 609 into random access memory (RAM) 604. The RAM 604 also stores various programs and data required for the operation of the device 600. The CPU 601, GPU 602, ROM 603, and RAM 604 are interconnected via a bus 605. An input / output (I / O) interface 606 is also connected to the bus 605.

[0086] The following components are connected to I / O interface 606: an input section 607 including a keyboard, mouse, etc.; an output section 608 including an LCD, speakers, etc.; a storage section 609 including a hard disk, etc.; and a communication section 610 including a network interface card, such as a LAN card or modem. The communication section 610 performs communication processing via a network such as the Internet. A drive 611 may also be connected to I / O interface 606 as needed. A removable medium 612, such as a hard disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 611 as needed so that computer programs read from it can be installed into storage section 609 as needed.

[0087] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program in accordance with embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 610, and / or installed from the removable medium 612. When the computer program is executed by the central processing unit (CPU) 601 and the graphics processor (GPU) 602, the above-described functions defined in the methods of the present application are executed.

[0088] It should be noted that the computer readable medium described in the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, device or means, or any combination of the above. More specific examples of computer readable medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, device or means. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable medium that can send, propagate or transmit the program for use by or in conjunction with an instruction execution device, device or means. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0089] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0090] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0091] The modules described in the embodiments of the present application can be implemented by software, or by hardware. The described modules can also be arranged in a processor.

[0092] As another aspect, the present application also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: construct a business library based on a meta model, listen to data change information of a plurality of source systems, generate a task scheduling sheet based on the data change information; obtain the task scheduling sheet according to the in-library configuration of the business library to generate an information task scheduling queue, clean the information task scheduling queue to obtain an effective information task scheduling queue; generate an in-library table according to the effective information scheduling task queue, and establish a transfer relationship between the source system and the business library and an information in-library type in the in-library table; obtain data of the source system according to the task scheduling sheet in the effective information scheduling task queue, obtain change information based on the information mapping relationship between the source system and the business library, and write the change information into the in-library information in the in-library table; and change metadata in the meta model of the business library according to the in-library information and the information in-library type.

[0093] The above description is merely the preferred embodiments of the present application and the explanation of the applied technical principles. It should be understood by those skilled in the art that the inventive scope involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features can be replaced with the technical features disclosed in the present application (but not limited to) having similar functions to form the technical solutions.

Claims

1. A data cleaning and merging method based on a multi-source system, characterized in that, Includes the following steps: S1. Construct a business library based on a meta-model, monitor data change information from multiple source systems, and generate a task scheduling order based on the data change information. S2, obtain the task scheduling order according to the inbound configuration of the business database to generate an information task scheduling queue, clean the information task scheduling queue to obtain an effective information task scheduling queue, the inbound configuration of the business database includes inbound frequency and inbound rules; S3, Generate an entry table based on the effective information scheduling task queue, and establish the transfer relationship and information entry type between the source system and the business database in the entry table; S4. Obtain data from the source system based on the task scheduling order in the valid information scheduling task queue, obtain change information based on the information mapping relationship between the source system and the business database, and write the change information into the entry information in the entry table. The inbound table is cleaned using the inbound rules, which include inbound weights and / or inbound approvals. The inbound table cleaning process specifically includes: S41: Determine whether the inbound form has undergone an approval process according to the inbound rules. If so, approve the inbound form according to the approval process, and determine whether to adjust the inbound form during the approval process. If so, adjust the inbound form and then proceed to step S5; otherwise, proceed directly to step S5; and / or S42: Compare the entry weights corresponding to the entry tables in the entry rules, and filter out valid entry tables based on the comparison results to proceed to step S5; and S5, Modify the metadata in the metamodel of the business database according to the database entry information and the database entry type.

2. The data cleaning and merging method based on a multi-source system according to claim 1, characterized in that, Step S1 specifically includes: after the data change information is generated in the source system, the task schedule is generated through a trigger, and communication between the source system and the business database is established through the task schedule.

3. The data cleaning and merging method based on a multi-source system according to claim 1, characterized in that, Step S2 specifically includes: The task scheduling order is obtained according to the inbound frequency, and the information task scheduling queue is formed. The invalid task schedules in the information task scheduling queue are cleared, and the redundant task schedules are merged to obtain the valid information task scheduling queue.

4. The data cleaning and merging method based on a multi-source system according to claim 1, characterized in that, The business database includes a basic database and a benchmark database. The basic database stores the supporting data corresponding to the information in the benchmark database. The information entry types include the creation, modification, or deletion of the change information.

5. A data cleaning and merging device based on a multi-source system, characterized in that, include: The task scheduling module is configured to build a business library based on a meta-model, monitor data change information from multiple source systems, and generate task scheduling orders based on the data change information. The queue cleaning module is configured to obtain the task scheduling order according to the inbound configuration of the business database to generate an information task scheduling queue, clean the information task scheduling queue to obtain an effective information task scheduling queue, and the inbound configuration of the business database includes inbound frequency and inbound rules. The inbound table generation module is configured to generate an inbound table based on the valid information scheduling task queue, and to establish the transfer relationship and information inbound type between the source system and the business database in the inbound table; The inbound information generation module is configured to obtain data from the source system based on the task schedule in the valid information scheduling task queue, obtain change information based on the information mapping relationship between the source system and the business database, and write the change information into the inbound information in the inbound table. The inbound table is cleaned using the inbound rules, which include inbound weights and / or inbound approvals. The inbound table cleaning process specifically includes: Based on the stated inbound rules, determine whether the inbound table has undergone an approval process. If so, approve the inbound table according to the approval process, and within the approval process, determine whether the inbound table needs adjustment. If so, adjust the inbound table before proceeding to the inbound module; otherwise, proceed directly to the inbound module. And / or The system compares the entry weights corresponding to the entry tables in the entry rules, and selects valid entry tables for entry into the entry module based on the comparison results; and The data entry module updates the data in the business database based on the data entry information and the data entry type.

6. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

Images