Data checking method, device, system and electronic equipment

Abstract

The application discloses a data checking method, device, system and electronic equipment, and the method comprises the steps of receiving and analyzing a checking task to obtain configuration information of the checking task, the checking task being published to a cloud service platform by the checking management platform in response to a checking task establishment instruction; establishing a checking flow application of the checking task according to the configuration information, the checking flow application comprising at least a pulling component, at least one processing component and a converging component; calling the pulling component to pull first data from at least one target data source according to the configuration information; calling the processing component to pull second data from a corresponding business system, and checking the first data and the second data according to the configuration information; and calling the converging component to converge the checking result to obtain a data checking result. The application realizes cross-system checking of data, can support online configuration and one-key deployment, realizes flexible configuration of data checking and rapid iteration of changes.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a data verification method, apparatus, system, and electronic device. Background Technology

[0002] The development of software products generates a lot of data. As data flows between different systems, inconsistencies can easily arise due to various reasons. Inconsistencies in valuable data can lead to many anomalies. Therefore, in some business scenarios, it is necessary to verify data from different business systems to identify abnormal data.

[0003] Cross-system data verification presents numerous challenges, such as data silos and the chimney dilemma. As business processes become increasingly lengthy, data is stored independently across systems, making it prone to distributed system problems, network errors, and system anomalies. Timely detection of data inconsistencies is extremely difficult; many inconsistencies are only discovered after operational feedback, failing to be detected promptly when changes or failures occur. Furthermore, frequent changes and complex logic mean that constantly evolving business processes lead to changes in reconciliation rules and data. Each data verification requires experienced staff to rapidly update the verification rules through code development, consuming significant human and time resources.

[0004] In existing technologies, data verification or acquisition is carried out using traditional programmatic methods, or mainly focuses on verification within a single system, or lacks universality and is difficult to meet the needs of large-scale cross-system data verification. Summary of the Invention

[0005] In view of the above-mentioned situation, this application proposes a data verification method, apparatus, system and electronic device to overcome or partially overcome the shortcomings of the prior art.

[0006] In a first aspect, embodiments of this application provide a data verification method, which is implemented through a data verification system, the verification system including a verification management platform and a cloud service platform connected by communication;

[0007] The method is applied to the cloud service platform, and the method includes:

[0008] The system receives and parses verification tasks to obtain configuration information for the verification tasks, which are published to the cloud service platform by the verification management platform in response to the verification task creation instruction.

[0009] A verification stream application for the verification task is established based on the configuration information. The verification stream application includes at least a pull component, at least one processing component, and a convergence component.

[0010] The pull component is invoked to pull first data from multiple target business systems according to the configuration information;

[0011] The processing component is invoked to retrieve the second data from the corresponding business system, and the first data and the second data are verified according to the configuration information.

[0012] The aggregation component is invoked to summarize the results obtained from the verification, thus obtaining the data verification results.

[0013] Secondly, embodiments of this application also provide a data verification method, which is implemented through a data verification system, the verification system including a verification management platform and a cloud service platform connected by communication;

[0014] The method is applied to the verification management platform, and the method includes:

[0015] A configuration interface is provided, which includes a verification task configuration control, and the task configuration control includes a data source configuration sub-control and a verification rule configuration sub-control.

[0016] Obtain the configuration information for the verification task through the configuration interface;

[0017] A verification task is generated based on the configuration information, and the verification task is published to the cloud service platform.

[0018] Thirdly, embodiments of this application also provide a data verification device, which can implement any of the methods described above.

[0019] Fourthly, embodiments of this application also provide a data verification system, which includes a verification management platform and a cloud service platform connected by communication. The verification management platform is equipped with a data verification device, which can realize the data verification method applied to the verification management platform as described above. The cloud service platform is equipped with a data verification device, which can realize the data verification method applied to the cloud service platform as described above.

[0020] Fifthly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0021] Sixthly, embodiments of this application also provide a computer-readable storage medium storing one or more programs that, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.

[0022] The above at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects:

[0023] The present application designs a checking system, which comprises a checking management platform and a cloud service platform connected in communication. The checking management platform is regarded as a client of the cloud service platform. A checking personnel can configure a checking task on the checking management platform and publish the checking task on the cloud service platform. After receiving the management task, the cloud service platform can establish a checking application according to the configuration information of the checking application. The checking application comprises a plurality of flow applications which can be configured in advance. The flow applications pull first data and second data from a plurality of systems according to the configuration information and complete data checking and result aggregation. The present application can gather data of different data sources together for comparison and processing, realizes cross-system checking of data, supports online configuration and one-key deployment, realizes flexible configuration and rapid iteration of data checking, and separates business and checking, so that data checking does not affect business access. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate the illustrative embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0025] Figure 1 Fig. 1 shows a structure schematic diagram of a data checking system according to one embodiment of the present application;

[0026] Figure 2 Fig. 2 shows a flow schematic diagram of a data checking method according to one embodiment of the present application;

[0027] Figure 3 Fig. 3 shows a flow schematic diagram of a data checking method according to another embodiment of the present application;

[0028] Figure 4 Fig. 4 shows a flow schematic diagram of a data checking method according to still another embodiment of the present application;

[0029] Figure 5 Fig. 5 shows a structure schematic diagram of a first data checking device according to one embodiment of the present application;

[0030] Figure 6 Fig. 6 shows a structure schematic diagram of a second data checking device according to one embodiment of the present application;

[0031] Figure 7 Fig. 7 shows a structure schematic diagram of an electronic device according to one embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0033] The technical solutions provided by the embodiments of the present application will be described in detail below in combination with the drawings.

[0034] The idea of the present application is that, in view of the pain points of cross-system data checking in the prior art, a data checking system is designed, which at least includes a communication connection checking management platform and a cloud service platform. The checking management platform is taken as the cloud service platform, and checking personnel can submit checking tasks on the management platform at any time. The cloud service platform can pull corresponding data from multiple business systems according to the checking tasks to perform data checking.

[0035] Figure 1 A structural schematic diagram of a data checking system according to an embodiment of the present application is shown, from Figure 1 It can be seen that the data checking system 100 includes a checking management platform 110 and a cloud service platform 120, and the checking management platform 110 and the cloud service platform 120 are in communication connection. Among them, the cloud service platform 120 can be any kind of cloud service in the prior art, such as a Kubernetes platform.

[0036] The Kubernetes platform is a portable and scalable open source platform for managing containerized workloads and services, which can promote declarative configuration and automation. Kubernetes has a large and rapidly growing ecosystem, and its services, support and tools are widely used.

[0037] Figure 2 A flowchart of a data checking method according to an embodiment of the present application is shown, the method is applied to the cloud service platform, and the method at least includes steps S210-S250:

[0038] Step S210: receiving and parsing a checking task to obtain configuration information of the checking task, the checking task being published to the cloud service platform by the checking management platform in response to a checking task establishment instruction.

[0039] The data checking method of the present application can be applied to Figure 1The illustrated data checking system 100 is implemented and applied to the cloud service platform 120 of the data checking system 100. In use, the checking management platform 110 is a client of the cloud service platform 120. The checking management platform 110 is a platform for checking personnel. The checking personnel can configure a checking task on the checking management platform 110. After the configuration is completed, the checking personnel can initiate a task establishment instruction on the checking management platform 110. The checking management platform 110 publishes the checking task to the cloud service platform 120 in response to the task establishment instruction.

[0040] The checking personnel configures a checking task on the checking management platform 110. The configuration mainly includes two core configurations, one is a data source to be processed, and the other is a checking rule for checking the data.

[0041] After the cloud service platform 120 receives a checking task, the cloud service platform 120 first analyzes the checking task to obtain configuration information of the checking task. The configuration information at least includes a data source of first data and a checking rule.

[0042] Step S220: Establish a checking flow application of the checking task according to the configuration information. The checking flow application at least includes a pulling component, at least one processing component, and a converging component.

[0043] After the configuration information of the checking task is obtained, a checking flow application of the checking task needs to be established on the cloud service platform 120. The checking flow application at least includes the following three components: a pulling component, a processing component, and a converging component.

[0044] The so-called flow application is an application that is processed or loaded through a stream, that is, it is used as soon as it is installed. The general process of checking data is to pull data, process data, and aggregate results. Therefore, the flow application of the present application at least includes the three components of the pulling component, the processing component, and the converging component, but is not limited thereto. The flow application composed of the three components of the pulling component, the processing component, and the converging component can be understood as a general flow application template for data checking. The flow application is deployed on the cloud service platform 120 in advance. When data checking is performed, the flow application can be directly called. Only the relevant content needs to be replaced according to the configuration information, such as a data source and a checking rule.

[0045] It should be noted that the flow application is deployed on the cloud service platform 120 in advance. However, the so-called "deployment" does not mean installation. In some embodiments, in order to save the hardware expenditure and occupation of the cloud service platform 120, the flow application is not installed when it is not used, but is installed and used through stream loading when it is used.

[0046] In addition, the developed data checking script can be formed into a component according to the personalized needs of data checking, and stored on the cloud service platform 120. When the flow application is established according to a checking task, the required component can be queried and loaded from the plurality of functional components stored on the cloud service platform 120 according to the setting of the configuration information, so as to quickly realize the rapid establishment of the checking flow application.

[0047] In some cases, the data processing method of each business system can be different, and therefore the generality of the processing component can be poor. Therefore, in order to ensure the smooth progress of data checking, a corresponding processing component can be configured for each business. When the flow application is established, the corresponding processing component is called according to the current business needs. For example, the processing component 1 corresponds to the business A, and the processing component 2 corresponds to the business B. The flow application deployed on the cloud service platform 120 can increase the container as needed to accommodate more components, and support the checking of massive business data.

[0048] After the checking flow application is established, each component in the checking flow application is called, and the corresponding operation is sequentially executed according to the function of each component. In this embodiment, steps S230-S250 are sequentially executed:

[0049] Step S230: calling the pulling component, and pulling the first data from at least one target data source according to the configuration information.

[0050] Step S240: calling the processing component, pulling the second data from the corresponding business system, and checking the first data and the second data according to the configuration information.

[0051] Step S250: calling the converging component, and converging the result obtained by checking to obtain a data checking result.

[0052] That is, the pulling component is first called, and the pulling component is mainly responsible for pulling the first data to be checked from the first data source upstream configured in the configuration information.

[0053] Then the processing component is called, which can pull the second data of the business system corresponding to the processing component and match the first data with the second data. The business system corresponding to the processing component can be the business system where the reconciliation management platform 110 is located, i.e., the current business system, or a third-party business system. For example, the upstream business system is system A, the business system where the reconciliation management platform 110 is located is system B, and the reconciliation personnel can configure the target data source as business system A in the reconciliation management platform 110, and the current business system is system B by default, i.e., the business system corresponding to the processing component is system B. When data reconciliation is performed, the pulling component pulls the first data from business system A, and the processing component can pull the second data from business system B. For another example, the upstream business system is system A, and the third-party business system is system C. The reconciliation personnel can configure the target data source as business system A and the third-party business system as system C in the reconciliation management platform 110. When data reconciliation is performed, the pulling component pulls the first data from business system A, and the processing component can pull the second data from business system C to reconcile the first data and the second data.

[0054] In the reconciliation process, the reconciliation rule can be obtained from the configuration information, and the comparison and matching are performed according to the reconciliation rule, so that the abnormal situation is found.

[0055] The aggregation component is mainly used for aggregating the results obtained by reconciliation to obtain the data reconciliation result.

[0056] From Figure 2 As can be seen from the method shown in the figure, the present application designs a reconciliation system, which includes a reconciliation management platform and a cloud service platform connected in communication. The reconciliation management platform is regarded as a client of the cloud service platform. The reconciliation personnel can configure the reconciliation task on the reconciliation management platform and publish the reconciliation task to the cloud service platform. After receiving the management task, the cloud service platform can establish a reconciliation application according to the configuration information. The reconciliation application includes a plurality of stream applications that can be configured in advance. The stream applications pull the first data and the second data from a plurality of systems according to the configuration information and complete data reconciliation and result aggregation. The present application can gather data from different data sources for comparison and processing, realizes cross-system data reconciliation, supports online configuration and one-key deployment, realizes flexible configuration and rapid iteration of data verification, and separates business and reconciliation, which does not affect business access at all.

[0057] In some embodiments of the present application, in the above method, the data checking system further comprises a visual observation platform connected to the cloud server platform; the method further comprises: sending the data checking result to the visual observation platform to enable the visual observation platform to display the data checking result; and / or sending the abnormal result to a specified terminal through a specified communication method.

[0058] In order to facilitate the checking personnel to intuitively view the checking result, in some embodiments of the present application, an observable platform can also be provided in the checking system. The observable platform can be a Grafana (an open observable platform, no unified Chinese name in the industry) platform or a prometheus (a monitoring system and time series database, no unified Chinese name in the industry) platform, etc. The observable platform is communicatively connected to the cloud server platform. After confirming the checking result, the cloud service platform can send the checking result to the observable platform for display. If there is abnormal data in the checking result, the terminal device (such as a mobile phone) of the person in charge of the checking task can be sent through a specified communication means (such as a short message) to remind the checking personnel to query the abnormal data on the observable platform.

[0059] In some embodiments of the present application, in the above method, the establishment of the checking flow application of the checking task according to the configuration information comprises: providing a resource scheduling interface, and in response to an interface calling instruction corresponding to the checking task, establishing the checking flow application of the checking task in the cloud service platform.

[0060] Please refer to Figure 1 , the checking management platform 110 publishes the checking task on the cloud service platform 120 through the resource scheduling interface provided by the cloud service platform 120. The checking management platform 110 can be regarded as a client of the cloud service platform 120. The checking management platform 110 sends the interface calling instruction corresponding to the checking task to the cloud service platform 120 by calling the interface. The resource scheduling interface receives the interface calling instruction and establishes the checking flow application of the checking task in the environment of the cloud service platform 120.

[0061] In some embodiments of the present application, in the above method, the configuration information comprises at least one target data source and a checking rule; the calling of the pulling component to pull the first data from at least one target data source according to the configuration information comprises: establishing a communication connection with the target data source; and pulling the first data from the target data source based on the communication connection. The checking of the first data and the second data comprises: performing operation on the first data and the second data according to the checking rule to confirm whether the first data and / or the second data is abnormal.

[0062] The pulling component first establishes a communication connection with the target data source when pulling data from the target data source. Specifically, a connection request can be sent to the target data source, and then feedback of the target data source is received. If the feedback is that the connection can be established, a communication connection with the target data source is established. The communication connection can be a connection based on plaintext or a connection based on a key (in cooperation with a public key), such as a Talent connection.

[0063] The target data source can be a MySQL database, an oracle database, or the like, or a source file, a web page, or the like. When pulling data from a database, data can be retrieved according to a field, a keyword, or the like, and the retrieved data can be pulled. When pulling data from a source file and a web page, a regular expression can be used to extract data, and then related data can be pulled.

[0064] The processing component compares and matches the first data and the second data according to the comparison rule parsed from the configuration information when performing data comparison. Data that meets the comparison rule is considered normal data, and data that does not meet the comparison rule is considered abnormal data. The comparison and matching can determine that the first data, the second data, or both the first data and the second data are abnormal.

[0065] For example, the first data is an order number obtained from a web page, and the second data is a corresponding order number in a current business system. The comparison rule limits the first data as a reference to determine whether the second data is consistent with the first data. If the second data is consistent with the first data, the second data is determined to be normal. If the second data is not consistent with the first data, the second data is determined to be abnormal. In the present application, the comparison rule is not limited and can be configured according to a specific business.

[0066] In some embodiments of the present application, in the above method, the processing component is a plurality of processing components, each processing component corresponds to a business system, and the plurality of processing components are connected according to a dependency relationship of the corresponding business. The processing component is called to pull the second data from the corresponding business system, and the first data and the second data are compared according to the configuration information. The method further includes: a preceding processing component transmits at least one of the first data, the second data of the corresponding business, and a comparison result of the preceding processing component to a subsequent processing component, so that the subsequent processing component performs data comparison.

[0067] In actual process, there are dependencies or relationships between businesses many times, so one data may need to be checked with data in multiple business systems, at this time, multiple processing components can be set, each processing component corresponds to a business system, multiple components are connected according to the dependency relationship of the business, such as business C depends on business B, business B depends on business A, then 3 processing components can be set, processing component 1 corresponds to business A, processing component 2 corresponds to business B, and component 3 corresponds to business C, processing component 1 is in communication connection with business system A of business A; processing component 2 is in communication connection with business system B of business B; processing component 3 is in communication connection with business system C of business C. When checking data, the pulling component first pulls the first data from the target data source, the pulling component first transmits the first data to the processing component 1, the processing component 1 pulls the second data of business A from the corresponding business system A, checks the first data and the second data according to the checking rule, and obtains the first checking result; then the processing component 1 transmits the first data and / or the second data and the first checking result (if necessary) to the processing component 2, the processing component 2 can pull the third data of business B from the corresponding business system B, compare the third data with at least one of the first data and the second data according to the checking rule, and obtain the second checking result; in turn, the processing component 3 obtains the third checking result. If more components are set, the data can be transmitted to more components for comparison of more business data. However, it should be noted that at this time, based on the capacity of the cloud service platform, when the number of components increases, the container of the cloud service platform needs to be expanded.

[0068] Figure 3 A flowchart of a data checking method according to another embodiment of the application is shown, from Figure 3 It can be seen that the method of the embodiment is applied to the checking management platform 110 Figure 1 The method comprises the following steps:

[0069] Step 310: providing a configuration interface, the configuration interface comprising a checking task configuration control, the checking task configuration control comprising a data source configuration sub-control and a checking rule configuration sub-control.

[0070] As described above, the checking management platform 110 as a client of the cloud service platform 120 is for checking personnel, in order to facilitate the operation of the checking personnel, a configuration interface is provided, in which a plurality of task configuration controls are deployed, the checking task configuration control comprises a data source configuration sub-control and a checking rule configuration sub-control. The checking personnel can directly add the target data source and the checking rule on the configuration interface provided by the checking management platform 110. It should be noted that the configuration information is not limited to the scenario of simple options or simple expressions, but also can be a script for checking data, the script can be directly imported into the configuration item of the checking management platform 110.

[0071] Step 320: Obtain the configuration information of the reconciliation task through the configuration interface.

[0072] The reconciliation management platform 110 obtains the configuration information of the reconciliation task through the configuration interface, and the configuration information includes but is not limited to at least one target data source and reconciliation rules, etc.

[0073] Step 330: Generate the reconciliation task according to the configuration information, and publish the reconciliation task to the cloud service platform.

[0074] After obtaining the configuration information, the reconciliation management platform 110 generates the reconciliation task and publishes the reconciliation task to the cloud service platform 120. Specifically, the resource scheduling interface provided by the cloud service platform 120 is called, and the interface call instruction of the reconciliation task is sent to the cloud service platform 120, so as to realize the publication of the reconciliation task. As mentioned above, the cloud service platform 120 receives the interface call instruction through the resource scheduling interface, and establishes the reconciliation flow application of the reconciliation task, which means that the reconciliation task is published successfully.

[0075] Figure 4 A flowchart of a data reconciliation method according to another embodiment of the application is shown, from Figure 4 It can be seen that the present embodiment includes:

[0076] The reconciliation management platform receives the configuration information of the reconciliation task, and generates the reconciliation task according to the configuration information; and calls the resource scheduling interface of the cloud service platform to publish the reconciliation task.

[0077] The cloud service platform receives and parses the reconciliation task to obtain the target data source and the reconciliation rule; and establishes the reconciliation flow application according to the configuration rule

[0078] The cloud service platform calls the pulling component according to the configuration information, establishes a connection with the target data source, and pulls the first data.

[0079] The cloud service platform calls the processing component according to the configuration information, establishes a connection with the corresponding business system, pulls the second data, and calls the processing component to reconcile the first data and the second data according to the reconciliation rule.

[0080] The cloud service platform calls the aggregation component to aggregate the reconciliation result.

[0081] The cloud service platform sends the reconciliation result to the observability platform in response to the query request.

[0082] Figure 5 A structural diagram of a first data reconciliation device according to an embodiment of the application is shown, the first data reconciliation device 500 is deployed in the cloud service platform 120 of the data reconciliation system 100, and the first data reconciliation device 500 includes:

[0083] The receiving and parsing unit 510 is configured to receive and parse a verification task to obtain configuration information of the verification task, the verification task being published by the verification management platform to the cloud service platform in response to a verification task establishment instruction;

[0084] The application establishment unit 520 is configured to establish a verification flow application of the verification task according to the configuration information, the verification flow application comprising at least a pulling component, at least one processing component, and a converging component.

[0085] The calling unit 530 is configured to call the pulling component to pull first data from at least one target data source according to the configuration information, call the processing component to pull second data from a corresponding business system, and perform verification on the first data and the second data according to the configuration information, and call the converging component to aggregate a result obtained through verification to obtain a data verification result.

[0086] In some embodiments of the present application, the verification system further comprises a visual observation platform connected to the cloud server platform; the first data verification device 500 further comprises a result sending unit configured to send the data verification result to the visual observation platform to enable the visual observation platform to display the data verification result; and / or, send an abnormal result in the data verification result to a specified terminal through a specified communication mode.

[0087] In some embodiments of the present application, in the first data verification device 500 described above, the application establishment unit 520 is configured to provide a resource scheduling interface and establish the verification flow application of the verification task on the cloud service platform in response to an interface calling instruction corresponding to the verification task.

[0088] In some embodiments of the present application, in the first data verification device 500 described above, the configuration information comprises at least one target data source and a verification rule; the calling unit 530 is configured to establish a communication connection with the target data source, pull first data from the target data source based on the communication connection, and perform operation on the first data and the second data according to the verification rule to determine whether the first data and / or the second data is abnormal.

[0089] In some embodiments of the present application, in the first data verification device 500 described above, the processing component is a plurality of processing components, each processing component corresponding to a business system, and the plurality of processing components are connected according to a dependency relationship of corresponding businesses.

[0090] The calling unit 530 is further configured to call the pre-processing component and pass at least one of the first data, the second data corresponding to the service, and the checking result of the calling unit 530 to the post-processing component, so that the post-processing component performs data checking.

[0091] Figure 6 A structural diagram of a second data checking apparatus according to an embodiment of the present application is shown. The second data checking apparatus 600 is deployed in the checking management platform 110 of the data checking system 100. The second data checking apparatus 600 includes:

[0092] The configuration unit 610 is configured to provide a configuration interface. The configuration interface includes a checking task configuration control. The checking task configuration control includes a data source configuration sub-control and a checking rule configuration sub-control.

[0093] The obtaining unit 620 is configured to obtain configuration information of a checking task through the configuration interface.

[0094] The publishing unit 630 is configured to generate a checking task according to the configuration information and publish the checking task to the cloud service platform.

[0095] In some embodiments of the present application, in the second data checking apparatus 600 described above, the publishing unit 630 is configured to call a resource scheduling interface provided by the cloud service platform, and send an interface calling instruction of the checking task to the cloud service platform, so as to implement publishing of the checking task.

[0096] It should be noted that the first data checking apparatus 500 and the second data checking apparatus 600 described above can respectively implement the data checking method described above, and thus will not be described herein.

[0097] Figure 7 FIG. 1 is a structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 7 At the hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The memory can include a memory such as a random-access memory (RAM), and can further include a non-volatile memory such as at least one disk memory. Of course, the electronic device can further include other hardware required by a service.

[0098] The processor, the network interface and the memory can be connected with each other through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus or the like. The bus can be divided into an address bus, a data bus, a control bus and the like. For the convenience of representation, Figure 7 Only one bidirectional arrow is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0099] The memory is used to store programs. Specifically, the programs can include program codes including computer operation instructions. The memory can include an internal memory and a non-volatile memory, and provide instructions and data for the processor.

[0100] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs, and forms the data checking device at a logical level. The processor executes the programs stored in the memory, and is specifically used for executing the foregoing method.

[0101] The above as described in the present application Figure 5The method performed by the data checking device disclosed in the embodiments of the present application can be applied to a processor or implemented by the processor. The processor can be an integrated circuit chip with a processing capability. In the implementation, each step of the method can be completed by integrated logic circuits in the processor or instructions in the form of software. The processor can be a general processor, including a central processing unit (CPU), a network processor (NP), etc. It can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the method.

[0102] The electronic device can also execute Figure 5 the method performed by the data checking device in the embodiments of the present application, and implement the functions of the data checking device in the embodiments of the present application. The embodiments of the present application will not be repeated here. Figure 5

[0103] The embodiments of the present application also propose a computer readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by an electronic device including a plurality of applications, can enable the electronic device to execute Figure 5 the method performed by the data checking device in the embodiments of the present application, and specifically for executing the foregoing method.

[0104] ​Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0105] The present application is described in reference to the flowchart illustrations and / or block diagrams according to the embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0106] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0108] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0109] The memory can include non-persistent memory, random access memory (RAM), and / or non-volatile memory, such as read only memory (ROM) or flash memory, among others. The memory is an example of computer-readable media.

[0110] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0111] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0112] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system or computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] The above only describes the embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A data collation method, characterized by, The method is implemented through a data checking system, and the checking system comprises a checking management platform and a cloud service platform connected in communication; The method is applied to the cloud service platform, and the method comprises: receiving and analyzing a checking task to obtain configuration information of the checking task, the checking task being published to the cloud service platform by the checking management platform in response to a checking task establishment instruction; establishing a checking flow application of the checking task according to the configuration information, the checking flow application comprising at least a pulling component, at least one processing component and a converging component; calling the pulling component to pull first data from at least one target data source according to the configuration information; calling the processing component to pull second data from a corresponding business system and to check the first data and the second data according to the configuration information; calling the converging component to converge results obtained through checking to obtain a data checking result.

2. The method of claim 1, wherein, The checking system further comprises a visual observation platform connected to the cloud service platform; The method further comprises: sending the data checking result to the visual observation platform to enable the visual observation platform to display the data checking result; and / or, sending an abnormal result in the data checking result to a specified terminal through a specified communication mode.

3. The method of claim 1, wherein, The establishing of the checking flow application of the checking task according to the configuration information comprises: providing a resource scheduling interface to establish the checking flow application of the checking task in the cloud service platform in response to an interface calling instruction corresponding to the checking task.

4. The method of claim 1, wherein, The configuration information comprises at least one target data source and a checking rule; The calling of the pulling component to pull first data from at least one target data source according to the configuration information comprises: establishing a communication connection with the target data source; pulling the first data from the target data source based on the communication connection; The checking of the first data and the second data comprises: operating the first data and the second data according to the checking rule to confirm whether the first data and / or the second data has a difference.

5. The method of claim 4, wherein, The processing component is multiple, and each processing component corresponds to a business system, and multiple processing components are connected according to a dependency relationship of corresponding businesses; The calling of the processing component to pull second data from a corresponding business system and to check the first data and the second data according to the configuration information further comprises: passing, by a preceding processing component, at least one of the first data, second data of the corresponding business and a checking result of the preceding processing component to a subsequent processing component to enable the subsequent processing component to perform data checking.

6. A data collation method characterized by comprising: The method is implemented through a data checking system, and the checking system comprises a checking management platform and a cloud service platform connected in communication; The method is applied to the checking management platform, and the method comprises: providing a configuration interface, the configuration interface comprising a checking task configuration control, the checking task configuration control comprising a data source configuration sub-control and a checking rule configuration sub-control; obtaining configuration information of a checking task through the configuration interface; generating a collation task according to the configuration information, and publishing the collation task to the cloud service platform; The collation management platform receives and resolves the collation task to obtain configuration information of the collation task, establishes a collation flow application of the collation task according to the configuration information, the collation flow application at least including a pulling component, at least one processing component, and a converging component, calls the pulling component to pull first data from at least one target data source according to the configuration information, calls the processing component to pull second data from a corresponding business system, and collates the first data and the second data according to the configuration information, and calls the converging component to converge a result obtained through collation to obtain a data collation result.

7. The method of claim 6, wherein, The publishing of the collation task to the cloud service platform includes: calling a resource scheduling interface provided by the cloud service platform to send an interface call instruction of the collation task to the cloud service platform to implement the publishing of the collation task.

8. A data collating apparatus, characterized by comprising: The data collation apparatus can implement the method of any one of claims 1-5 or any one of claims 6-7.

9. A data collation system characterized by, The data collation system includes a collation management platform and a cloud service platform in communication connection, the collation management platform is deployed with the data collation apparatus of claim 8, and can implement the method of claim 6 or 7; and the cloud service platform is deployed with the data collation apparatus of claim 8, and can implement the method of any one of claims 1-5.

10. An electronic device comprising: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of any one of claims 1-5 or any one of claims 6-7.

Images