Data processing method and apparatus, storage medium, and electronic device

Abstract

The present disclosure provides a data processing method and device, electronic equipment and computer readable storage medium, and relates to the technical field of data processing. The method comprises: obtaining business data, and extracting seed data from the business data, wherein the seed data is used to generate test data; configuring a data rule according to an exchange interface definition of a target data warehouse, and generating test data based on the seed data according to the data rule, and sending the test data to the target data warehouse, so as to provide the test data to a data application party of a downstream system through the target data warehouse, and realize the system development and test requirements of the data application party. Based on the seed data, a large amount of test data can be generated in batches and automatically through the exchange interface definition and the data rule, so that the test requirements can be better met, and the generation efficiency of the test data can be improved and the cost can be reduced.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and more specifically, to a data processing method, a data processing apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In existing technologies, when developing a new requirement, the upstream system must first perform its development. After development is complete, it provides test data according to the exchange interface definition. Only after the data warehouse and downstream systems receive the test data can they proceed with their own development and testing. The sequential development and testing of the upstream system and data warehouse result in low overall development efficiency.

[0003] Therefore, in order to solve the above problems, the present disclosure provides a data processing method, a data processing apparatus, an electronic device, and a computer-readable storage medium. Summary of the Invention

[0004] The purpose of this disclosure is to provide a data processing method, a data processing device, an electronic device, and a computer-readable storage medium, thereby solving to some extent the problems in related technologies, such as the serial development and testing of upstream systems and data warehouses, which leads to low overall development efficiency.

[0005] According to a first aspect of this disclosure, a data processing method is provided, comprising: acquiring business data and extracting seed data from the business data, the seed data being used to generate test data; defining configuration data rules according to an exchange interface of a target data warehouse, generating test data based on the seed data according to the data rules, and sending the test data to the target data warehouse, so as to provide the test data to the data application party of a downstream system through the target data warehouse, thereby fulfilling the system development and testing requirements of the data application party.

[0006] In one exemplary embodiment of this disclosure, generating test data based on the seed data according to the data rules includes: verifying the seed data according to the data rules; if the verification fails, generating the test data that conforms to the data rules, wherein the data rules include interface rules and field generation rules.

[0007] In one exemplary embodiment of this disclosure, the step of generating test data that conforms to the data rules if the verification fails includes: if there is a failing field in the seed data, generating test data for the failing field according to the corresponding requirements in the data rules, and replacing it in the corresponding position of the seed data.

[0008] In one exemplary embodiment of this disclosure, the step of generating test data that conforms to the data rules if the verification fails includes: if the seed data has missing fields, supplementing the missing fields according to the corresponding requirements in the data rules, generating test data for the missing fields, and supplementing the seed data.

[0009] In one exemplary embodiment of this disclosure, after replacing the non-passing field in the seed data or supplementing the missing field into the seed data, the method further includes: storing the seed data into a seed database.

[0010] In one exemplary embodiment of this disclosure, sending the test data to the target data warehouse includes: exporting the test data into a data exchange interface file according to the exchange interface definition, and sending it to the target data warehouse.

[0011] According to a second aspect of this disclosure, a data processing apparatus is provided, comprising: a data extraction module for acquiring business data and extracting seed data from the business data, the seed data being used to generate test data; and a data processing module for defining configuration data rules based on the exchange interface of a target data warehouse, generating test data based on the seed data according to the data rules, and sending the test data to the target data warehouse, so as to provide the test data to the data application party of the downstream system through the target data warehouse, thereby fulfilling the system development and testing requirements of the data application party.

[0012] In one exemplary embodiment of this disclosure, the data processing module is specifically used to: if there is a failing field in the seed data, generate the test data for the failing field according to the corresponding requirements in the data rules, and replace it in the corresponding position of the seed data.

[0013] In one exemplary embodiment of this disclosure, the data processing module is further configured to: if the seed data has missing fields, supplement the missing fields according to the corresponding requirements in the data rules, and generate the test data of the missing fields to supplement the seed data.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method described in any of the preceding methods by executing the executable instructions.

[0015] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the preceding claims.

[0016] The exemplary embodiments disclosed herein may have some or all of the following beneficial effects:

[0017] In the data processing method provided in the exemplary embodiments of this disclosure, business data is acquired, and seed data is extracted from the business data. The seed data is used to generate test data. Data rules are configured according to the exchange interface definition of the target data warehouse, and test data is generated based on the seed data according to the data rules. The test data is then sent to the target data warehouse to provide test data to data application parties in downstream systems, thereby fulfilling the system development and testing needs of the data application parties. This embodiment of the disclosure generates test data based on seed data, through exchange interface definition and data rules. On the one hand, this improves the fidelity between the test data and actual business data, better meeting testing requirements. On the other hand, it enables the batch and automated generation of large amounts of test data, improving test data generation efficiency and reducing implementation costs.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 The schematic diagram illustrates a flow chart of a data processing method according to an embodiment of the present disclosure;

[0021] Figure 2 A block diagram of a data processing apparatus according to an embodiment of the present disclosure is shown schematically;

[0022] Figure 3 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0023] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0024] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0026] In daily R&D, when a new requirement needs to be developed, the upstream system must first complete its development. After completion, it provides test data via an exchange interface. Only after the data warehouse and downstream systems receive the test data can they proceed with their own development and testing. This sequential development and testing of the upstream system and data warehouse leads to low overall development efficiency. Furthermore, in some cases, the upstream system's data supply, besides being ready in its own system, also relies on migrating data from the old upstream system to the new one. This data migration work can consume half or more of the entire project cycle. Consequently, the downstream data warehouse and downstream systems cannot conduct effective development, unit testing, and integration testing in the first half of the project cycle. All work is concentrated in the later stages of the project, running in parallel with user testing. This results in shorter development and testing times, decreased quality, increased costs, and even the risk of affecting the project's on-time launch, leading to decreased customer satisfaction.

[0027] To address the aforementioned problems, this exemplary embodiment provides a data processing method, a data processing apparatus, an electronic device, and a computer-readable storage medium. The technical solutions of the embodiments of this disclosure are described in detail below:

[0028] This example implementation first provides a data processing method. (See reference...) Figure 1 As shown, this data processing method specifically includes the following steps:

[0029] Step S110: Obtain business data and extract seed data from the business data. The seed data is used to generate test data.

[0030] Step S120: Define and configure data rules according to the exchange interface of the target data warehouse, generate test data based on seed data according to the data rules, and send the test data to the target data warehouse so as to provide test data to the data application party of the downstream system through the target data warehouse, thereby realizing the system development and testing needs of the data application party.

[0031] In the data processing method provided in this exemplary embodiment, test data is generated based on seed data through exchange interface definitions and data rules. On the one hand, this improves the fidelity between the test data and actual business data, better meeting testing requirements. On the other hand, it enables the batch and automated generation of large amounts of test data, improving test data generation efficiency and reducing implementation costs.

[0032] The above steps will now be described in more detail in another embodiment.

[0033] In step S110, business data is acquired, and seed data is extracted from the business data. The seed data is used to generate test data.

[0034] The data processing method provided in this disclosure is used to generate test data for a data line. The aforementioned data line refers to the entire process of data flow from the upstream system to the data warehouse and then to the downstream system. Specifically, in a typical data line, data is collected or generated by the upstream system, transmitted to the data warehouse in file format according to the exchange interface format, and then processed by the data warehouse for use by the downstream system.

[0035] The upstream systems mentioned above refer to transactional systems that provide data to the data warehouse, information systems that record enterprise production and operations, and are the data source for the data warehouse, providing data to the data warehouse in the form of messages or files. The data warehouse refers to a subject-oriented, integrated, time-varying, but relatively stable collection of data used to support management decision-making processes. The database warehouse stores all business data within the enterprise, which, after processing, supports enterprise management decisions. The downstream systems mentioned above refer to management and analysis systems that obtain data from the data warehouse. These systems acquire raw or processed data from the data warehouse, process it, and provide decision support to enterprise managers in the form of queries, reports, and graphs.

[0036] To address the low development efficiency caused by sequential development and testing of upstream systems and data warehouses, this disclosure provides a data processing tool that uses the data processing method proposed in this disclosure to generate test data instead of the upstream system. The aforementioned business data refers to the data generated to implement the corresponding business scenario; for example, this business data could be data generated from deposit transactions.

[0037] For example, the aforementioned business data can be obtained from the aforementioned upstream system. Specifically, the extraction of seed data from the business data can be achieved as follows: collecting data supplemented by testers in the upstream system through testing and using it as seed data. Furthermore, after generating test data using the method provided in this disclosure embodiment, the generated test data can also be used as seed data and accumulated and stored in a seed database. Then, this disclosure embodiment can extract seed data from the seed database according to the configuration requirements of the current project.

[0038] In step S120, configuration data rules are defined according to the exchange interface of the target data warehouse, and test data is generated based on seed data according to the data rules. The test data is sent to the target data warehouse so as to provide test data to the data application party of the downstream system through the target data warehouse, thereby realizing the system development and testing needs of the data application party.

[0039] In this embodiment of the disclosure, the target data warehouse is a data warehouse corresponding to a specific business. For example, if the current development project is related to banking business, the target data warehouse can be a data warehouse corresponding to deposit, withdrawal, and transaction-related businesses.

[0040] The aforementioned exchange interface is a data exchange interface agreed upon by the upstream system and the data warehouse. The definition of the aforementioned exchange interface mainly includes the definition of the interface and the field definition of each interface. Among them, the interface definition includes the interface name, the upstream system number from which the interface originates, the transmission frequency, etc., and the field definition of each interface includes the field name, field type, field length, and the value range of code-type fields, etc.

[0041] The aforementioned test data is required during the project requirement development and testing process. The aforementioned data rules are the rules for generating test data in this embodiment of the disclosure. These data rules must be configured according to the aforementioned exchange interface definition; that is, the test data generated according to these data rules should meet the requirements of the aforementioned exchange interface definition. For example, taking deposit business as an example, assuming that in the exchange interface definition, the user password field has a length of 6 and the field value is required to be a combination of numbers or letters, then when generating test data for the user password field, the data rules can generate multiple strings of length 6 composed of numbers or letters as test data. It should be noted that the above scenario is only an illustrative example, and the scope of protection of this embodiment of the disclosure is not limited thereto. Depending on the actual situation, the aforementioned data rules may also be different. For example, the data rules may also stipulate that the value of a certain integer field does not exceed two digits, etc.

[0042] In this embodiment of the disclosure, exemplarily, the process of generating test data based on seed data according to data rules can be implemented as follows: The seed data is checked according to the data rules; if the check fails, test data conforming to the data rules is generated. The data rules include interface rules and field generation rules. For example, the completeness and quality of the seed data can be checked according to the data rules, and the failed fields in the seed data can be identified. It should be noted that the above scenario is merely an illustrative example, and the scope of protection of this embodiment of the disclosure is not limited thereto.

[0043] Specifically, when a non-compliant field is found in the seed data, test data is generated for the non-compliant field according to the corresponding requirements in the data rules, and this test data is then inserted into the corresponding position in the seed data to optimize and supplement the non-compliant field in the seed data, forming test data that meets the data rules. For example, for a field in the seed data, the above data rule defines it as the first three characters being letters and the last three characters being numbers, while the actual data of this field in the seed data is the first two characters being letters and the last four characters being numbers. In this case, test data that meets the requirements is randomly generated according to the data rules, and the generated test data replaces the original data of the corresponding field in the seed data. It should be noted that the above scenario is only an exemplary illustration, and the scope of protection of this disclosure embodiment is not limited thereto.

[0044] If missing fields are found in the seed data, they are added according to the corresponding requirements in the aforementioned data rules, and test data for the missing fields is generated and added to the seed data. That is, for data types missing in the seed data, incremental data records are added. For example, if the seed data for transaction business is found to lack the data type "consumer," this data type is added, and test data for this data type, such as user ID and age fields, is generated according to the aforementioned data rules. It should be noted that the above scenario is merely an illustrative example, and the scope of protection of this disclosure is not limited thereto.

[0045] In one specific embodiment, after checking the completeness and quality of the seed data according to the above data rules, optimizing and supplementing the non-passing fields in the seed data, and supplementing the missing data types in the seed data with incremental data records, the optimized seed data and the supplemented incremental data can be summarized to form test data and saved into the seed database.

[0046] In this embodiment of the disclosure, the above-mentioned sending of test data to the target data warehouse can be achieved as follows: according to the exchange interface definition, the test data is exported into a data exchange interface file and sent to the target data warehouse.

[0047] Furthermore, in this embodiment of the disclosure, test data can be provided from the target data warehouse to the data application party based on the request from the data application party of the downstream system, so as to meet the system development and testing needs of the data application party. That is, the test data in the data warehouse is provided to the downstream system for development and testing, so as to solve the problems in the prior art that the amount of test data is small and it is difficult to cover various business scenarios, which makes it difficult to meet the data line system development and testing needs in the early stage of the project.

[0048] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0049] Furthermore, in this exemplary embodiment, a data processing apparatus is also provided, with reference to... Figure 2 As shown, the data processing device 200 may include a data extraction module 210 and a data processing module 220. Wherein:

[0050] The data extraction module 210 can be used to acquire business data and extract seed data from the aforementioned business data. The seed data is used to generate test data.

[0051] The data processing module 220 can be used to define and configure data rules based on the exchange interface of the target data warehouse, generate test data based on seed data according to the above data rules, and send the test data to the target data warehouse so as to provide test data to the data application party of the downstream system through the target data warehouse, thereby realizing the system development and testing needs of the data application party.

[0052] In one exemplary embodiment of this disclosure, the data processing module is specifically used to: check the seed data according to data rules; if the check fails, generate test data that conforms to the data rules, wherein the data rules include interface rules and field generation rules.

[0053] Depending on the verification results, the aforementioned data processing module can be used in the following ways: If there are non-compliant fields in the seed data, test data will be generated for the non-compliant fields according to the corresponding requirements in the data rules, and then replaced in the corresponding positions in the seed data. For example, for a certain field in the seed data, the aforementioned data rule defines it as the first three characters being letters and the last three characters being numbers, while the actual data for that field in the seed data is the first two characters being letters and the last four characters being numbers. In this case, test data that meets the requirements will be randomly generated according to the data rules, and the generated test data will replace the original data of the corresponding field in the seed data.

[0054] Alternatively, if the seed data contains missing fields, the missing fields are supplemented according to the corresponding requirements in the data rules, and test data for the missing fields is generated and added to the seed data. For example, if the seed data for a transaction business is found to be missing the "consumer" data type, this data type is supplemented, and test data for this data type, such as user ID and age fields, is generated according to the aforementioned data rules. It should be noted that the above scenario is only an illustrative example, and the scope of protection of this disclosure embodiment is not limited thereto.

[0055] In one exemplary embodiment of this disclosure, the data processing module is further configured to: store seed data into a seed database. Specifically, after checking the completeness and quality of the seed data according to the above data rules, optimizing and supplementing non-compliant fields in the seed data, and supplementing incremental data records for missing data types in the seed data, the optimized seed data and the supplemented incremental data can be summarized to form test data and saved into the seed database.

[0056] In one exemplary embodiment of this disclosure, the data processing module is further configured to: export the test data into a data exchange interface file according to the exchange interface definition, and send it to the target data warehouse.

[0057] In one exemplary embodiment of this disclosure, the data processing apparatus is further configured to: provide test data from the target data warehouse to the data application party based on the data application party's request for the target data warehouse, so as to meet the system development and testing needs of the data application party.

[0058] The specific implementation details of the aforementioned data processing device have been explained in detail in the corresponding section of the aforementioned data processing method, and therefore will not be repeated here.

[0059] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0060] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 3 It shows a schematic diagram of a structure suitable for implementing the electronic device 300 in the embodiments of this disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0061] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303 to implement the data processing method as described in the embodiments of this disclosure. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing device 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0062] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0063] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the data processing method as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 309, or installed from a storage device 308, or installed from a ROM 302. When the computer program is executed by the processing device 301, it performs the functions defined in the methods of embodiments of this disclosure.

[0064] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0065] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0066] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0067] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0068] Acquire business data and extract seed data from it. The seed data is used to generate test data.

[0069] Based on the exchange interface definition of the target data warehouse, configure data rules, generate test data based on seed data according to the data rules, and send the test data to the target data warehouse so as to provide test data to the data application parties of downstream systems through the target data warehouse, thereby realizing the system development and testing needs of the data application parties.

[0070] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0071] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0073] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0074] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0075] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0076] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0077] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0078] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A data processing method, characterized in that, include: Acquire business data and extract seed data from the business data, the seed data being used to generate test data; Configure data rules according to the exchange interface definition of the target data warehouse, and verify the seed data according to the data rules. The data rules include interface rules and field generation rules. The exchange interface definition includes the definition of the interface and the field definition of each interface. The interface definition includes the interface name, the upstream system number of the interface source, and the transmission frequency. The field definition of each interface includes the field name, the field type, the field length, and the value range of the code field. If there are any failing fields in the seed data, then test data is generated for the failing fields according to the corresponding requirements in the data rules, and replaced in the corresponding position of the seed data; If the seed data has missing fields, the missing fields are supplemented according to the corresponding requirements in the data rules, and the test data for the missing fields is generated and added to the seed data. The test data is sent to the target data warehouse so that it can be provided to the data application parties of downstream systems, thereby fulfilling the system development and testing needs of the data application parties.

2. The data processing method according to claim 1, characterized in that, After replacing the non-passing field in the seed data or supplementing the missing field into the seed data, the method further includes: The seed data is stored in the seed database.

3. The data processing method according to claim 1, characterized in that, Sending the test data to the target data warehouse includes: According to the defined exchange interface, the test data is exported as a data exchange interface file and sent to the target data warehouse.

4. A data processing apparatus, characterized in that, include: The data extraction module is used to acquire business data and extract seed data from the business data, the seed data being used to generate test data; The data processing module is used to configure data rules based on the exchange interface definition of the target data warehouse, and to verify the seed data according to the data rules. The data rules include interface rules and field generation rules. The exchange interface definition includes the definition of the interface and the field definition for each interface. The interface definition includes the interface name, the upstream system number from which the interface originates, and the transmission frequency. The field definition for each interface includes the field name, field type, field length, and the value range for code-type fields. If there are any non-compliant fields in the seed data, test data is generated for the non-compliant fields according to the corresponding requirements in the data rules, and this data is then inserted into the corresponding position in the seed data. If the seed data has missing fields, the missing fields are supplemented according to the corresponding requirements in the data rules, and test data for the missing fields is generated and added to the seed data; the test data is sent to the target data warehouse so as to provide the test data to the data application party of the downstream system through the target data warehouse, thereby realizing the system development and testing needs of the data application party.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-3.

6. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1-3 by executing the executable instructions.

Images