Zipper table data processing method and device, electronic equipment and readable storage medium

Abstract

The invention provides a data processing method and device of a zipper table, electronic equipment and a readable storage medium. The method comprises the steps that an original time interval of a zipper table to be processed and service data of the original time interval are acquired, and the original time interval comprises a time interval between a first starting time node and a first ending time node; determining a second time interval based on the first starting time node and the first ending time node; and querying target business data in the second time interval from the to-be-processed zipper table, and integrating the target business data to obtain a combined zipper table of the to-be-processed zipper table. According to the method and the device, a plurality of zipper tables can be combined, and efficient generation of the zipper tables is realized.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a data processing method, device, electronic device and readable storage medium for a zipper table. Background Art

[0002] In some business scenarios, zipper tables are often used as tables commonly used in data warehouses to record data change information. Compared with full data tables, zipper tables are widely used because they are easy to browse equivalent information and have small storage space.

[0003] In the related art, considering that different users may have different special businesses, in order to store these special business data, it is necessary to create multiple zipper tables with the same primary key granularity but different business indicators to record the data changes of the corresponding special businesses. For massive business data, the number of date sections of the corresponding stored zipper table will be very large. When making a zipper table, the zipper table corresponding to each special business may perform a new chain operation on the batch date, and then compare the zipper data of the new chain with the business data of each date section to determine the details of the changes in the business data. When the number of special businesses and the number of date sections are very large, the production of zipper tables requires a lot of computing costs and time costs, and the production efficiency is low. Summary of the invention

[0004] The embodiments of the present application provide a zipper table data processing method, device, electronic device and readable storage medium, which can merge multiple zipper tables to achieve efficient generation of zipper tables.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a method for processing zipper table data, the method comprising:

[0007] Acquire a first time interval of the zipper table to be processed and business data of the first time interval, wherein the first time interval includes a time interval between a first start time node and a first end time node;

[0008] Determining a second time interval based on the first start time node and the first end time node;

[0009] The target business data in the second time interval is queried from the zipper table to be processed, and the target business data is integrated to obtain a merged zipper table of the zipper table to be processed.

[0010] The present application embodiment provides a zipper table data processing device, including:

[0011] An acquisition module, used to acquire a first time interval of the zipper table to be processed and business data of the first time interval, wherein the first time interval includes a time interval between a first start time node and a first end time node;

[0012] A determination module, configured to determine a second time interval based on the first start time node and the first end time node;

[0013] An integration module is used to query the target business data in the second time interval from the zipper table to be processed, and integrate the target business data to obtain a merged zipper table of the zipper table to be processed.

[0014] An embodiment of the present application provides an electronic device, including:

[0015] Memory for storing computer executable instructions or computer programs;

[0016] The processor is used to implement the data processing method of the zipper table provided in the embodiment of the present application when executing the computer executable instructions or computer programs stored in the memory.

[0017] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions or a computer program for implementing the zipper table data processing method provided in the embodiment of the present application when executed by a processor.

[0018] An embodiment of the present application provides a computer program product, including a computer program or a computer executable instruction. When the computer program or the computer executable instruction is executed by a processor, the data processing method of the zipper table provided in the embodiment of the present application is implemented.

[0019] The embodiments of the present application have the following beneficial effects:

[0020] Through the embodiment of the present application, the first time interval in the zipper table to be processed and the business data corresponding to the first time interval are first obtained, and the first start time node and the first end time node of the first time interval are integrated as the second time interval of the merged zipper table. Finally, the target business data of the second time interval of the merged zipper table is queried from the zipper table to be processed, and the target business data is integrated to obtain the merged zipper table of the zipper table to be processed. The merging of the zipper tables is achieved by only integrating the first time interval of the zipper table to be processed to obtain the second time interval of the merged zipper table, and then querying the target business data according to the second time interval. In addition, there is no need to consider the number of business data of each zipper table to be processed during the merging process, nor is there a need to compare the time interval of each date section to determine the change of business data when the zipper table to be processed performs a new chain operation. In this way, when the number of zipper tables to be processed and the number of date sections is large, the time cost and calculation cost of making the zipper table can be saved, and the efficiency of making the zipper table is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A is a schematic diagram of the architecture of a zipper table data processing system 100 provided in an embodiment of the present application;

[0022] Figure 1B It is a processing schematic diagram of the data processing method of the zipper table provided in the embodiment of the present application;

[0023] Figure 2 is a schematic diagram of the structure of the server 200 provided in an embodiment of the present application;

[0024] Figure 3A It is a first flow chart of the data processing method of the zipper table provided in the embodiment of the present application;

[0025] Figure 3B It is a second flow chart of the data processing method of the zipper table provided in the embodiment of the present application;

[0026] Figure 3C is a third flow chart of the data processing method of the zipper table provided in the embodiment of the present application;

[0027] Figure 3D is a fourth flow chart of the zipper table data processing method provided in an embodiment of the present application;

[0028] Figure 3E is a fifth flow chart of the zipper table data processing method provided in an embodiment of the present application;

[0029] Figure 3F is a sixth flow chart of the zipper table data processing method provided in an embodiment of the present application;

[0030] Figure 4 is a diagram of the manufacturing process of the zipper watch provided in the embodiment of the present application;

[0031] Figure 5 This is a zipper representation of different service indicators of the same user provided in the embodiment of the present application;

[0032] Figure 6 It is a zipper representation of multiple data fields provided in the embodiment of the present application;

[0033] Figure 7 is a schematic diagram of a zipper table merging method provided in an embodiment of the present application;

[0034] Figure 8 It is a schematic diagram of using coordinate axes to display the time interval of zipper data provided by an embodiment of the present application;

[0035] Fig. 9 It is a schematic diagram of the node projection result provided in an embodiment of the present application;

[0036] Fig.10 It is a schematic diagram of determining interval results from node results provided by an embodiment of the present application;

[0037] Fig.11 is a time interval comparison diagram of a zipper table provided in an embodiment of the present application;

[0038] Fig.12 is a schematic diagram of interval field information of a target zipper table provided in an embodiment of the present application;

[0039] Fig.13 It is a schematic diagram of the zipper table time interval comparison association provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. Other embodiments obtained by ordinary technicians in this field without making creative work are all within the scope of protection of this application.

[0041] In the following description, reference is made to “some embodiments” which describe a subset of possible embodiments, but it is understood that “some embodiments” may be the same subset or different subsets of possible embodiments and may be combined with each other without conflict.

[0042] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0043] Unless otherwise defined, the technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0044] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0045] 1) Zipper table, a table that records the change information of things from the beginning to the current state. Compared with the full table, the zipper table records the life cycle of data through the time interval of the zipper data (that is, the time range between the zipper start time and the zipper end time), reduces duplicate records, and can save storage space to the maximum extent.

[0046] 2) Opening a new chain means adding zipper data to the zipper table. It is a common data operation for zipper tables. When the business data corresponding to the zipper data in the zipper table changes, the time interval for recording the business data will change. New zipper data needs to be added to record the details of the business data after the change and the corresponding current time. Alternatively, when a new special business needs to correspond to a new zipper table, a zipper data will be added to the new zipper table to record the details of the business data after the change and the corresponding current time.

[0047] 3) Section refers to the time interval from the earliest time node recorded in the zipper table to the current moment when the data warehouse extracts business data. All business data recorded in the zipper table within this time interval is called a section, and this time interval can also be called a date section.

[0048] The embodiments of the present application provide a zipper table data processing method, device, electronic device, computer-readable storage medium and computer program product, which can merge multiple zipper tables to achieve efficient generation of zipper tables.

[0049] The following describes exemplary applications of the electronic device provided in the embodiments of the present application. The electronic device provided in the embodiments of the present application can be implemented as various types of user terminals such as laptop computers, tablet computers, desktop computers, set-top boxes, mobile devices (for example, mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smart phones, smart speakers, smart watches, smart televisions, and vehicle-mounted terminals, and can also be implemented as servers.

[0050] See also Figure 1A , Figure 1A It is a schematic diagram of the architecture of the zipper table data processing system 100 provided in an embodiment of the present application. The terminal 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0051] The terminal 400 runs data processing programs (Application, APP) for various business scenarios, such as instant messaging APP, reading APP, video APP, game APP, or other software programs. When a user (such as a business person or other artificial intelligence program) uploads the business data of the zipper table to be processed on the terminal 400 and initiates a data processing request, and then sends it to the server 200 through the network 300, the server 200 responds to the data processing request initiated by the user on the terminal 400, and obtains the first time interval of the zipper table to be processed and the business data of the first time interval, and determines the second time interval based on the first start time node and the first end time node of the first time interval, and then queries the target business data in the second time interval from the zipper table to be processed, and integrates the target business data to obtain the merged zipper table of the zipper table to be processed. Finally, the business data of the merged zipper table is returned to the terminal 400 through the network 300 as the data processing result, and the user can obtain the data processing result, that is, the business data of the merged zipper table, in the terminal 400.

[0052] In some embodiments, Figure 1A The server 200 shown can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiments of the present application.

[0053] The embodiments of the present application can be implemented with the help of artificial intelligence (AI) technology, which is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0054] Taking the server provided in the embodiment of the present application as an example, for example, a server cluster can be deployed in the cloud, thereby opening up artificial intelligence cloud services (AI as a Service, AIaaS) to users or developers. The AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to opening an AI theme mall, where users or developers can access one or more artificial intelligence services provided by the AIaaS platform through an application programming interface.

[0055] For example, the server in the cloud encapsulates the program of the data processing method of the zipper table provided in the embodiment of the present application. The user calls the data processing service of the zipper table in the cloud service through the terminal (the terminal runs an APP, such as an instant messaging APP, a reading APP, etc.), so that the server deployed in the cloud calls the program of the encapsulated data processing method of the zipper table. After the terminal uploads the business data of the zipper table to be processed and initiates a data processing request, the server responds to the data processing request initiated by the user at the terminal, and obtains the first time interval of the zipper table to be processed and the business data of the first time interval, and determines the second time interval based on the first start time node and the first end time node of the first time interval, and then queries the target business data in the second time interval from the zipper table to be processed, and integrates the target business data to obtain the merged zipper table of the zipper table to be processed. Finally, the business data of the merged zipper table is returned to the terminal, and the user can obtain the business data of the merged zipper table in the terminal.

[0056] Next, an exemplary application when the device is implemented as a terminal will be described, see Figure 1B , Figure 1BThis is a processing diagram of the data processing method of the zipper table provided in the embodiment of the present application. The user uploads the business data of the zipper table to be processed on the client and initiates a data processing request. The client responds to the data processing request initiated by the user on the terminal, first directly obtains the first time interval of the zipper table to be processed and the business data of the first time interval, and determines the second time interval based on the first start time node and the first end time node of the first time interval. Next, the target business data in the second time interval is queried from the zipper table to be processed, and the target business data is integrated to obtain a merged zipper table of the zipper table to be processed. Finally, the business data of the merged zipper table is directly displayed on the client for the user to browse.

[0057] See also Figure 2 , Figure 2 is a schematic diagram of the structure of the server 200 provided in an embodiment of the present application, Figure 2 The terminal 400 shown includes: at least one processor 410, a memory 450, and at least one network interface 420. The various components in the server 200 are coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 2 Various buses are labeled as bus system 440 .

[0058] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0059] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0060] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0061] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0062] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0063] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.;

[0064] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 The data processing device 453 of the zipper table stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: an acquisition module 4531, a determination module 4532 and an integration module 4533. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0065] In some embodiments, the terminal or server can implement the data processing method of the zipper table provided in the embodiment of the present application by running various computer executable instructions or computer programs. For example, the computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in the operating system; it can be a native application (Application, APP), that is, a program that needs to be installed in the operating system to run, such as a live broadcast APP or an instant messaging APP; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0066] The data processing method of the zipper table provided in the embodiment of the present application will be described in combination with the exemplary application and implementation of the server 200 provided in the embodiment of the present application.

[0067] See also Figure 3A , Figure 3A is a flow chart of a method for processing zipper table data provided by an embodiment of the present application, comprising Figure 1A The server 200 shown is the execution entity, which will be combined with Figure 3A The steps shown are explained.

[0068] In step 101, a first time interval of a zipper table to be processed and business data of the first time interval are obtained.

[0069] In some embodiments, the first time interval of the zipper table to be processed and the business data of the first time interval are first obtained. Since the time interval and the business data corresponding to the time interval are the main components of the zipper table, merging the zipper table requires obtaining the time interval and the business data for processing. For massive business data, business data is generally classified and stored according to business type during storage operations, and each category of business data is stored using a zipper table to record data changes for special businesses. In the zipper table, in response to changes in business data of a business scenario, the corresponding zipper table will perform a new chain operation to record changes in business data and the current time of the change, thereby constructing a time interval for business data changes. In this way, each business data corresponding to the zipper table to be processed will have a time interval, namely the first time interval.

[0070] When the business data of the zipper table to be processed changes or new business data is added, the corresponding first time interval will be automatically generated to record the effective time of the business data, wherein the first time interval includes the time interval between the first start time node and the first end time node, the first start time node is used to record the corresponding effective start time of the business data, and the first end time node is used to record the corresponding effective end time of the business data. When merging N zipper tables to be processed, it is necessary to construct a new zipper table as a merged zipper table based on the first time intervals of the N zipper tables to be processed and the business data corresponding to the first time intervals. Therefore, when merging the zipper tables, it is necessary to obtain all the first time intervals of the N zipper tables to be processed and the business data corresponding to each first interval, wherein N is an integer greater than 1, indicating that there are multiple zipper tables to be processed to be merged.

[0071] In step 102, a second time interval is determined based on the first start time node and the first end time node.

[0072] After obtaining the first time interval of the zipper table to be processed and the business data corresponding to each first time interval, the second time interval is determined based on the first start time node and the first end time node, wherein the first time interval includes M second time intervals, and M is an integer greater than 1. Considering that the first time interval may come from multiple zipper tables to be processed, there will be repeated time intervals. The time intervals of a single zipper table are adjacent, and there is no time interval or interval duplication. Because the zipper table will only open a new chain and update the time interval when the business data changes and new business data is added, otherwise it will continue to record the effective time of the business data. Therefore, it is necessary to merge and deduplicate the time nodes in the first time interval in the zipper table to be processed to ensure the uniqueness of each time node. The process of determining the second time interval is described in detail below.

[0073] In some embodiments, see Figure 3B , Figure 3A The step 102 shown in the figure can be implemented by following the steps 1021 to 1022, which are described in detail below.

[0074] In step 1021, P time nodes are determined based on the first start time node and the first end time node.

[0075] First, based on the first starting time node and the first ending time node, P time nodes are determined. Specifically, the first starting time node and the first ending time node of all the first time intervals in the N zipper lists to be processed are obtained to form a first time node set. After obtaining the first time node set, the first time node set can be regarded as a one-dimensional array, and then the one-dimensional array can be deduplicated using the array deduplication method. Or the time node set can be regarded as a first time node sequence, and then the first time node sequence can be deduplicated using the sequence deduplication method to obtain the corresponding second time node set, which includes P time nodes. Among them, the array deduplication method can be to use the group deduplication function or the distinct deduplication function of the database, and directly deduplicate the first time node set to obtain P time nodes. Therefore, according to the first starting time node and the first ending time node, P time nodes can be determined, and P is an integer greater than 1.

[0076] In some embodiments, for the first time node set, an exhaustive method can also be used to obtain P time nodes. Specifically, starting from the first time node of the first time node set, the first time node is taken as a time node sequence, wherein each time node in the time node sequence has a time sequence. Next, each subsequent time node in the first time node set is traversed one by one, and each time node is added to the time node sequence one by one. When the traversed current time node does not exist in the time node sequence, the current time node is added to the corresponding position of the time node sequence. When the traversed current time node already exists in the time node sequence, the current time node is removed. Thus, when all time nodes are traversed, the final time node sequence is obtained, and this time node sequence includes P time nodes.

[0077] In step 1022, a second time interval is generated based on each of the P time nodes, so as to obtain M second time intervals.

[0078] Continuing with the above embodiment, after deduplication processing is performed on the first time node set to obtain P time nodes, a second time interval is generated based on each time node in the P time nodes, and M second time intervals are obtained, where the value of P is the same as M. For these P time nodes, the P time nodes can be sorted according to the time sequence of each time node, that is, the earlier time nodes are sorted in front of the time node sequence, and the later time nodes are sorted after the time node sequence. Next, a second time interval is generated based on the P time nodes. Specifically, for each time node, the time node is used as the second starting time node. When the time node is not the last time node among the P time nodes, the adjacent time nodes adjacent to the time node and located after the time node in the P time nodes are obtained, and the adjacent time nodes are used as the second ending time nodes.

[0079] Considering that each of the P time nodes belongs to the time node of the first time interval in the zipper list to be processed, the time range between every two adjacent time nodes in the sorted P time nodes can form a new time interval, and each of these new time intervals must be a subset of one of the first time intervals in the zipper list to be processed. That is, the new time interval composed of every two adjacent time nodes can find a target first time interval corresponding to it from the first time interval in the zipper list to be processed. In the embodiment of the present application, for P time nodes, every two adjacent time nodes in the P time nodes can be combined into a new time interval, and this new time interval can be used as the second time interval of the merged zipper list. That is, each time node can generate a second time interval respectively, so that P time nodes can generate M second time intervals, and the value of P is the same as that of M.

[0080] Specifically, the time node is first used as the second starting time node, as the second starting time node of the second time interval. After determining the second starting time node of the second time node, the second ending time node corresponding to each second starting time node needs to be determined next. Considering that the time node with the last (i.e., the end) order among the P time nodes generally appears in the first time interval corresponding to the zipper data of the last row of the zipper table to be processed. When this time node with the last order order is used as the first starting time node, the corresponding second ending time node is generally a preset moment, and this preset moment can be a boundary moment set by the system, that is, the default value set by the system, such as December 31, 9999. There is only one preset moment in the first time interval in each zipper table to be processed. Therefore, when determining the second ending time node, it is necessary to determine whether each time node (i.e., the first starting node time) is the last time node among the P time nodes in the P time nodes. In an embodiment of the present application, when the time node (the second starting time node) is not the last time node among the P time nodes, the adjacent time nodes adjacent to the time node and located after the time node in the P time nodes are obtained, and the adjacent time nodes are determined as the second ending time nodes.

[0081] In some embodiments, when the time node (ie, the second starting time node) is the last time node among the P time nodes, the preset time is used as the second ending time node.

[0082] Specifically, when it is determined that a certain time node is the last time node among P time nodes, it means that this time node is the first starting time node of the first time interval corresponding to the last row of zipper data in the zipper table to be processed, and the first ending time node corresponding to this first starting node in the zipper table to be processed is the system default preset time. When this time node is used as the second starting time node, the preset time can be used as the second ending time node of this time node.

[0083] Following the above embodiment, after determining the second start time node and the corresponding second end time node from the P time nodes, the time interval formed by the second start time node and the second end time node can be used as the second time interval. That is, the time interval formed by each second start time node and the corresponding second end time node is used as a second time interval. Correspondingly, the second start time node at the end of the P nodes also corresponds to a second time interval corresponding to the preset time, thereby obtaining M second time intervals, and the second time interval is used as the time interval of the merged zipper table.

[0084] Through the embodiment of the present application, the first start time node and the first end time node of all the first time intervals in the zipper table to be processed are sorted and deduplicated to obtain P time nodes, and then M second time intervals are determined by the P time nodes as the time intervals of the merged zipper table. Therefore, the second time interval of the merged zipper table is obtained only by integrating the first time intervals of multiple zipper tables to be processed, thereby realizing the time interval merging of the zipper tables. In addition, in the time interval merging process, there is no need to consider the amount of business data of each zipper table to be processed, thereby improving the merging efficiency.

[0085] Continue to see Figure 3A In step 103, the target business data in the second time interval is queried from the zipper table to be processed, and the target business data is integrated to obtain a merged zipper table of the zipper table to be processed.

[0086] In some embodiments, after all first time intervals of the zipper table to be processed are integrated and the second time interval of the zipper data corresponding to the merged zipper table is determined, the business data in the zipper table to be processed needs to be synchronized to the second time interval. Since each second time interval must be a subset of one of the first time intervals of the zipper table to be processed, in the embodiment of the present application, the target business data in the second time interval is queried from the zipper table to be processed, and the target business data is integrated to obtain a merged zipper table of the zipper table to be processed, thereby completing the business data synchronization between the zipper table to be processed and the merged zipper table. Among them, the zipper table to be processed includes N zipper tables to be processed, and N is an integer greater than 1, which is used to represent the merging of multiple zipper tables.

[0087] In some embodiments, see Figure 3C , Figure 3A In the illustrated step 103, "querying the target business data in the second time interval from the to-be-processed zipper table" can also be implemented by the following step 1031A or step 1032A, which is described in detail below.

[0088] In step 1031A, for each of the N zipper lists to be processed, the business data in the M second time intervals is queried from the zipper list to be processed.

[0089] As mentioned above, each second time interval must be a subset of one of the first time intervals of the N zipper tables to be processed. The business data in the zipper table to be processed needs to be synchronized during the merging process, and no business data loss should occur. Therefore, in the embodiment of the present application, based on this subset relationship, for each of the N zipper tables to be processed, the business data of the M second time intervals is queried from the zipper table to be processed.

[0090] In order to save query time and improve the efficiency of data synchronization, and at the same time to cope with emergencies that may occur during data query, the specific method for querying business data in the embodiment of the present application can be determined in an emergency situation based on the number of second time intervals and the number of zipper tables to be processed.

[0091] Specifically, in order to cope with emergencies that may occur during the business data query process, the business system or network environment may suddenly stop operating. When the number of zipper tables to be processed is large (such as greater than the table number threshold) but the number of second time intervals is small (such as less than the interval number threshold), for each of the N zipper tables to be processed, the business data of M integrated time intervals can be queried from the zipper table to be processed, that is, the business data of all integrated time intervals in one zipper table are queried first. In this way, in an emergency, the query tasks of more zipper tables to be processed can be completed as much as possible within the same time.

[0092] For example, there are three zipper tables to be processed, namely, Table 1, Table 2, and Table 3, and two second time intervals, Interval 1 and Interval 2. First, query the business data of interval 1 and interval 2 from the business data corresponding to the zipper data in Table 1 and synchronize them. Next, query the business data of interval 1 and interval 2 from the business data corresponding to the zipper data in Table 2 and synchronize them. Next is Table 3, and so on. However, if an emergency occurs when querying the business data of interval 1 for Table 3 after querying Table 1 and Table 2, that is, the business system stops running, resulting in query interruption. At this time, the query task of the two zipper tables to be processed has been completed. Even if the data cannot be recovered due to the emergency, the data loss can be reduced, so that the data that has been queried has availability.

[0093] In step 1032A, for each of the M second time intervals, the business data in the second time interval is queried from the N to-be-processed zipper tables.

[0094] Optionally, in some other embodiments, in order to cope with the sudden decline in the performance of the business system or the gradual deterioration of the network environment during the business data query process, it is necessary to stop the business system to maintain the network. When the number of zipper tables to be processed is small (such as less than the table number threshold) but the number of second time intervals is large (such as greater than the interval number threshold), for each second time interval in the M second time intervals, the business data in the second time interval is queried from the N zipper tables to be processed. That is, the business data of all zipper tables to be processed corresponding to a second time interval is queried first. In this way, in an emergency, the query task example of more second time intervals can be completed as much as possible in the same time, there are two zipper tables to be processed, table 1 and table 2, and there are three second time intervals, interval 1, interval 2, and interval 3. First, for interval 1, query the business data corresponding to interval 1 from table 1 and table 2, then for interval 2, query the business data corresponding to interval 2 from table 1 and table 2, then for interval 3, query the business data corresponding to interval 2 from table 1 and table 2, then for interval 3, and so on. If an emergency occurs when querying table 1 for interval 3, that is, the business system stops running and the query is interrupted, the query tasks of the two intervals have been completed. Even if the data cannot be restored due to the emergency, the data loss can be reduced, making the queried data available.

[0095] It should be noted that Figure 3C In the embodiment of the present application, only one of the steps 1031A and 1032A shown will be executed, that is, only step 1031A or only step 1032A is executed, and there is no need to execute both steps. There is a logical "or" relationship between step 1031A and step 1032A.

[0096] In some embodiments, see Figure 3D , Figure 3C The step 1032A shown in the figure can be implemented by executing the following steps 10321A to 10322A for each of the N zipper lists to be processed, as described in detail below.

[0097] In step 10321A, a target first time interval including a second time interval is obtained from the first time interval of the zipper list to be processed.

[0098] First, here, the target first time interval including the second time interval can be obtained from the first time interval of the zipper list to be processed. Because each second time interval must be a subset of one of the first time intervals of the N zipper lists to be processed. In this way, for each second time interval, a target time interval associated with the second time interval can be found from the first time interval of the N zipper lists to be processed, that is, the second time interval is a subset of this associated first time interval. Based on this, for each second time interval, the target first time interval including the current second time interval can be determined from the first time interval of the N zipper lists to be processed, that is, the target first time interval associated with the second time interval is determined. Among them, the association condition satisfied by the association of the second time interval with the first time interval is that the start time node in the second time interval is not less than the start time node of the first time interval, and the end time node in the second time interval is not greater than the end time node of the second time interval.

[0099] According to this association condition, for N zipper lists to be processed, a target first time interval associated with a second time interval can be determined from the first time interval of each zipper list to be processed, and M target first time intervals can be obtained from M second time intervals.

[0100] For example, the first time intervals of the zipper table 1 to be processed are "May 1 to May 3, May 3 to May 6", and the first time intervals of the zipper table 2 to be processed are "May 2 to May 6, May 6 to May 12". For a second time interval "May 5 to May 10", it is determined from the zipper table 1 to be processed and the zipper table 2 to be processed that the second time interval "May 5 to May 10" is a subset of the first time interval "May 6 to May 12" in the zipper table 2 to be processed, that is, the first time interval "May 6 to May 12" includes the second time interval "May 5 to May 10", and the first time interval "May 6 to May 12" in the zipper table 2 to be processed is used as the target first time interval of the second time interval.

[0101] In step 10322A, the business data in the target first time interval is queried from the zipper table to be processed as the business data in the second time interval.

[0102] Following the above embodiment, for each second time interval, after the target first time interval including the second time interval is queried from the first time intervals of the plurality of zipper tables to be processed, in order to keep the zipper data of the zipper table to be processed synchronized before and after the merge, the business data will not be lost. Next, the business data in the target first time interval is queried from the zipper table to be processed as the business data of the second time interval, that is, for each second time interval, the business data corresponding to the target first time interval queried from the zipper table to be processed is directly used as the business data of the second time interval, so that the synchronization of the business data can be ensured.

[0103] In some embodiments, see Figure 3E , Figure 3D Step 10322A shown in can be implemented by following steps 103221A to 103222A, which are described in detail below.

[0104] In step 103221A, if no business data in the target first time interval is found from the zipper table to be processed, a null value is used as the business data in the second time interval.

[0105] In some embodiments, when querying business data from the target first time interval of the zipper table to be processed according to the second time interval, if the business data in the target first time interval is not queried from the zipper table to be processed, a null value is used as the business data in the second time interval. Specifically, for each second time interval, there will be a situation where the business data in the target first time interval is not queried from the zipper table to be processed. This means that there are null values ​​in the business data of the zipper data in the zipper table to be processed, that is, in a certain first time interval of the zipper table to be processed, the business data record may be cleared due to the user's cancellation of the handled business, so that the business data value of the zipper data in the corresponding zipper table is "null", that is, a null value. At this time, if the first time interval where the null value is located is used as the target first time interval through the query of the second time interval, the null value is used as the business data corresponding to the second time interval.

[0106] For example, a user registered a business account at the beginning of the month (May 1st), but cancelled the business account during the period of "May 6th to May 9th", and then re-registered the business account on May 10th and used it until the end of the month (May 31st). Therefore, the zipper data of the zipper table recording the user's business account balance is "account balance 100: May 1st to May 5th, null: May 6th to May 9th, account balance 200: May 10th to May 31st". For a second time interval "May 6th to May 8th", the first time interval "May 6th to May 9th" is queried from the account balance zipper table as the target first time interval, but the business data corresponding to the target first time interval is "null", so "null" is used as the business data of the second time interval "May 6th to May 8th".

[0107] In step 103222A, if the business data in the target first time interval is found from the zipper table to be processed, the business data in the target first time interval is used as the business data in the second time interval.

[0108] In some embodiments, when querying business data from the target first time interval of the zipper table to be processed according to the second time interval, if the business data in the target first time interval is queried from the zipper table to be processed, the business data in the target first time interval will be used as the business data for the second time interval. Specifically, for each second time interval, when the business data in the target first time interval is queried from the zipper table to be processed, the business data in the target first time interval will be directly used as the business data for the second time interval. It should be noted here that according to the second time interval, a corresponding target first time interval can be determined in each zipper table to be processed, that is, the corresponding business data can be queried. In the case of N zipper tables to be processed, N business data can be queried in the N zipper tables to be processed according to the second time interval.

[0109] For example, the zipper data of the zipper table 1 to be processed include "account balance 100: May 2 to May 6, account balance 200: May 6 to May 12". For a second time interval "May 5-May 10", "May 6 to May 12" is determined as the target first time interval from the zipper table 2 to be processed. Then the business data "account balance 200" corresponding to "May 6 to May 12" is used as the business data of the second time interval "May 5-May 10". Correspondingly, the zipper data of the pending zipper table 2 includes "credit line 500: May 2 to May 3, credit line 300: May 4 to May 10". For the second time interval "May 5-May 10", "May 4 to May 10" is determined as the target first time interval from the pending zipper table 2, and the corresponding business data "credit line 300" of "May 4 to May 10" is used as the business data of the second time interval "May 5-May 10". In this way, the two business data "account balance 200" and "credit line 300" are queried and obtained from the pending zipper table 1 and the pending zipper table 2 respectively as the business data of the second time interval "May 5-May 10". Similarly, the corresponding business data can also be queried for the pending zipper table 3. In this way, the three business data queried from the three pending zipper tables are all used as the business data of the second time interval.

[0110] In some embodiments, Figure 3E Step 103221A and step 103222A shown in the figure are executed in parallel, and there is no time sequence in the execution order. In order to facilitate the description of the step flow in the embodiment of the present application, there is a sequence.

[0111] Through the embodiment of the present application, the business data in the first time interval of N zipper tables to be processed are synchronized to the second time interval by using the second time interval, so that each second time interval can query and obtain N business data. And in this process, according to the amount and number of business data in the zipper table to be processed, different methods can be adjusted to query business data, thereby improving the efficiency of data query. Therefore, the synchronization of business data is completed only by finding the association between the first time interval and the second time interval, without the need for the zipper table to be processed to perform multiple new chain operations, and then compare the time intervals one by one, which simplifies the process of merging zipper tables and reduces the computing cost.

[0112] In some embodiments, see Figure 3F , Figure 3A In step 103 shown, "integrating the business data of the second time interval to obtain a merged zipper table of the zipper tables to be processed" can be implemented by following steps 1031B to 1033B, which are described in detail below.

[0113] In step 1031B, multiple fields of the N to-be-processed zipper lists are sorted to obtain a field sequence.

[0114] In some embodiments, for each second time interval, after querying and obtaining N business data from N zipper tables to be processed, it is necessary to integrate the N business data of each second time interval, and here the multiple fields of the N zipper tables to be processed are sorted to obtain the corresponding field sequence. That is, the N business fields of the N zipper tables to be processed are sorted. Since each business data is data corresponding to a field in the zipper table to be processed, the business data recorded in the zipper table to be processed all belong to the same field, and the business data recorded in different zipper tables to be processed have different fields, which are used to characterize the business data of the special business stored in the zipper table to be processed. Therefore, in the embodiment of the present application, the multiple fields of the N zipper tables to be processed are sorted, wherein the sorting order is not limited, and can be sorted arbitrarily or specified according to business needs, thereby determining the field arrangement order of each row of zipper data in the merged zipper table, that is, determining the order of the fields corresponding to each column of data in the merged zipper table, and obtaining the corresponding field sequence.

[0115] For example, the zipper table 1 to be processed is the account balance zipper table, and the business field of the corresponding business data is "account balance", the zipper table 2 to be processed is the credit limit zipper table, and the business field of the corresponding business data is "credit limit", and the zipper table 3 to be processed is the business type zipper table, and the business field of the corresponding business data is "business type". At this time, the fields of these business data are sorted by any type of field to obtain a field sequence, such as "account balance, business type, credit limit". It can be determined that the business data field of the first column in the merged zipper table is "account balance", the business data field of the second column is "business type", and the business data field of the third column is "credit limit". If there are more zipper tables to be processed, the same applies.

[0116] In step 1032B, for each of the M second time intervals, the business data in the second time interval is first concatenated based on the field sequence to obtain row data of the merged zipper table in the second time interval.

[0117] After sorting each business data field in the N to-be-processed zipper tables to obtain a field sequence, next, for each of the M second time intervals, the business data in the second time interval is first spliced ​​based on the field sequence to obtain the row data of the merged zipper table in the second time interval. Specifically, for each of the M second time intervals, the business data in the second time interval is horizontally spliced ​​according to the order of the field sequence to obtain the row data of the merged zipper table in the second time interval. Thus, multiple business data with a field sequence order in the second time interval of the merged zipper table are obtained, and these multiple business data with a field sequence order are the row data of the second time interval. Similarly, the row data of the merged zipper table in each second time interval can be determined.

[0118] For example, for a second time interval "May 6 to May 8", the business data corresponding to the three fields queried from the three pending zipper tables are "account balance: 200, credit limit: 300, business type: primary", and then the three business data are first concatenated in the order of the field sequence "account balance, business type, credit limit" to obtain "200-primary-300", which is the row data of the merged zipper table in the second time interval "May 6 to May 8". For another second time interval "May 13 to May 19", the business data corresponding to the three fields queried from the three pending zipper tables are "credit limit: 100, business type: advanced, account balance: 400", and then they are also concatenated in the order of the field sequence "account balance, business type, credit limit" to obtain "400-advanced-100", which is the row data of the merged zipper table in the second time interval "May 13 to May 19". Similarly, the row data of the merged zipper table in each second time interval can be determined.

[0119] In step 1033B, the row data of the merged zipper table in the plurality of second time intervals are second-joined according to the time sequence of the second time intervals to obtain a merged zipper table of N zipper tables to be processed.

[0120] Continuing with the above embodiment, after determining the row data of the merged zipper table in each second time interval, the row data of the merged zipper table in multiple second time intervals are subjected to a second splicing according to the time sequence of the second time intervals, thereby obtaining a merged zipper table of N zipper tables to be processed. That is, the row data of the merged zipper table in each second time interval are vertically spliced ​​according to the time sequence between the second time intervals, thereby obtaining a merged zipper table of N zipper tables to be processed. Considering that the merged zipper table is essentially a zipper table, the zipper data of each row are spliced ​​in sequence according to the time sequence of the time intervals, and there are no gaps or repetitions in the time intervals. Moreover, the business data fields of each row data of the merged zipper table are spliced ​​according to the same field sequence, so that when performing vertical splicing, the business data in the field column where the row data corresponding to each second time interval is located will not be misplaced.

[0121] For example, the row data corresponding to the second time interval "May 6 to May 8" in the merged zipper table is "200-primary-300", the row data in the second time interval "May 13 to May 19" is "400-advanced-100", and the row data in the second time interval "May 9 to May 12" is "300-intermediate-200". Finally, according to the chronological order of the three second time intervals, the three row data are vertically spliced, that is, "200-primary-300, 300-intermediate-200, 400-advanced-100".

[0122] Therefore, through the embodiment of the present application, multiple business data of each second time interval are integrated, the column data field and each row data of the merged zipper table are determined in sequence, and then each row data is vertically spliced ​​directly according to the sequence of the second time interval, without comparing and arranging the business data of each second time interval one by one, thereby improving the efficiency of the merge. Finally, a large zipper table integrating the business data of the zipper table to be processed is obtained as the merged zipper table.

[0123] In some embodiments, although the data processing method of the zipper table provided in the embodiment of the present application merges N to-be-processed zipper tables into one large zipper table as the final merged zipper table. However, since the business systems may be separated, each business system may only process the special business of its own business type independently to meet different business needs. In this way, each business system will only generate zipper data of the corresponding business zipper table to record the business data changes of the corresponding business type. Therefore, when each to-be-processed zipper table generates zipper data and performs a new chain opening operation, the corresponding business data needs to be synchronized to the merged zipper table. Therefore, in the business scenario, it is necessary to ensure in real time that the business data of the to-be-processed zipper table and the merged zipper table of each special business are synchronized.

[0124] For the pending zipper table of each business type (i.e., the original business zipper table), if a new chain opening operation is performed on the pending zipper table, the newly added business data of the pending zipper table when the new chain opening operation is performed, as well as the current time when the new chain opening operation is performed, are obtained. The corresponding merged zipper table will also perform a new chain opening operation, adding a second time interval. Considering that the second end time node of the last second time interval of the merged zipper table must be a preset time (such as December 31, 9999), after the second time interval is added, the second end time node of the last second time interval in the merged zipper table is modified from the preset time to the current time. Next, the current time is used as the second start time node of the newly added second time interval of the merged zipper table, and the preset time is used as the second end time node of the newly added second time interval. In this way, the second start time node and the second end time node of the newly added second time interval of the merged zipper table are determined.

[0125] After determining the newly added second time interval of the merged zipper table, the newly added business data is used as the business data of the merged zipper table in the newly added second time interval. That is, the newly added business data of the pending zipper table when executing the new chain opening operation is used as the business data of the merged zipper table in the newly added second time interval, thereby completing the business data synchronization between the pending zipper table and the merged zipper table.

[0126] It should be noted that there are multiple business data fields in the merged zipper table, which come from different pending zipper tables. When a pending zipper table adds new business data to perform a new chain operation, the business data of the corresponding business data fields in other pending zipper tables may not have changed at the current moment. Therefore, in the merged zipper table, the business data corresponding to the business field that has not changed in the last second time interval is directly retained and synchronized to the data field corresponding to the newly added second time interval.

[0127] For example, the business data corresponding to the last second time interval "May 19, 2000 to December 31, 9999" in the merged zipper table is "credit limit: 100, business type: advanced, account balance: 400". On May 21, 2000, a new business data "account balance: 0" was added to one of the pending zipper tables, namely the account balance zipper table. When executing the new chain operation, the preset time "December 31, 9999" of the last second time interval "May 19, 2000 to December 31, 9999" in the merged zipper table is changed to the current time "May 21, 2000", so that the last second time interval is "May 19, 2000 to May 21, 2000". Then the current time "May 21, 2000" is used as the second starting time node of the newly added second time interval, and the preset time "December 31, 9999" is used as the second ending time node of the newly added second time interval, so the newly added second time interval of the merged zipper table is "May 21, 2000 to December 31, 9999". Next, the business data of the account balance field in the corresponding row data also needs to be changed, while the business data of the remaining fields that have not changed are directly retained for synchronization, so the business data corresponding to the last second time interval in the merged zipper table "Credit line: 100, business type: advanced, account balance: 400" is changed to "Credit line: 100, business type: advanced, account balance: 0" as the business data of the newly added second time interval "May 21, 2000 to December 31, 9999", thereby completing the data synchronization of the pending zipper table and the merged zipper table.

[0128] In some embodiments, it is generally necessary to integrate the business data of multiple business systems for comparative analysis. When the business data changes, you can directly perform a new chain operation in the merged zipper table to record the changes in the business data of each business system. In some business scenarios, it is necessary to extract the business data change records of the merged zipper table and distribute them to each business system for verification or validation. At this time, it is necessary to generate zipper data in the merged zipper table and perform a new chain operation, and synchronize the corresponding business data to the original zipper table corresponding to each business system, that is, synchronize the business data of the merged zipper table to the corresponding zipper table to be processed, and keep the business data of the zipper table to be processed and the merged zipper table synchronized.

[0129] For the merged zipper table, if a new chain operation is performed on the merged zipper table, the newly added business data of the merged zipper table when the new chain operation is performed, and the current time when the new chain operation is performed are obtained. Then, according to the newly added business data, the pending zipper table to which the newly added business data belongs is determined. A first time interval is added to the determined pending zipper table. Similarly, considering that the end time node of the last first time interval of the pending zipper table must be a preset time (such as December 31, 9999), after the first time interval is added, the first end time node of the last first time interval in the pending zipper table is modified from the preset time to the current time. Next, the current time is used as the first starting time node of the newly added first time interval of the pending zipper table, and the preset time is used as the first end time node of the newly added first time interval. After determining the first starting time node and the first end time node of the newly added first time interval of the pending zipper table, the newly added business data when the merged zipper table performs the new chain operation is used as the business data of the pending zipper table in the newly added first time interval, thereby completing the synchronization of business data.

[0130] For example, the business data corresponding to the current last first time interval "May 19, 2000 to December 31, 9999" in a certain account balance zipper table (i.e., the pending zipper table) is "account balance: 400". On May 21, 2000, a new business data "account balance 0" is added to the account balance business field of the merged zipper table. When the merged zipper table performs a new chain operation, it is first determined that it is an account balance business field based on the business data "account balance 0", and the corresponding pending zipper table is the account balance zipper table. The preset time "December 31, 9999" of the current last first time interval "May 19, 2000 to December 31, 9999" in the account balance zipper table is modified to the current time "May 21, 2000". In this way, the current last first time interval in the account balance zipper table is "May 19, 2000 to May 21, 2000". Then the current time "May 21, 2000" is used as the first starting time node of the newly added first time interval, and the preset time "December 31, 9999" is used as the first ending time node of the newly added first time interval, so the newly added first time interval of the account balance zipper table is "May 21, 2000 to December 31, 9999". Next, the business data "Account Balance: 0" in the merged zipper table that performs the new chain operation is used as the business data of the newly added first time interval "May 21, 2000 to December 31, 9999" of the account balance zipper table, thereby completing the data synchronization between the merged zipper table and the zipper table to be processed.

[0131] Through the embodiment of the present application, the first time intervals in the N zipper tables to be processed are integrated by sorting and deduplicating the time nodes to obtain M second time intervals as the time intervals of the merged zipper table. Next, the business data in the zipper table to be processed is synchronized using the second time interval, so that N business data can be queried in each second time interval. Finally, the N business data in each second time interval are integrated, and the column data fields and each row data of the merged zipper table are determined in turn, and then each row data is vertically spliced, and finally a large zipper table that integrates the business data of the zipper table to be processed is obtained as the merged zipper table. In addition, on the basis of the merged zipper table, the synchronization of business data of the zipper table to be processed and the merged zipper table in certain business scenarios is also realized. Therefore, when using N unprocessed zipper lists to create a merged zipper list for data synchronization, there is no need for the unprocessed zipper list to perform multiple new chain opening operations to generate a new first time interval, and then compare it with the existing first time interval one by one to determine the changes in business data. The time interval integration of the unprocessed zipper list and the synchronization of business data are completed, thereby simplifying the production process of zipper table merging, reducing the computational cost and time cost of the merging, and improving the merging efficiency.

[0132] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0133] In some business scenarios, different storage strategies are required for massive amounts of business data, such as customer data, transaction data, and status data. Usually, data is stored in the form of full tables. Compared with full tables, zipper tables can retain equivalent information while consuming less space. Therefore, zipper tables, as a table commonly used in data warehouses to record information about changes in things, are widely used in applications in various industries.

[0134] See also Figure 4 , Figure 4 1 is a diagram of the manufacturing process of the zipper table provided in the embodiment of the present application. Figure 4 As shown, the zipper table is initialized first, using the user information of the section with the earliest section time (i.e. 2020-01-01) in the entire table as the first row of data. The start time field (start_dt) is set to the zipper start time, i.e. 2020-01-01. Accordingly, the end time field (end_dt) is set to the zipper end time, and filling it with 9999-12-31 means that the row of data is in effect at the latest point in time.

[0135] Next is the closed-chain operation. On each batch date (i.e., the period when the system periodically performs data processing tasks), the user information on the batch date is compared with the effective data that already exists in the initial zipper table (i.e., the data of end_dt = "9999-12-31"). If the information changes, the zipper end time end_dt is set to the batch date, thus obtaining the "20200102 zipper table". For example, on the batch date, i.e., the 2020-01-02 section, the user's phone number changes from "111-111" to "111-222", so the end_dt in the zipper table is changed from 9999-12-31 to 2020-01-02.

[0136] Opening a new chain is the most common operation of a zipper table. There are two situations that will cause a zipper table to open a new chain. The first is that if a data chain is closed, a new chain needs to be added as valid data. Figure 4 The "20200102 Zipper Table" shown above has a new phone number 111-222, the zipper start time start_dt is 2020-01-02, and the zipper end time end_dt is 9999-12-31. The second is if a new user is added, such as Figure 4 The initialization information of "zipper table initialization" shown adds a valid zipper data of the user, that is, a new zipper table is created.

[0137] In the related art, the efficiency of making zipper tables for historical data is low, which is specifically reflected in the large number of date sections. For example, there is a full year of data (including 366 section dates from 2020-01-01 to 2020-12-31). When making a zipper table, a new chain operation needs to be performed for each batch date, and then compared with each section once. For example, if the 2020-01-01 information is used for initialization, the data generated every day thereafter must be compared with the zipper table, which requires 365 comparisons, which is time-consuming and inefficient. In addition, in the actual application process, different users may have zipper data with the same granularity (such as primary key granularity, the same account identifier (id)) but different dimensions or indicators (different indicators on different zipper tables, such as account balance, overdue date, credit amount) for their respective special applications. See also Figure 5 , Figure 5 This is a zipper representation of different service indicators of the same user provided in the embodiment of the present application, such as Figure 5 As shown in the figure, there are three zipper tables with different field information at the account granularity, among which a is the zipper for recording the account balance, b is the zipper for the account overdue date, and c is the zipper for the account credit limit. However, when a new application requires multiple zipper table data fields, you can refer to Figure 6 , Figure 6It is a zipper representation of multiple data fields provided in the embodiment of the present application, such as Figure 6 As shown in the figure, in the same zipper table, for the user with the same account ID A, the zipper data of the three fields of account balance, overdue date, and credit amount need to be recorded. Figure 5 The three zipper tables shown in the figure are merged, and the usual method is to re-make a new zipper from scratch, that is, to continuously perform the new chain opening operation, which is time-consuming and inefficient.

[0138] Based on the above scenario, the present application embodiment provides a zipper table merging method. The flowchart of the method can be found in Figure 7 , Figure 7 It is a schematic diagram of the zipper table merging method provided in an embodiment of the present application, which is mainly divided into two processing modules, namely an interval determination module and a field integration module, wherein the interval determination module is used to determine the start date and end date of different zipper data in the merged zipper table, and the date interval is a left-closed and right-open period, expressed as [start_dt, end_dt). After the detailed interval of each zipper data is determined, the interval value obtained for each zipper data can be associated with the original data through the field integration module to obtain the value of each field of the zipper data in the limited interval, and the merged zipper result is finally obtained after processing by the two modules. The efficiency of zipper table merging can be improved in big data scenarios. The processing process of these two modules will be specifically introduced below.

[0139] Normally, in the process of making a zipper table, the field value of each data is first determined (the value of the corresponding field column in the zipper table, such as the account balance field column, the value is 500). If the field value changes, a new zipper row needs to be added, and then the time interval (start_dt / end_dt) of the zipper row is determined. Based on the existing zipper information, the embodiment of the present application first determines the interval range of each zipper data by projection method, and then associates the obtained interval with the original zipper tables to obtain the target zipper data. For details, please refer to Figure 8 , Figure 8 is a schematic diagram of a zipper data time interval using a coordinate axis provided in an embodiment of the present application. Figure 8 As shown, for each zipper table to be merged, the time node of each row of field data in each zipper table is projected (mapped) onto a one-dimensional time axis. Each node on the time axis represents a date ( Figure 8 For simplicity, only the date is retained, and the year and month are omitted). Different time nodes can be obtained. The time range between each node is the time interval in which the zipper data record is effective. From a visual perspective, the number axis nodes of different zipper tables can be projected onto the same time axis. See Fig. 9 , Fig. 9is a schematic diagram of the node projection result provided by the embodiment of the present application, such as Fig. 9 As shown, Figure 8 The nodes on the time axis corresponding to the three zipper tables in the zipper table are integrated into a new time axis. The nodes on the new time axis are the nodes of the merged zipper table (if there are duplicate nodes, they will be deduplicated, which will be explained in detail below), and the interval between the nodes on the time axis is the time interval of each zipper data record in the zipper table.

[0140] The specific steps of projecting (mapping) the time node of each row of field data in the zipper table to the one-dimensional time axis are as follows:

[0141] (1) The start_dt information of the zipper table data before the merger is integrated into a complete set. This method can be applied to the integration of any number of tables (that is, it can be multiple zipper tables with the same primary key granularity).

[0142] (2) De-duplicate the start_dt data obtained after integration in step (1), and map the resulting set of de-duplicate results onto the time axis, which is the node of the target zipper table under projection. This is because multiple zipper tables may have the same start_dt data, and mapping such data onto the time axis will result in the same node, which is of little significance. Therefore, in the zipper data merging process, the embodiment of the present application deletes the duplicate start_dt data, that is, only retains one corresponding start_dt data, so that there are no duplicate nodes in the target zipper table under projection.

[0143] In some embodiments, after the projection is completed to obtain the node set of the merged zipper table, considering that the time intervals of the field records of the zipper table are in order, the nodes in the node set are sorted in ascending order, that is, the nodes with earlier time are placed at the head of the sequence. Fig.10 , Fig.10 is a schematic diagram of determining interval results from node results provided by an embodiment of the present application, such as Fig.10 As shown, since the node result of the merged zipper table only has start_dt data at this time, after sorting the start_dt data in ascending order, it is also necessary to determine the end_dt data of each node (i.e., each start_dt data) to obtain the next node value of each zipper data, thereby obtaining the entire time interval of each zipper data. In the embodiment of the present application, based on the original three zipper table data, the load function is called to obtain the next node value (i.e., the corresponding end_dt data) of each node (i.e., each start_dt data). After determining each node value of each node, the following can be obtained: Fig.10 The interval results are shown.

[0144] Next, we will introduce the specific processing of the field integration module. Since the merged target zipper table is the integration of field information of multiple zipper tables, each two nodes (i.e. each time interval) of the target zipper table on the time axis must be a subset of each time interval of the original zipper table. This can also be understood as an increase in information fields and a decrease in granularity. Fig.11 , Fig.11 is a time interval comparison diagram of the zipper table provided in the embodiment of the present application, such as Fig.11 As shown, in the new target interval, the nodes on the time axis are extended and represented by dotted lines. It can be seen that each target interval range corresponds to a part of the original interval of the original three zipper tables, that is, a subset. That is, the time interval between each node on the time axis of the target zipper table can be traced back to the original interval of the corresponding original zipper table.

[0145] See also Fig.12 , Fig.12 It is a schematic diagram of the interval field information of the target zipper table provided in an embodiment of the present application. The start date (start_dt) and the end date end_dt of the target zipper table time interval are obtained through the interval determination module. To facilitate the distinction from the original zipper table time interval, the embodiment of the present application records the start date of the target zipper table time interval as start_dt_target, and the end date of the target zipper table time interval as end_dt_target, so the time interval can be recorded as [start_dt_target, end_dt_target].

[0146] It should be noted that in Fig.12 In the example, the column where the serial number is located is not necessary and has no influence or effect on the embodiment of the present application. It is only used to intuitively display the number of rows of zipper table data after the merger, as a comparison with the difference in the number of rows of zipper table data before the merger. Fig.12 Each row of data in represents a zipper data in the target zipper table. Next, the time interval of the target zipper table is associated with the time interval of the original three zipper tables (as mentioned above, the time interval between each node on the time axis of the target zipper table can be traced back to the original interval of the corresponding zipper table). Fig.11The three zipper tables shown are the account balance zipper table, the overdue date zipper table, and the credit amount zipper table. In the embodiment of the present application, the start date and end date of each zipper data in the account balance zipper table are defined as [start_dt_a, end_dt_a], the start date and end date of each zipper data in the overdue date zipper table are defined as [start_dt_b, end_dt_b], and the start date and end date of each zipper data in the credit amount zipper table are defined as [start_dt_c, end_dt_c]. The association condition requires that the time interval of the current target zipper table be included in the time interval of the original zipper table, that is, the time interval data cannot be omitted. For the association of the account balance zipper table, the corresponding logical relationship must satisfy "start_dt_target>=start_dt_a and end_dt_target<=end_dt_a". Similarly, for the association of the overdue date zipper table, the corresponding logical relationship must satisfy "start_dt_target>=start_dt_b and end_dt_target<=end_dt_b". And for the association of the credit amount zipper table, the corresponding logical relationship must satisfy "start_dt_target>=start_dt_c and end_dt_target<=end_dt_c".

[0147] In some embodiments, the purpose of associating the time interval of the target zipper table with the time interval of the original zipper table is to compare the time interval of each row of zipper data in the target zipper table with the time interval of each row of zipper data in the original three zipper tables, find the field information of the unique corresponding time interval from the original zipper table, and the result of the association is the three field information in the original three zipper tables corresponding to the row of data in the target zipper table.

[0148] Among them, the process of searching for comparison can be started from the first row of the original zipper table and traversed in sequence by the number of rows, because the time interval of one and only one row of data in the original zipper table matches the time interval of the row of data in the target zipper table, that is, it satisfies the logical relationship of the association, so after the match is successfully associated during the traversal process, the traversal can be ended.

[0149] For example, see Fig.13 , Fig.13 is a schematic diagram of the zipper table time interval comparison association provided by the embodiment of the present application, such as Fig.13As shown, to calculate the account balance, overdue date, credit amount and other field information corresponding to the zipper data in the fifth row of the target zipper table, the time interval of the zipper data in the fifth row of the target zipper table (i.e., 2020-06-15, 2020-06-17) is used to compare with the zipper data interval of each row in the account balance zipper table (i.e., [start_dt_a, end_dt_a]), and then the zipper data interval of each row in the account balance zipper table is traversed. When traversing to the third row, it is determined that the time interval of the zipper data in the third row of the account balance zipper table (i.e., 2020-06-15, 2020-06-15) satisfies the associated logical condition (i.e., the logical relationship above), so the zipper data in the third row of the account balance zipper table (account balance: 3000) is used as the information of the account balance field of the target zipper table.

[0150] Similarly, if Fig.13 As shown, for the account overdue date zipper table, the time interval of the zipper data in the fifth row of the target zipper table (i.e., 2020-06-15, 2020-06-17) is compared with the zipper data interval of each row in the account overdue date zipper table (i.e., [start_dt_b, end_dt_b]), and then the zipper data interval of each row in the account overdue date zipper table is traversed. When traversing to the third row, it is determined that the time interval of the second row of zipper data in the account balance zipper table (i.e., 2020-06-15, 2020-06-22) meets the associated logical condition, so the second row of zipper data in the account overdue date zipper table (overdue date: 2020-06-15) is used as the information of the account overdue date field of the target zipper table. For the account credit limit zipper table, the time interval of the zipper data in the fifth row of the target zipper table (i.e., 2020-06-15, 2020-06-17) is compared with the zipper data interval of each row in the account credit limit zipper table (i.e., [start_dt_c, end_dt_c]), and then the zipper data interval of each row in the account overdue date zipper table is traversed. When traversing to the third row, it is determined that the time interval of the zipper data in the third row of the account credit limit zipper table (i.e., 2020-06-10, 2020-06-17) meets the associated logical conditions, so the zipper data in the third row of the account credit limit zipper table (credit limit: 8000) is used as the information of the account credit limit field of the target zipper table.

[0151] After the above processing, we can finally get Figure 6 As shown in the target zipper table, each row of zipper data in the target zipper table can be compared with the corresponding three original zipper tables to obtain the information of the corresponding field, such as Figure 6 The account balance data column, overdue date data column, and credit amount data column of the merged zipper table are integrated, and the field information of the data list is integrated to obtain the final merged zipper table.

[0152] Through the embodiment of the present application, a method for merging zipper tables is provided, which makes full use of the field information in the existing original zipper table. When the primary key of the original zipper table is the same (i.e., the same account), based on the time interval of the zipper data in the original zipper table, the time interval after the target zipper table is merged is calculated in advance using the projection method. Then, using the merged time interval, the information of the target field is found corresponding to the time interval of the zipper data in the original zipper table. Finally, based on the information of the target field and the merged time interval, the final merged zipper table is constructed. This merging of zipper tables avoids the process of starting to re-create a new zipper table, and there is no need to perform multiple new chain opening operations and then start the comparative calculation from scratch, which saves time and calculation costs and improves the efficiency of merging zipper tables.

[0153] The following further describes the exemplary structure of the zipper table data processing device 453 provided in the embodiment of the present application implemented as a software module. In some embodiments, for example Figure 2 As shown, the software modules in the data processing device 453 of the zipper table stored in the memory 450 may include: an acquisition module 4531, which acquires the first time interval of the zipper table to be processed and the business data of the first time interval, wherein the first time interval is the time interval from the first start time node to the first end time node; a determination module 4532, which is used to determine the second time interval based on the first start time node and the first end time node; an integration module 4533, which is used to query the target business data in the second time interval from the zipper table to be processed, and integrate the target business data to obtain a merged zipper table of the zipper table to be processed.

[0154] In some embodiments, the determination module 4532 is also used to determine P time nodes based on the first starting time node and the first ending time node; generate a second time interval based on each time node in the P time nodes to obtain M second time intervals, where the value of P is the same as M; wherein, generating the second time interval based on the time node includes: taking the time node as the second starting time node; when the time node is not the last time node among the P time nodes, obtaining the adjacent time nodes in the P time nodes that are adjacent to the time node and located after the time node, and taking the adjacent time nodes as the second ending time node; when the time node is the last time node among the P time nodes, taking the preset time as the second ending time node; taking the time interval formed by the second starting time node and the second ending time node as the second time interval.

[0155] In some embodiments, the integration module 4533 is also used to query the business data in M ​​second time intervals from the zipper list to be processed for each of the N zipper lists to be processed; or to query the business data in the second time interval from the N zipper lists to be processed for each of the M second time intervals.

[0156] In some embodiments, the integration module 4533 is also used to obtain, for each of the N zipper lists to be processed, a target first time interval including a second time interval from the first time interval of the zipper list to be processed; and query the business data in the target first time interval from the zipper list to be processed as the business data of the second time interval.

[0157] In some embodiments, the integration module 4533 is also used to use a null value as the business data for the second time interval if no business data in the target first time interval is found from the zipper table to be processed; and to use the business data in the target first time interval as the business data for the second time interval if business data in the target first time interval is found from the zipper table to be processed.

[0158] In some embodiments, the integration module 4533 is also used to sort multiple fields of N zipper tables to be processed to obtain a field sequence; for each second time interval in the M second time intervals, the business data in the second time interval is first spliced ​​based on the field sequence to obtain the row data of the merged zipper table in the second time interval; the row data of the merged zipper table in multiple second time intervals is second spliced ​​according to the time sequence of the second time intervals to obtain a merged zipper table of the N zipper tables to be processed.

[0159] In some embodiments, the integration module 4533 is also used to obtain the newly added business data of the zipper table to be processed when the new chain opening operation is performed on the zipper table to be processed, as well as the current time when the new chain opening operation is performed; modify the second end time node of the last second time interval in the merged zipper table from the preset time to the current time; use the current time as the second start time node of the newly added second time interval of the merged zipper table, and use the preset time as the second end time node of the newly added second time interval; use the newly added business data as the business data of the merged zipper table in the newly added second time interval.

[0160] In some embodiments, the integration module 4533 is also used to obtain the newly added business data of the merged zipper table when the new chain opening operation is performed on the merged zipper table, as well as the current time when the new chain opening operation is performed; determine the zipper table to be processed to which the newly added business data belongs; modify the first end time node of the last first time interval in the zipper table to be processed from the preset time to the current time; use the current time as the first start time node of the newly added first time interval of the zipper table to be processed, and use the preset time as the first end time node of the newly added first time interval; use the newly added business data as the business data of the zipper table to be processed in the newly added first time interval.

[0161] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the data processing method of the zipper table described in the embodiment of the present application.

[0162] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the zipper table data processing method provided in the embodiment of the present application, for example, FIG. 3A to FIG. 3F The data processing method of the zipper table is shown.

[0163] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0164] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0165] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0166] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0167] In summary, through the embodiments of the present application, the first time intervals in the N zipper tables to be processed are integrated by sorting and deduplicating the time nodes to obtain multiple second time intervals as the time intervals of the merged zipper table. Next, the business data in the zipper table to be processed is synchronized using the second time interval, so that multiple business data can be queried in each second time interval. Finally, the multiple business data in each second time interval are integrated, and the column data fields and each row data of the merged zipper table are determined in turn, and then each row data is vertically spliced, and finally a large zipper table that integrates the business data of the zipper table to be processed is obtained as the merged zipper table. In addition, on the basis of the merged zipper table, the synchronization of business data of the zipper table to be processed and the merged zipper table in certain business scenarios is also realized. Therefore, when using N unprocessed zipper lists to create a merged zipper list for data synchronization, there is no need for the unprocessed zipper list to perform multiple new chain opening operations to generate a new first time interval, and then compare it with the existing first time interval one by one. The time interval integration of the unprocessed zipper list and the synchronization of business data are completed, thereby simplifying the production process of zipper table merging, reducing the computational cost and time cost of the merging, and improving the merging efficiency.

[0168] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A data processing method for a zipper table, characterized in that: The method comprises: Acquire a first time interval of the zipper table to be processed and business data of the first time interval, wherein the first time interval includes a time interval between a first start time node and a first end time node; Determining a second time interval based on the first start time node and the first end time node; The target business data in the second time interval is queried from the zipper table to be processed, and the target business data is integrated to obtain a merged zipper table of the zipper table to be processed.

2. The method according to claim 1, characterized in that The second time interval includes M second time intervals, where M is an integer greater than 1, and the determining the second time interval based on the first start time node and the first end time node includes: Determine P time nodes based on the first start time node and the first end time node; Generate a second time interval based on each of the P time nodes to obtain M second time intervals, where the value of P is the same as that of M; The step of generating the second time interval based on the time node includes: Taking the time node as the second starting time node; In the case that the time node is not the last time node among the P time nodes, obtaining an adjacent time node among the P time nodes that is adjacent to the time node and located after the time node, and taking the adjacent time node as the second end time node; When the time node is the last time node among the P time nodes, the preset time is used as the second end time node; The time interval formed by the second start time node and the second end time node is used as the second time interval.

3. The method according to claim 1, characterized in that The to-be-processed zipper table includes N to-be-processed zipper tables, where N is an integer greater than 1, and querying the target business data in the second time interval from the to-be-processed zipper table includes: For each of the N to-be-processed zipper tables, query the service data in the M second time intervals from the to-be-processed zipper table; or For each of the M second time intervals, query the business data in the second time interval from the N to-be-processed zipper tables.

4. The method according to claim 3, characterized in that The querying the business data in the second time interval from the N to-be-processed zipper tables includes: For each of the N to-be-processed zipper lists, acquiring a target first time interval including the second time interval from the first time interval of the to-be-processed zipper list; The business data in the target first time interval is queried from the to-be-processed zipper table as the business data in the second time interval.

5. The method according to claim 4, characterized in that The querying of the service data in the target first time interval from the to-be-processed zipper table as the service data in the second time interval includes: If no business data in the target first time interval is found from the to-be-processed zipper table, a null value is used as the business data in the second time interval; If the business data in the target first time interval is found from the to-be-processed zipper table, the business data in the target first time interval is used as the business data in the second time interval.

6. The method according to claim 5, characterized in that The business data recorded in the to-be-processed zipper tables all belong to the same field, and the business data recorded in different to-be-processed zipper tables have different fields; The integrating the business data of the second time interval to obtain a merged zipper table of the to-be-processed zipper table includes: Sorting multiple fields of the N to-be-processed zipper lists to obtain a field sequence; For each second time interval of the M second time intervals, performing a first concatenation on the business data in the second time interval based on the field sequence to obtain row data of the merged zipper table in the second time interval; The row data of the merged zipper table in the plurality of second time intervals are second-joined in the time sequence of the second time intervals to obtain a merged zipper table of the N zipper tables to be processed.

7. The method according to claim 1, characterized in that The method further comprises: If a new chain opening operation is performed on the zipper list to be processed, then newly added business data of the zipper list to be processed when the new chain opening operation is performed, and the current time of performing the new chain opening operation are obtained; Modify the second end time node of the last second time interval in the merged zipper table from the preset time to the current time; Using the current time as the second starting time node of the newly added second time interval of the merged zipper table, and using the preset time as the second ending time node of the newly added second time interval; The newly added business data is used as the business data of the merged zipper table in the newly added second time interval.

8. The method according to claim 1, characterized in that The method further comprises: If a new chain opening operation is performed on the merged zipper table, newly added business data of the merged zipper table when the new chain opening operation is performed, and the current time when the new chain opening operation is performed are obtained; Determine the zipper table to be processed to which the newly added business data belongs; Modify the first end time node of the last first time interval in the to-be-processed zipper list from the preset time to the current time; Using the current time as the first starting time node of the newly added first time interval of the to-be-processed zipper table, and using the preset time as the first ending time node of the newly added first time interval; The newly added business data is used as the business data of the to-be-processed zipper table in the newly added first time interval.

9. A data processing device for a zipper table, characterized in that: The device comprises: An acquisition module, used to acquire a first time interval of the zipper table to be processed and business data of the first time interval, wherein the first time interval includes a time interval between a first start time node and a first end time node; A determination module, configured to determine a second time interval based on the first start time node and the first end time node; An integration module is used to query the target business data in the second time interval from the zipper table to be processed, and integrate the target business data to obtain a merged zipper table of the zipper table to be processed.

10. An electronic device, characterized in that: The electronic device comprises: Memory for storing computer executable instructions or computer programs; A processor is used to implement the zipper table data processing method according to any one of claims 1 to 8 when executing the computer executable instructions or computer programs stored in the memory.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer programs are executed by a processor, the zipper table data processing method according to any one of claims 1 to 8 is implemented.

Images